Recent Miriad changes
Moderator: Mark.Wieringa
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Recent Miriad changes
This thread will be updated regularly with changes to Miriad tasks. We recommend that Miriad users visit this thread periodically (or subscribe to receive notification when new posts are made to this thread). You will not be able to post replies in this thread, so if you want to ask further questions about these changes, please do so in a new thread.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Miriad changes: 18 Jan 2012
A large number of changes were comitted last week, primarily to improve large file support (see thread http://atcaforum.atnf.csiro.au/viewtopic.php?f=10&t=92).
The tasks that have better support for large files are:
The task attsys was updated with a bugfix, and a new option 'inverse' was added.
We encourage all users who use these tasks to immediately upgrade.
The tasks that have better support for large files are:
- uvacflag
- imdiff
- uvfstats
- tvclip
- fits
- uvpflag
- invert
- cgspec
- uvglue
- uvmodel
- selfcal
- linmos
- tvflag
- uvlin
- gpscal
- pgflag
The task attsys was updated with a bugfix, and a new option 'inverse' was added.
We encourage all users who use these tasks to immediately upgrade.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Miriad changes: 26 March 2012
A couple of recent changes to report on here.
The first change is to uvsplit to allow for large numbers of files to be created when splitting into small frequency chunks using the maxwidth parameter. It is now possible for uvsplit to produce up to 1024 files - see thread http://atcaforum.atnf.csiro.au/viewtopic.php?f=10&t=106 for more details.
Miriad is now also fully capable of reading the FITS spectral standard. This feature has been progressively implemented by Mark Calabretta and is now ready. His email to the Miriad users list:
The first change is to uvsplit to allow for large numbers of files to be created when splitting into small frequency chunks using the maxwidth parameter. It is now possible for uvsplit to produce up to 1024 files - see thread http://atcaforum.atnf.csiro.au/viewtopic.php?f=10&t=106 for more details.
Miriad is now also fully capable of reading the FITS spectral standard. This feature has been progressively implemented by Mark Calabretta and is now ready. His email to the Miriad users list:
To support this change, Miriad now requires WCSLIB 4.12 or later. If you wish to compile Miriad from source, you can get the latest WCSLIB from ftp://ftp.atnf.csiro.au/pub/software/wcslib/Greetings to Miriad users,
ATNF's Miriad now supports the FITS spectral standard as well as
maintaining backwards compatibility with the older AIPS spectral
convention.
In practical terms, this means that you should be able to read
any FITS image with a standard spectral coordinate header, or
an older AIPS-style header, and have the spectral coordinates
interpreted correctly. That includes IR, optical, UV, X-ray,
and gamma ray spectra that conform to the FITS standard.
Task 'velsw' can also now switch the spectral axis to any of the
new spectral types such as redshift, wavelength, or relativistic
velocity.
If you are not familiar with the FITS spectral standard, here is
a brief summary of the main differences:
1) The Doppler frame is recorded in a separate SPECSYS header
record in FITS - now copied to the 'specsys' header item in
Miriad and listed by 'prthd'. In all cases the spectral
axis is Doppler shifted to the frame specified by specsys.
The old method recorded the Doppler frame as part of
CTYPEi, e.g. 'FREQ-LSR', 'VELO-HEL', etc. and these are
still written by 'invert'.
2) You may see a three-letter code in place of the old Doppler
frame, for example 'VOPT-F2W'. The 'F2W' bit is a non-
linear algorithm code analogous to the codes used for
celestial projections. 'F2W' indicates that the axis is
linear in frequency but is being expressed in terms of a
variable (VOPT) that is linear in wavelength. If the code
is absent, the axis is linear, e.g. 'FREQ'.
While there are dozens of possibilities, the following table
lists some of the new spectral types you may encounter:
Old New
ctype ctype specsys
-------- ------------------
FREQ-OBS FREQ TOPOCENT ...frequency
VELO-HEL VRAD BARYCENT ...radio velocity
FELO-LSR VOPT-F2W LSRK ...optical velocity
n/a ZOPT-F2W (any) ...redshift (radio spectra)
n/a ZOPT (any) ...redshift (optical spectra)
n/a WAVE-F2W (any) ...wavelength (radio spectra)
n/a WAVE (any) ...wavelength (optical spectra)
n/a VELO-F2V (any) ...relativistic velocity
n/a VELO-W2V (any) ...relativistic velocity
n/a ENER (any) ...photon energy (e.g. X-ray)
A linear wavelength axis, 'WAVE' as above, might come from an
optical spectrum or a regridded radio spectrum. Notice that the
old VELO-{OBS,HEL,LSR} is radio velocity, but in all other cases
VELO now indicates relativistic velocity.
You should be aware that, historically, Miriad adopted an
unconventional interpretation for FREQ-HEL and FREQ-LSR. In a
Miriad header and FITS files written by Miriad, these axes are
actually topocentric, not barycentric or LSRK as it might
appear. The help for task 'velsw' explains how this is handled,
and 'impos' also has a new option related to it.
Mark Calabretta
ATNF
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Re: Recent Miriad changes
Recently, the task pgflag has been upgraded to include an automatic flagger based on the 'SumThreshold' algorithm used by the LOFAR AOFlagger. This algorithm is tunable, but even with the default parameters it has been shown to be quite effective at quickly removing RFI at 16cm.
Adding to it's usefulness with automatic flagging, pgflag can now be run in non-interactive mode. See the updated documentation for full details about how to use this task.
The task atrecal can now calculate a Tsys spectrum from the autocorrelations and the new option fsystemp in invert lets you use this in the imaging weights. This will result in a better noise optimised image when there is a large change in Tsys across the observed band.
A new task uvfmeas is now available, and can be used to measure fluxes for point sources (at the phase centre), similar to other tasks uvflux and calred, but is more useful for the wider bandwidths that CABB makes available. The uvfmeas task makes it possible to measure spectral index across the band, and compare the scalar and vector averages via a plot.
A change has been made to atlod to allow for opacity corrections to be computed and applied in smaller subbands across the band.
Bug fixes have been made to the tasks gpaver, uvcat and uvsplit.
Adding to it's usefulness with automatic flagging, pgflag can now be run in non-interactive mode. See the updated documentation for full details about how to use this task.
The task atrecal can now calculate a Tsys spectrum from the autocorrelations and the new option fsystemp in invert lets you use this in the imaging weights. This will result in a better noise optimised image when there is a large change in Tsys across the observed band.
A new task uvfmeas is now available, and can be used to measure fluxes for point sources (at the phase centre), similar to other tasks uvflux and calred, but is more useful for the wider bandwidths that CABB makes available. The uvfmeas task makes it possible to measure spectral index across the band, and compare the scalar and vector averages via a plot.
A change has been made to atlod to allow for opacity corrections to be computed and applied in smaller subbands across the band.
Bug fixes have been made to the tasks gpaver, uvcat and uvsplit.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Re: Recent Miriad changes
Miriad task atlod now flags out continuum channels 9, 17 and 25 if the option 'birdie' is used with a 64 MHz zoom dataset.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Re: Recent Miriad changes
A number of changes have been made since the last update.
A new task uvsfit has been made (by Dave McConnell) that can be used to fit source parameters such as flux, spectral index and curvature, and position using only the uv data files. It requires that you already know the rough position of the sources that you want to fit, and can be used to generate either a model dataset (or datasets if multiple files are supplied to it) or the residual dataset(s) after the fit has been subtracted.
Updates to other tasks (in no particular order):
A new task uvsfit has been made (by Dave McConnell) that can be used to fit source parameters such as flux, spectral index and curvature, and position using only the uv data files. It requires that you already know the rough position of the sources that you want to fit, and can be used to generate either a model dataset (or datasets if multiple files are supplied to it) or the residual dataset(s) after the fit has been subtracted.
Updates to other tasks (in no particular order):
- uvfit can now output multiple model or residual visibility files, in the same way as for uvsfit.
- imfit now returns uncertainty estimates for the fitted total flux and position error ellipse.
- uvcat got a bug fix for missing frequency specifications in output.
- gpcal now removes any previous binned solutions if a single continuum solution is specified.
- linmos now uses the correct frequency for the primary beam of each image, when combining images at different frequencies.
- mfboot got a bug fix for crashes on integer overflow when used with large datasets with many channels.
- invert copes with larger cubes (>8192 channels) now.
- uvspec gets the 'hdr' option for plotting RFI affected data.
- imcomb gets the 'fqaver' option and a bug fix for the output frequency in header.
- atlod has an improved birdie mode for 64MHz zoom data.
- pgflag got a bug fix for the convolution command '*'.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Re: Recent Miriad changes
A couple of bugfixes were made last week that may affect recent data reduction.
- The position angle of the coordinate error ellipse reported by imfit was incorrect, but is now fixed.
- The opacity correction made by atlod was found to be significantly wrong, but is now fixed.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Re: Recent Miriad changes
An important calibration bugfix was made in the last week, which should make calibration more reliable in the presence of RFI or in spectral windows where large fractions of the band are flagged. The gpcal task was found to produce NaN values for the leakages under certain conditions, but will now fail more gracefully.
The gaufit and atlod tasks were also modified, but their operation should be unaltered for most users. Please let us know if you experience troubles with these tasks.
The gaufit and atlod tasks were also modified, but their operation should be unaltered for most users. Please let us know if you experience troubles with these tasks.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
-
- ATCA Expert
- Posts: 297
- Joined: Mon Feb 08, 2010 1:37 pm
Re: Recent Miriad changes
Some more changes to cope with wide bandwidths have been made recently:
selfcal now has frequency binning, and an option to provide a separate model image for each bin,
linmos and restor can now use the spectral index plane to make a better primary beam correction,
linmos can also create an image of the effective frequency of a primary beam corrected mfs image.
Here is a full list of the changes to Miriad since the previous post:
closure.for
Added option for creating a log file with plot values and uncertainties.
fits.for
Correctly check for buffer overflow in uvin
gpboot.for
Complain about large scaling factors
Complain if adjacent frequency bins differ markedly
gpedit.for
fix mem alloc issue
linmos.for
add options=frequency, fix cube bug
add mfs option to restor, handle mfs plane in linmos
mfcal.for
Avoid producing NaNs when there are zeroes in the data
regrid.for
don't check 3rd axis if only 2 requested
restor.for
add mfs option to restor, handle mfs plane in linmos
selfcal.for
Add mmfs option
add frequency binning
uvedit.for
Fix handling of negative dec between 0 and -1, thanks to MRC
uvflag.for
fix format error
avoid duplicate version messages
deal with larger numbers of visibilities
Convert version numbering to versan().
Cope with 128 antennas (MWA)
uvimage.for
add options time,freq and improve axis labelling
Initial revision (copied from CARMA)
uvlist.for
avoid duplicate version messages
fix output overflow for large number of channels
uvplt.for
cope with >99 antennas
Fix scaling of uv coordinates when nanosec requested
selfcal now has frequency binning, and an option to provide a separate model image for each bin,
linmos and restor can now use the spectral index plane to make a better primary beam correction,
linmos can also create an image of the effective frequency of a primary beam corrected mfs image.
Here is a full list of the changes to Miriad since the previous post:
closure.for
Added option for creating a log file with plot values and uncertainties.
fits.for
Correctly check for buffer overflow in uvin
gpboot.for
Complain about large scaling factors
Complain if adjacent frequency bins differ markedly
gpedit.for
fix mem alloc issue
linmos.for
add options=frequency, fix cube bug
add mfs option to restor, handle mfs plane in linmos
mfcal.for
Avoid producing NaNs when there are zeroes in the data
regrid.for
don't check 3rd axis if only 2 requested
restor.for
add mfs option to restor, handle mfs plane in linmos
selfcal.for
Add mmfs option
add frequency binning
uvedit.for
Fix handling of negative dec between 0 and -1, thanks to MRC
uvflag.for
fix format error
avoid duplicate version messages
deal with larger numbers of visibilities
Convert version numbering to versan().
Cope with 128 antennas (MWA)
uvimage.for
add options time,freq and improve axis labelling
Initial revision (copied from CARMA)
uvlist.for
avoid duplicate version messages
fix output overflow for large number of channels
uvplt.for
cope with >99 antennas
Fix scaling of uv coordinates when nanosec requested
-
- ATCA Expert
- Posts: 297
- Joined: Mon Feb 08, 2010 1:37 pm
Re: Recent Miriad changes
Since the last update (May 2013) a number of bug fixes and enhancements were made to Miriad:
Mark
------------------------------------------
Here is the full changelog:
atfix.for
Fix optimization bug causing tsys jumps at freq change
blflag.for
reduce static memory usage
calred.for
expand docs and digits
cgdisp.for
Add cubehelix colour scheme
cgspec.for
Add cubehelix colour scheme
gpcopy.for
fix apply mode, add relax option
handle bins in merge and apply mode
imcomb.for
Fix fqaver option - failed for > MAXOPEN files
imsad.for
fix ofmmod call / avoid fatal error
invert.for
add beam and res options to imsize and cellsize to change default values
mfboot.for
Fix nospec option
mfclean.for
avoid overflow when displaying sub-beam size
update docs for region keyword
mfspin.for
Read mostable for pointing centre if present
mirmax.for
Print Miriad's maxdim parameters
pgflag.for
add 'e' command to extend flags
psrbl.for
add 'l' to the uvflags to avoid a crash downstream
psrfix.for
add 'l' to the uvflags to avoid a crash downstream
psrplt.for
Add title parameter
regrid.for
previous fix introduced new bug, fix that one too
rmclean.for
update to George Heald's version 1.8
uvflux.for
merge vlbi option
add long option - more decimal places in output
- For mm observers: a problem was found that affects users who apply atfix with default settings to observations at multiple frequencies in the 15 or 7mm band, e.g., to apply an dantpos correction file. The resulting Tsys could have jumps or steps. This has now been fixed, but the recommendation is not to change the online tsys calibration in these bands (use tsyscal=any).
- The cg display tasks now support a series of new colour tables based on the cubehelix scheme.
- A few tasks gained some extra digits of precision in the log or printout: calred, mfclean, uvflux
- The gpcopy task now handles binning correctly and has a relax option to make it less strict about interpolation intervals in apply mode.
- The task imcomb can handle more files in fqaver mode.
- Invert has two new shortcuts to specify the image size in primary beams and the cell size in number of pixels per resolution element.
- Mfspin now handles images with differing pointing and phase centre correctly.
- The flagging task pgflag now has an 'e' command which 'grows' or extends the flagged areas in an attempt to catch the data affected by low level RFI.
- The rmclean task was updated to the latest version supplied by its author.
Mark
------------------------------------------
Here is the full changelog:
atfix.for
Fix optimization bug causing tsys jumps at freq change
blflag.for
reduce static memory usage
calred.for
expand docs and digits
cgdisp.for
Add cubehelix colour scheme
cgspec.for
Add cubehelix colour scheme
gpcopy.for
fix apply mode, add relax option
handle bins in merge and apply mode
imcomb.for
Fix fqaver option - failed for > MAXOPEN files
imsad.for
fix ofmmod call / avoid fatal error
invert.for
add beam and res options to imsize and cellsize to change default values
mfboot.for
Fix nospec option
mfclean.for
avoid overflow when displaying sub-beam size
update docs for region keyword
mfspin.for
Read mostable for pointing centre if present
mirmax.for
Print Miriad's maxdim parameters
pgflag.for
add 'e' command to extend flags
psrbl.for
add 'l' to the uvflags to avoid a crash downstream
psrfix.for
add 'l' to the uvflags to avoid a crash downstream
psrplt.for
Add title parameter
regrid.for
previous fix introduced new bug, fix that one too
rmclean.for
update to George Heald's version 1.8
uvflux.for
merge vlbi option
add long option - more decimal places in output