Recent Miriad changes

Is MIRIAD being a pain? Let us know your experience.

Moderator: Mark.Wieringa

ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Recent Miriad changes

Post by ste616 »

This thread will be updated regularly with changes to Miriad tasks. We recommend that Miriad users visit this thread periodically (or subscribe to receive notification when new posts are made to this thread). You will not be able to post replies in this thread, so if you want to ask further questions about these changes, please do so in a new thread.
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Miriad changes: 18 Jan 2012

Post by ste616 »

A large number of changes were comitted last week, primarily to improve large file support (see thread http://atcaforum.atnf.csiro.au/viewtopic.php?f=10&t=92).

The tasks that have better support for large files are:
  • uvacflag
  • imdiff
  • uvfstats
  • tvclip
  • fits
  • uvpflag
  • invert
  • cgspec
  • uvglue
  • uvmodel
  • selfcal
  • linmos
  • tvflag
  • uvlin
  • gpscal
  • pgflag
Updates were also made to mbspect and imspect to fix a "bug" by changing the axis name "felocity" to "velocity" on plots generated by these tasks, in order to make them publishable.

The task attsys was updated with a bugfix, and a new option 'inverse' was added.

We encourage all users who use these tasks to immediately upgrade.
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Miriad changes: 26 March 2012

Post by ste616 »

A couple of recent changes to report on here.

The first change is to uvsplit to allow for large numbers of files to be created when splitting into small frequency chunks using the maxwidth parameter. It is now possible for uvsplit to produce up to 1024 files - see thread http://atcaforum.atnf.csiro.au/viewtopic.php?f=10&t=106 for more details.

Miriad is now also fully capable of reading the FITS spectral standard. This feature has been progressively implemented by Mark Calabretta and is now ready. His email to the Miriad users list:
Greetings to Miriad users,

ATNF's Miriad now supports the FITS spectral standard as well as
maintaining backwards compatibility with the older AIPS spectral
convention.

In practical terms, this means that you should be able to read
any FITS image with a standard spectral coordinate header, or
an older AIPS-style header, and have the spectral coordinates
interpreted correctly. That includes IR, optical, UV, X-ray,
and gamma ray spectra that conform to the FITS standard.

Task 'velsw' can also now switch the spectral axis to any of the
new spectral types such as redshift, wavelength, or relativistic
velocity.

If you are not familiar with the FITS spectral standard, here is
a brief summary of the main differences:

1) The Doppler frame is recorded in a separate SPECSYS header
record in FITS - now copied to the 'specsys' header item in
Miriad and listed by 'prthd'. In all cases the spectral
axis is Doppler shifted to the frame specified by specsys.

The old method recorded the Doppler frame as part of
CTYPEi, e.g. 'FREQ-LSR', 'VELO-HEL', etc. and these are
still written by 'invert'.

2) You may see a three-letter code in place of the old Doppler
frame, for example 'VOPT-F2W'. The 'F2W' bit is a non-
linear algorithm code analogous to the codes used for
celestial projections. 'F2W' indicates that the axis is
linear in frequency but is being expressed in terms of a
variable (VOPT) that is linear in wavelength. If the code
is absent, the axis is linear, e.g. 'FREQ'.

While there are dozens of possibilities, the following table
lists some of the new spectral types you may encounter:

Old New
ctype ctype specsys
-------- ------------------
FREQ-OBS FREQ TOPOCENT ...frequency
VELO-HEL VRAD BARYCENT ...radio velocity
FELO-LSR VOPT-F2W LSRK ...optical velocity
n/a ZOPT-F2W (any) ...redshift (radio spectra)
n/a ZOPT (any) ...redshift (optical spectra)
n/a WAVE-F2W (any) ...wavelength (radio spectra)
n/a WAVE (any) ...wavelength (optical spectra)
n/a VELO-F2V (any) ...relativistic velocity
n/a VELO-W2V (any) ...relativistic velocity
n/a ENER (any) ...photon energy (e.g. X-ray)

A linear wavelength axis, 'WAVE' as above, might come from an
optical spectrum or a regridded radio spectrum. Notice that the
old VELO-{OBS,HEL,LSR} is radio velocity, but in all other cases
VELO now indicates relativistic velocity.

You should be aware that, historically, Miriad adopted an
unconventional interpretation for FREQ-HEL and FREQ-LSR. In a
Miriad header and FITS files written by Miriad, these axes are
actually topocentric, not barycentric or LSRK as it might
appear. The help for task 'velsw' explains how this is handled,
and 'impos' also has a new option related to it.

Mark Calabretta
ATNF
To support this change, Miriad now requires WCSLIB 4.12 or later. If you wish to compile Miriad from source, you can get the latest WCSLIB from ftp://ftp.atnf.csiro.au/pub/software/wcslib/
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: Recent Miriad changes

Post by ste616 »

Recently, the task pgflag has been upgraded to include an automatic flagger based on the 'SumThreshold' algorithm used by the LOFAR AOFlagger. This algorithm is tunable, but even with the default parameters it has been shown to be quite effective at quickly removing RFI at 16cm.

Adding to it's usefulness with automatic flagging, pgflag can now be run in non-interactive mode. See the updated documentation for full details about how to use this task.

The task atrecal can now calculate a Tsys spectrum from the autocorrelations and the new option fsystemp in invert lets you use this in the imaging weights. This will result in a better noise optimised image when there is a large change in Tsys across the observed band.

A new task uvfmeas is now available, and can be used to measure fluxes for point sources (at the phase centre), similar to other tasks uvflux and calred, but is more useful for the wider bandwidths that CABB makes available. The uvfmeas task makes it possible to measure spectral index across the band, and compare the scalar and vector averages via a plot.

A change has been made to atlod to allow for opacity corrections to be computed and applied in smaller subbands across the band.

Bug fixes have been made to the tasks gpaver, uvcat and uvsplit.
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: Recent Miriad changes

Post by ste616 »

Miriad task atlod now flags out continuum channels 9, 17 and 25 if the option 'birdie' is used with a 64 MHz zoom dataset.
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: Recent Miriad changes

Post by ste616 »

A number of changes have been made since the last update.

A new task uvsfit has been made (by Dave McConnell) that can be used to fit source parameters such as flux, spectral index and curvature, and position using only the uv data files. It requires that you already know the rough position of the sources that you want to fit, and can be used to generate either a model dataset (or datasets if multiple files are supplied to it) or the residual dataset(s) after the fit has been subtracted.

Updates to other tasks (in no particular order):
  • uvfit can now output multiple model or residual visibility files, in the same way as for uvsfit.
  • imfit now returns uncertainty estimates for the fitted total flux and position error ellipse.
  • uvcat got a bug fix for missing frequency specifications in output.
  • gpcal now removes any previous binned solutions if a single continuum solution is specified.
  • linmos now uses the correct frequency for the primary beam of each image, when combining images at different frequencies.
  • mfboot got a bug fix for crashes on integer overflow when used with large datasets with many channels.
  • invert copes with larger cubes (>8192 channels) now.
  • uvspec gets the 'hdr' option for plotting RFI affected data.
  • imcomb gets the 'fqaver' option and a bug fix for the output frequency in header.
  • atlod has an improved birdie mode for 64MHz zoom data.
  • pgflag got a bug fix for the convolution command '*'.
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: Recent Miriad changes

Post by ste616 »

A couple of bugfixes were made last week that may affect recent data reduction.
  • The position angle of the coordinate error ellipse reported by imfit was incorrect, but is now fixed.
  • The opacity correction made by atlod was found to be significantly wrong, but is now fixed.
In addition to this, pgflag can now generate flagging patch files than can be used to reapply or undo the flagging actions it makes in an automated fashion. This may be useful for those who wish to manually do some flagging with pgflag but have this same flagging repeated in an automated script later on.
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: Recent Miriad changes

Post by ste616 »

An important calibration bugfix was made in the last week, which should make calibration more reliable in the presence of RFI or in spectral windows where large fractions of the band are flagged. The gpcal task was found to produce NaN values for the leakages under certain conditions, but will now fail more gracefully.

The gaufit and atlod tasks were also modified, but their operation should be unaltered for most users. Please let us know if you experience troubles with these tasks.
cheers
Jamie Stevens
ATCA Senior System Scientist
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: Recent Miriad changes

Post by Mark.Wieringa »

Some more changes to cope with wide bandwidths have been made recently:

selfcal now has frequency binning, and an option to provide a separate model image for each bin,
linmos and restor can now use the spectral index plane to make a better primary beam correction,
linmos can also create an image of the effective frequency of a primary beam corrected mfs image.

Here is a full list of the changes to Miriad since the previous post:

closure.for
Added option for creating a log file with plot values and uncertainties.

fits.for
Correctly check for buffer overflow in uvin

gpboot.for
Complain about large scaling factors
Complain if adjacent frequency bins differ markedly

gpedit.for
fix mem alloc issue

linmos.for
add options=frequency, fix cube bug
add mfs option to restor, handle mfs plane in linmos

mfcal.for
Avoid producing NaNs when there are zeroes in the data

regrid.for
don't check 3rd axis if only 2 requested

restor.for
add mfs option to restor, handle mfs plane in linmos

selfcal.for
Add mmfs option
add frequency binning

uvedit.for
Fix handling of negative dec between 0 and -1, thanks to MRC

uvflag.for
fix format error
avoid duplicate version messages
deal with larger numbers of visibilities
Convert version numbering to versan().
Cope with 128 antennas (MWA)

uvimage.for
add options time,freq and improve axis labelling
Initial revision (copied from CARMA)

uvlist.for
avoid duplicate version messages
fix output overflow for large number of channels

uvplt.for
cope with >99 antennas
Fix scaling of uv coordinates when nanosec requested
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: Recent Miriad changes

Post by Mark.Wieringa »

Since the last update (May 2013) a number of bug fixes and enhancements were made to Miriad:
  • For mm observers: a problem was found that affects users who apply atfix with default settings to observations at multiple frequencies in the 15 or 7mm band, e.g., to apply an dantpos correction file. The resulting Tsys could have jumps or steps. This has now been fixed, but the recommendation is not to change the online tsys calibration in these bands (use tsyscal=any).
  • The cg display tasks now support a series of new colour tables based on the cubehelix scheme.
  • A few tasks gained some extra digits of precision in the log or printout: calred, mfclean, uvflux
  • The gpcopy task now handles binning correctly and has a relax option to make it less strict about interpolation intervals in apply mode.
  • The task imcomb can handle more files in fqaver mode.
  • Invert has two new shortcuts to specify the image size in primary beams and the cell size in number of pixels per resolution element.
  • Mfspin now handles images with differing pointing and phase centre correctly.
  • The flagging task pgflag now has an 'e' command which 'grows' or extends the flagged areas in an attempt to catch the data affected by low level RFI.
  • The rmclean task was updated to the latest version supplied by its author.
Cheers,

Mark

------------------------------------------
Here is the full changelog:

atfix.for
Fix optimization bug causing tsys jumps at freq change

blflag.for
reduce static memory usage

calred.for
expand docs and digits

cgdisp.for
Add cubehelix colour scheme

cgspec.for
Add cubehelix colour scheme

gpcopy.for
fix apply mode, add relax option
handle bins in merge and apply mode

imcomb.for
Fix fqaver option - failed for > MAXOPEN files

imsad.for
fix ofmmod call / avoid fatal error

invert.for
add beam and res options to imsize and cellsize to change default values

mfboot.for
Fix nospec option

mfclean.for
avoid overflow when displaying sub-beam size
update docs for region keyword

mfspin.for
Read mostable for pointing centre if present

mirmax.for
Print Miriad's maxdim parameters

pgflag.for
add 'e' command to extend flags

psrbl.for
add 'l' to the uvflags to avoid a crash downstream

psrfix.for
add 'l' to the uvflags to avoid a crash downstream

psrplt.for
Add title parameter

regrid.for
previous fix introduced new bug, fix that one too

rmclean.for
update to George Heald's version 1.8

uvflux.for
merge vlbi option
add long option - more decimal places in output
Post Reply