flux density bootstrapping (mfboot, gpboot, plboot)

Is MIRIAD being a pain? Let us know your experience.

Moderator: Mark.Wieringa

Post Reply
baerbel
Posts: 6
Joined: Mon Feb 08, 2010 1:27 pm

flux density bootstrapping (mfboot, gpboot, plboot)

Post by baerbel »

gpboot - flux density bootstrapping

mfboot - performs analagous function to gpboot, but for planets

plboot - obsolete, superceded by mfboot

It would be great to have a consistent and up-to-date description of these tasks on the web
and within miriad. If plboot is obsolete (wrong ?) then this should be made clear in the task
description and alternatives (mfboot) be provided.

See:
* http://www.atnf.csiro.au/observing/user ... ug_32.html" onclick="window.open(this.href);return false;
* http://www.atnf.csiro.au/computing/software/miriad" onclick="window.open(this.href);return false;
* http://bima.astro.umd.edu/miriad/userguide_US.pdf" onclick="window.open(this.href);return false; (see Section 23.2.5)
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: flux density bootstrapping (mfboot, gpboot, plboot)

Post by ste616 »

A lot of recent work has gone into updating the MIRIAD calibration tasks to ensure that they can handle CABB data. The documentation is lagging behind, but will be updated as soon as possible.

As for flux density bootstrapping, you are correct in stating that plboot is obseleted and shouldn't be used.

gpboot and mfboot are used for different purposes, but can be confusing, so I'll go into some detail here.

When one performs a calibration with mfcal, if the source's flux behaviour is known (as is the case for 1934-638, among others), then the MIRIAD task will adjust the gain solution to set the correct flux at the centre of the band, and it will also adjust the bandpass solution to set the correct flux across the band. This, for example, results in a 2 GHz spectrum of 1934-638 showing a significant decrease in flux from low frequency to high frequency.

However, when one uses mfcal on a source that has no known flux, the task makes its best guess at the flux at the centre of the band, but assumes that the flux is constant across the band. This is usually an incorrect assumption that requires careful further reduction to correct.

The simplest way to prevent this incorrect assumption from affecting the data is to take the bandpass from a source with known flux behaviour (the flux calibrator) and copy it to the source with unknown flux. This will set the bandpass to the correct shape across the band, and will thus give an accurate measure of the spectral behaviour of the unknown source. The task gpboot can then be used to scale the gains in the unknown calibrator to the flux scale of the flux calibrator. One must be careful not to re-solve for the bandpass after copying the flux calibrator's bandpass across.

However it is possible to correct for the slope of the bandpass after flux scaling with gpboot. This will be necessary if the flux calibrator is too weak to use as a bandpass calibrator, which will happen, for example, when using 1934-638 as a flux calibrator at mm wavelengths. It will also be necessary if the flux calibrator is a planet, which cannot be put through mfcal (which solves for the bandpasses) because they are spatially resolved to the array. In this case, one should use mfcal to produce a bandpass solution from a bandpass calibrator. This bandpass solution should be copied to both the flux calibrator and any other calibrators that will need to have their flux solved for. All these calibrators (except the flux calibrator, if it is a planet) should be further calibrated with gpcal, which will set the proper flux of the flux calibrator, and the complex gains of the other calibrators.

At this point, if the flux calibrator is not a planet, gpboot should be used to scale the gains of all the calibrators to that of the flux calibrator. If the calibrator is a planet, then the complex gains from the unknown calibrator should be copied across to the flux calibrator using gpcopy. At this point, if the flux calibrator is not a planet, all the calibrators will have the correct flux at the band centre, but because the bandpass has come from a calibrator whose flux was probably unknown to MIRIAD, the flux across the band can not be trusted. Up until only a week ago, there was no easy way for MIRIAD to correct this problem.

Now though, mfboot can be used to correct for bandpass slope, if it is given a source that it has flux information for. mfboot takes a list of datasets in its "vis" variable, and this should be set to a list of the flux calibrator and all other calibrators that need to be correctly flux calibrated; this step will be the same whether the flux calibrator is a planet or not. The source to act as the flux calibrator must be specified in the "select" field on mfboot, eg. "select=source(1934-638)", or "select=source(uranus)". Note that it is very important that all the input datasets have the same gain scaling and bandpass solution at this point.

Running mfboot now will scale the fluxes of all the listed sources based on the known flux of the flux calibrator. If the flux calibrator is a planet, then mfboot's scale factor will probably be different from 1. However, if the flux calibrator is not a planet, then gpboot should have already correctly scaled the fluxes, and so mfboot should report a flux scaling of 1. However, in both cases, mfboot will take a point on either side of the band centre, and will correct the slope of the bandpass (which will be identical for each source) to match the known fluxes at these points. mfboot will report the slope correction it made.

It should be noted that once mfboot is run on a dataset, then running it again on that same source - if neither the gains or the bandpass are recomputed in the meantime - will result in no further correction. That means that running mfboot with a flux calibrator and one other source will work the first time, but running it again with the same flux calibrator and a different source will result in no correction being made to either source, since the flux calibration is already good for the flux calibrator, and mfboot implicitly assumes that the gain and bandpass solutions of all input sources are identical.

So, in summary, gpboot works to scale gains only, and will only work where both input datasets have valid gain solutions, and that one is assumed to be correctly scaled, making it unsuitable for planetary flux calibration.

In contrast, mfboot works to correct both gains and bandpasses of any number of input datasets, with the assumption that all input datasets have identical solutions for gains and bandpasses before execution.
cheers
Jamie Stevens
ATCA Senior System Scientist
cat001
Posts: 14
Joined: Tue Mar 09, 2010 3:58 pm

Re: flux density bootstrapping (mfboot, gpboot, plboot)

Post by cat001 »

Hi Jamie,

When I run mfboot in kaputar I get the warming: " Cannot adjust spectral slope because there is no bandpass table".

I did a mfcal of the bandpass calibrator, then a gpcopy of the bandpass calibrator to the flux calibrator (uranus), and gain calibrator, then a mfcal of the gain calibrator before running mfboot.

Should I be using gpcal instead of mfcal ?

Thanks,

cheers,
Catarina
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: flux density bootstrapping (mfboot, gpboot, plboot)

Post by ste616 »

Hi Catarina,

The short answer is yes - if you don't intend (or want) to solve for the bandpass, gpcal is the task to use as it does everything (and more) that mfcal does, but does not solve for the bandpass.

However, the warning you report here is puzzling if you did indeed follow that procedure, since there should be a bandpass table present. Can you list all the tasks and datasets explicitly - maybe there is something that is failing along the way.
cheers
Jamie Stevens
ATCA Senior System Scientist
cat001
Posts: 14
Joined: Tue Mar 09, 2010 3:58 pm

Re: flux density bootstrapping (mfboot, gpboot, plboot)

Post by cat001 »

Hi Jamie,

So I start the calibration by doing a

mfcal:
vis = bandpass calibrator
interval=10
refant = 3

gpcopy:
vis = bandpass calibrator
options = nocal
out = gain calibrator

uvcat:
vis = gain calibrator
out = gaincalibrator.cal
options = unflagged, nocal

mfcal:
vis = gaincalibrator.cal
interval = 2,1
options = nopassol,interpolate

gpcopy:
vis = bandpass cal
options = nocal
out = fluxcal (uranus)

uvaver
vis = fluxcal
interval =2
options = nocal
out = fluxcal.ave

gpcopy:
vis = gaincal.cal
out = fluxcal.ave
options = nopass

mfboot
vis = bpcal, gaincal.cal, fluxcal.ave
select = source(uranus)
device = /xs
mode = scalar
clip =0.5

This is the procedure that we have been using to reduced the data based from Tony Wong script. I thought the intent of the first mfcal is to solve for the bandpass.
Also, I only started getting this warming recently in kaputar.

thanks,

cheers,
Catarina
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: flux density bootstrapping (mfboot, gpboot, plboot)

Post by ste616 »

Hi Catarina,

Thanks for the detail you provided. I'll go through each step and let you know what should be changed and why.
mfcal:
vis = bandpass calibrator
interval=10
refant = 3
Good start, you've now made a good bandpass solution. Setting the interval to 10 minutes is not recommended however, as if the phases are not stable over that time period, you'll get decorrelation. And changing the interval does not affect the bandpass solution since mfcal's default behaviour is to make one bandpass solution from a dataset, using all the data.
gpcopy:
vis = bandpass calibrator
options = nocal
out = gain calibrator
This is also correct, although it's not really necessary to exclude the gain tables since they will be overwritten later. Still, no harm done...
uvcat:
vis = gain calibrator
out = gaincalibrator.cal
options = unflagged, nocal
Although technically a valid step, I don't see any reason to do this. Instead, I would recommend acting upon the gain calibrator dataset as is. Still, not a bad way to keep a backup if that's what you want, and removing the flagged data from the data might speed up later steps (maybe).
mfcal:
vis = gaincalibrator.cal
interval = 2,1
options = nopassol,interpolate
Since you're only solving the gains at this step - and not the bandpass - you can safely replace this step with a gpcal. If you're using CABB, I would also recommend against using the interpolate option; with 2 GHz of bandwidth, interpolating across a few flagged channels in the bandpass solution isn't really going to be worth it, and more often than not the flagged channels in the bandpass calibrator will be bad in all/most of the other data as well. If this is for old correlator data, maybe it's worth it - try the reduction with the interpolate option, but see if replacing this step with gpcal reduces the sensitivity by any significant amount.
gpcopy:
vis = bandpass cal
options = nocal
out = fluxcal (uranus)
OK, but see below for why this can be removed.
uvaver
vis = fluxcal
interval =2
options = nocal
out = fluxcal.ave
This step should definitely be removed. uvaver should only be performed at the start of a reduction (if you want to average up channels to increase channel sensitivity), or at the end of calibration once you're sure your calibration solution is correct. It is at this point that you lose your bandpass solution, since it gets applied to your data and can no longer be modified. As for the intention of this command - to average up the flux calibrator in time - mfboot will use all the data when determining the scaling factor anyway.
gpcopy:
vis = gaincal.cal
out = fluxcal.ave
options = nopass
Since your gain calibrator has the bandpass from the bandpass calibrator, and the gain solution for your phases, you can do this step without the "nopass" option to copy all the required calibration information in one step to the flux calibrator - a real time saver!
mfboot
vis = bpcal, gaincal.cal, fluxcal.ave
select = source(uranus)
device = /xs
mode = scalar
clip =0.5
Remember, the calibration solutions of all the input datasets need to be identical before this step or else this will fail, and you should also make sure that the solutions are available for modification (ie. not applied). At this point in your reduction (as it has been described), your gain calibrator and flux calibrator have identical solutions, but the bandpass calibrator has a different gain calibration, and the flux calibrator has not got a bandpass table. So after this command is executed, you'll find that gain and flux calibrators will have correct fluxes at the centre of the band, but the scaling will produce the wrong result for the bandpass calibrator, since it has started off with a different scaling. The fluxes across the band will also be incorrect because mfboot is unable to correct the bandpass solution to reflect how the flux of Uranus changes with frequency; keeping the bandpass solution as a change-able table will solve this problem, and stop the warning you have been experiencing. Of course, if you're only dealing with a small bandwidth, the bandpass slope correction won't be very important, but for CABB data, it is essential.

So I'd recommend not including the bandpass calibrator in this step. If you want to know what the bandpass calibrator's flux is, you'll need to do the following step before mfboot.

gpboot
vis=bandpass cal
cal=gaincal.cal

This will equalise the gain solution.

I hope this helps :)
cheers
Jamie Stevens
ATCA Senior System Scientist
cat001
Posts: 14
Joined: Tue Mar 09, 2010 3:58 pm

Re: flux density bootstrapping (mfboot, gpboot, plboot)

Post by cat001 »

Hi Jamie,

Thanks, that does help.
I do have a question, how is the

gpcopy
vis=bandpass cal
cal=gaincal.cal

different from the second step ...

gpcopy:
vis = bandpass calibrator
options = nocal
out = gain calibrator

thanks,

cheers
catarina
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: flux density bootstrapping (mfboot, gpboot, plboot)

Post by ste616 »

Hi Catarina,

Oh bum, that's a typo, and should be gpboot instead of gpcopy! Sorry :oops:
cheers
Jamie Stevens
ATCA Senior System Scientist
Post Reply