gpcal interval vs. gpaver

[keithBannister]

Hi All,

My question is: what is the difference between the gpcal interval, and a gpaver interval?
A related question is: how are the gains/phases (and bandpass) interpolated between calibrator scans? Is it just interpolated between the nearest (in time) two calibration solutions?

The ATCA miriad guide http://www.atnf.csiro.au/computing/soft ... ode95.html says to use a short interval (0.1min) if the phases are changing fast. If such a short interval is required, I don't understand how you would have any hope of calibrating your data if you have >> 0.1min on your source.

My application: 6&3cm data, 40s on calibrator, ~10 mins between calibrator scans.

p.s. I've been trying to read the docs as closely as possible but I can't get my head around it. Sorry if this is a trivial question.

[ste616]

Hi Keith,

It's an important question, so thanks for asking it!

Miriad will interpolate the gain at any particular time from the closest gain measurement before and after the time in question. So, if you have done a gpcal with interval of 0.1 minutes, it will indeed only take the last calibrator cycle before and first cycle after as the reference points.

To get around this, you can either specify a larger gpcal interval or average the gain solutions with gpaver.

The gpcal interval specifies the amount of raw data to average together before calculating the complex gain. This can be useful if you have a weak phase calibrator and the signal-to-noise ratio is low. It can also be useful to do this if you expect that your data is decorrelating on timescales longer than your integration time - averaging your data over that timescale will then allow Miriad to determine the scaling factor to correct for this decorrelation. However, if your phase is rapidly varying, averaging data over some length of time before solving for the complex gain may lead to trouble.

The gpaver interval allows you to average a number of gpcal gain solutions. If you assume that you have set gpcal to determine the gain at every cycle, then even if the phase is rapidly varying, the phase will be solved for accurately by gpcal and gpaver will take the linear average of these gain values, which should not suffer from the data averaging problem as much.

Our general advice is, as long as you have a sufficiently bright phase calibrator, to solve for the complex gains using gpcal at every cycle, then use gpaver to smooth the gain solutions with gpaver to produce one point per phase calibrator visit.

For your specified application (and indeed most applications), this will be the best starting point.

[keithBannister]

Thanks Jamie. Super-useful reply. I'm off to gpaver my data now

[keithBannister]

OK, so here's my next question (again, sorry if this is trivial):

With the intervals (both for gpaver and gpcal) exactly which records are averaged over the interval? How does it detect gaps and not average over them?

So in my case (40s calibrator scan, 10 minutes between scans, 6cm) should I set the interval to:

- Slightly less than the scan duration, so gpcal doesn't average over 2 successive scans, or
- Slightly more than the scan duration, so gpcal gets averages over the whole calibrator scan

I'm asking because I'm a little surprised by the results I'm getting. If I do no gpaver, I get reasonably small scatter (1.5%), but if I do a gpaver over 0.5 minutes, I get a much bigger scatter (50%). (see below)

gpcal with interval = 0.1

gpcal0.1.png (8.5 KiB) Viewed 79970 times

gpcal with interval 0.1 and gpaver with interval = 0.5

gpcal0.1plusgpaver0.5.png (7.5 KiB) Viewed 79970 times

[keithBannister]

Hmm, so this is interesting.

I've done 3 plots:
- gpcal with interval=0.1
- gpcal with interval=0.2
- gpcal with interval=0.1 AND gpaver with interval=0.2


The bold ones have similar results to each other, but worse scatter in stokes I by a factor of 20! (in axis=time,imag) than gpcal with interval=0.1. It's 6cm data, so I didn't think the phases were changing on ~6s timescales. I'm missing something simple.

Here's a typical gpcal:
uvplt% tget gpcal
Task: gpcal
vis = 0614.i0.2
select =
line =
flux =
refant =
minants =
interval = 0.2
nfbin =
tol =
xyphase =
options = xyvary,qusolve


And a typical uvplt:
Task: uvplt
vis = ./0614.i0.1.gpaver0.2/
line =
select =
stokes = i,q,u,v
axis = time,imag
xrange =
yrange =
average =
hann =
inc =
options = nobase
subtitle =
device = gpivt_i0.1_0.2/png
nxy =
size =
log =
comment =

Uvplt says:
uvplt: Revision 1.13, 2011/06/14 04:40:21 UTC

Will plot unflagged data

Applying bandpass corrections to ./0614.i0.1.gpaver0.2/
Applying gain corrections to ./0614.i0.1.gpaver0.2/
Applying polarization leakage corrections to ./0614.i0.1.gpaver0.2/

i.e. not complaining of the plot buffer overflowing, etc.

gpcal with interval=0.1

gpivt_i0.1.png (11.59 KiB) Viewed 79970 times

gpcal with interval 0.2

gpivt_i0.2.png (8.47 KiB) Viewed 79970 times

gpcal with interval 0.1 and gpaver with interval 0.2

gpivt_i0.1_0.2.png (8.66 KiB) Viewed 79970 times

[keithBannister]

Well,

I think I worked out my problem - basically the phases are wandering all over the place (even at 6cm!) the atmosphere must have been particularly bad

Here's an example of 10 mins on 1934. As you can see, there are some excursions of up to 60 degrees in 2 minutes, which I'm sure is enough to confuse any averaging.

My lessons:
- Don't set interval > 0.1 unless you really, really need to (e.g. weak calibrator)
- Only observe bright calibrators for 1 integration cycle - that way you'll have no idea how bad your phases are
- Never look carefully at your data, you'll lose 3 days going down a rabbit hole and learn a lot

1934_pt.png (10.74 KiB) Viewed 79968 times