atlod options=notsys and calibration

Got a calibration problem? Discuss it here.

Moderator: Mark.Wieringa

Post Reply
tg1989
Posts: 17
Joined: Sun Dec 07, 2014 3:04 pm

atlod options=notsys and calibration

Post by tg1989 »

I am having a bit of trouble calibrating some C/X band data, and I was hoping for a bit of advice.

My data has the two IFs set at central frequencies of 7.7 and 9.5GHz, giving roughly 50MHz of overlap (after flagging out edge channels). At these frequencies 1934-638 is strong enough to perform a bandpass calibration and establish a flux density scale.

My steps of the reduction are as follows:

Code: Select all

mfcal vis=1934-638.9500 interval=0.1
gpcal vis=1934-638.9500 options=xyvary interval=0.1 nfbin=2
gpcopy vis=1934-638.9500 out=0327-241.9500
gpcal vis=0327-241.9500 options=xyvary,qusolve interval-0.1 nfbin=2
gpboot vis=0327-241.9500 cal=1934-638.9500

mfcal vis=1934-638.7700 interval=0.1
gpcal vis=1934-638.7700 options=xyvary interval=0.1 nfbin=2
gpcopy vis=1934-638.7700 out=0327-241.7700
gpcal vis=0327-241.7700 options=xyvary,qusolve interval-0.1 nfbin=2
gpboot vis=0327-241.7700 cal=1934-638.7700
Inspecting the spectrum of 1934-638 across both IFs with uvfmeas, it is as expected. However, when plotting the spectrum of 0327-241 with uvfmeas, the two bands are clearly discontinuous. The small overlapping region of ~50MHz gets average together.

When it comes to the target source, it I plan on imaging the data together, so to me it makes sense to ensure that the spectrum of the secondary calibrator, which is roughly 1Jy, is also consistent. I kept this small amount of overlap as a way to 'QA' things before imaging, particularly with the phase calibrator 0327-241. If you refer to the norml_reduction.png attachment you can see the behavior.

I have tried a couple thing - mostly mostly four nfbins and different combinations of flagging with no effect.

I have managed to recover something that is more consistent by loading the data in with the notsys keyword

Code: Select all

atlod out=uv_data_notsys.uv options=rfiflag,birdie,xycorr,noauto,notsys
A normal calibration after loading the data in with this option gives a much more consistent spectrum. If you refer to the option_notsys.png attachment you can see the difference.

I was hoping someone could explain what exactly is going on, and how can I ensure the most robust calibration strategy? My understanding of the system temperature was that it is an additional weight for each visibility, so it wouldn't change its amplitude, just reduced its weight when imaging or fitting. Have I got a more fundamental misunderstanding?

This was meant to be a quick look, as I got the data early this morning and am observing more over the coming weeks. I would like to be certain this it was not a mistake I made during observing.
Attachments
normal_reduction.png
normal_reduction.png (88.46 KiB) Viewed 9069 times
option_notsys.png
option_notsys.png (111.72 KiB) Viewed 9069 times
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: atlod options=notsys and calibration

Post by ste616 »

Hi Tim,

The most common cause of behaviour like this is including data across an acal. Are you sure you discarded data before the final acal on 1934?
cheers
Jamie Stevens
ATCA Senior System Scientist
tg1989
Posts: 17
Joined: Sun Dec 07, 2014 3:04 pm

Re: atlod options=notsys and calibration

Post by tg1989 »

Hi Jamie,

I am pretty sure I have. Going through the atlod history of the run I have

Code: Select all

ATLOD:    Processing file 2017-11-23_0624.C3171
ATLOD:     06:23:29 CAOBS> set file c3171_obs_goodweather
ATLOD:     06:23:42 CAOBS> start 1/12
ATLOD:     06:24:16 CAOBS> set c3171_cal
ATLOD:     06:24:22 CAOBS> stop
ATLOD:     06:24:27 CAOBS> set file c3171_cal
ATLOD:     06:24:39 CAOBS> track 3
ATLOD:    CABB data detected
ATLOD:     06:29:49 CAOBS> cor dcal
ATLOD:     06:31:57 CAOBS> cor dcal
ATLOD:     06:33:41 CAOBS> cor pcal
ATLOD:     06:34:39 CAOBS> cor acal
ATLOD:     06:44:04 CAOBS> stop
ATLOD:     06:44:07 CAOBS> corr closef
ATLOD:    RPFITS file version is 2.12
ATLOD:    Total number of spectra selected: 13680
ATLOD:    Summary of spectra flagged
ATLOD:    Flagging Reason            Fraction
ATLOD:    ---------------            --------
ATLOD:    Antenna off-source/off-line  3.51%
ATLOD:    ---------------------------------------
ATLOD:    Processing file 2017-11-23_0644.C3171
ATLOD:     06:44:21 CAOBS> track 3
ATLOD:    CABB data detected
ATLOD:     07:25:42 CAOBS> stop
ATLOD:     07:25:54 CAOBS> set file c3171_obs_goodweather
ATLOD:     07:26:01 CAOBS> start 1/100
I only issue one acal command when setting up, and I close that file. I only ever use CACOR command prompt for tvchan/fflags, so I would not have issued another acal there that would be missed in the atlod history. The initial track 1/12 of the 'c3171_obs_goodweather' was to make sure that the mosaic file was correct and to see if CAOBS would load it (assuming it checks for the mosaic files when it loads the schedule). There was no data that was written from this track. EDIT: Actually, I remember now that is was because the calibrator and field were still below the 12 degree elevation limit, which is why there is no data and I went to 1934.

Inspecting the data directly from atlod, including the initial junk file, you can in uvplt axis=time,amp where the acal command was issued and can be seen in the attached figures. I tracked 1934 for ~40 minutes as he field was not up. I only used the last ten minutes of the dwell. At both the 7.7 and 9.5GHz bands there was some RFI in some a set of narrow channels. Flagging those out the data was able to be calibrated - apparently. The second uvplt figure is the remaining flagged data that would go on to be calibrated.

I only include the 7.7GHz figures, but they are the same for the 9.5GHz.

I have used uvspec to look at the phases across the bands (without the calibrations being applied) and they phases look well behaved for the unflagged data. I have attached the output from this task - I used a interval of 10 and nxy of 6,5 so I could get the data into a single window.

Thanks,
Tim
Attachments
1934_7700_uvspec.png
1934_7700_uvspec.png (14.77 KiB) Viewed 9046 times
1934_7700_postflag.png
1934_7700_postflag.png (8.49 KiB) Viewed 9046 times
1934_7700.png
1934_7700.png (13.98 KiB) Viewed 9046 times
tg1989
Posts: 17
Joined: Sun Dec 07, 2014 3:04 pm

Re: atlod options=notsys and calibration

Post by tg1989 »

Hi Jamie,

I got around to looking at this again. If I were to carry out the reduction described in my initial post, I get the following plot for time vs amplitude of my phase reference calibrator. You can see that the first two scans are off when I include options=nocal.

Plotting the time vs systemp of the entire run (i.e. before uvsplit) using varplt, you can see at around the same time there is a noticeable jump on some antenna on the second polarization component. This jump, I think, corresponds to the changing of the tvchan range. Setting up on 1934-638 I don't recall there being bad rfi, but it must have gotten worse at the low elevation, and after a few integration cycles of assistance beeping I changed the tvchan range.

Talking to Cormac, I understand that the correlator coefficients are multiplied by the Tsys over a gain curve. And reading the atlod documentation, the Tsys is calculated on the tvchan range.

So, I think turning off the online Tsys correction is about the only thing I could do for this data, apart from flagging out the ~1 hour of data before i changed the tvchan range and performing a normal reduction strategy.

Does this sound reasonable? If I do go ahead with the atlod option notsys, are there any gotchas I should be aware of?

Thanks,
Tim
Attachments
day_time_systemp.png
day_time_systemp.png (15.04 KiB) Viewed 8751 times
0327-241_time_amp_nocal.png
0327-241_time_amp_nocal.png (7.78 KiB) Viewed 8751 times
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: atlod options=notsys and calibration

Post by ste616 »

Hi Tim,

You're right, changing tvchannels affects the Tsys and therefore changes the scaling factors in an unpredictable way. The only way to reverse this is via options=notsys as you say.

In most cases, reversing the online application of Tsys won't have much of a deleterious effect, unless you're mosaicking. While mosaicking, the correlator "fudges" the Tsys for any cycle when the antennas may have gotten on source part way through the cycle, in order to keep the amplitudes scaled correctly with less data than a normal cycle. Thus, if you're mosaicking and you need to use notsys, you'll need to flag out any cycle where this may have been the case; qvack is a useful task for that.
cheers
Jamie Stevens
ATCA Senior System Scientist
tg1989
Posts: 17
Joined: Sun Dec 07, 2014 3:04 pm

Re: atlod options=notsys and calibration

Post by tg1989 »

Jamie,

Ok, thanks for that.

Running qvack on my mosaic file (created using uvsplit options=mosaic), with mode=moasic and interval=0.17 (ten second integration), no records or correlations are returned as flagged. Does this sound good to you, or user error?

Code: Select all

qvack vis=mos.uv interval=0.17 mode=mosaic
Thanks,
Tim
Post Reply