atlod seems to be flagging my bandpass cal

[emahony]

Hi,

I've been trying to reduce my ATCA data taken in March and for some reason atlod won't read in my bandpass cal (0537-441). I've tried lots of different combinations of options, but I still get the same problem:

atlod% inp
Task: atlod
in = 2010-03-14_0908.C1964
out = test.uv
ifsel =
restfreq =
options = noif,opcorr,birdie,rfiflag,noauto,xycorr
nfiles =
nscans =
atlod% go

atlod: Version 1.24, 2010/04/16 00:33:26 UTC

Flagging channels in 22 frequency ranges
Processing file 2010-03-14_0908.C1964
CABB data detected
Scan 3 started at 10MAR14:09:23:44.9
Source: 0843-54
Scan 4 started at 10MAR14:09:26:45.7
Source: 0830-6725

I observed my bandpass (0537-441) first (which must have been scan 1 and scan 2 for this particular file) and it looked fine on vis when I was observing. I might have typed "track 1" instead of "start 1", but I don't think this should make a difference. It also happens on all nights, not just this file. The data must be in the rpfits file since it was opened at 0908 UT and the first scan read in is at 0923.

If anyone knows what is going on that would be great.

Thanks,
Elizabeth

[ste616]

Hi Elizabeth,

According to my database, the file '2010-03-14_0908.C1964' has one observation of 0537-441 starting at 09:08:54, and has 520 seconds of data. The files opened prior to this does as well: '2010-03-14_0839.C1964'.

So, why can't you access it? According to my calibrator database reduction procedure, that 0537-441 data is not usable. I will investigate and return with more information.

[emahony]

Hi Jamie,

Thanks for the quick response!

2010-03-14_0839 was my setup file. I quickly just tried to load that but it all comes up as "discarded cal data" which makes sense if it was all setup. When observing, I closed that file after setup and started tracking 0537 for my bandpass for 10 mins and then moved onto the pointing cal for 0843 and the rest of my sources. Are you saying that for some reason the bandpass just wasn't written to the file? It definitely came up on vis when I was observing. But then why does it say that 0843 was scan 3 in atlod?

Cheers,
Elizabeth

P.S. I forgot to mention before that this is 12mm data if that makes a difference.

[ste616]

Hi Elizabeth,

The 0537-441 source that is present in the file '2010-03-14_0908.C1964' is in fact a pointing scan. So even if you typed 'track 1' instead of 'start 1' when observing this source, the correlator has marked the data as being from a pointing scan and thus it is flagged.

To unflag this data is easy. When you are using atlod, include the option 'caldata', which will keep data associated with pointings, instead of discarding them at this step. Of course, this data will still be flagged, but since this is the only 0537-441 in this file, you can simply do:

uvflag vis=test.uv flagval=unflag select=source(0537-441)

See, easy. However, it becomes a lot trickier now! Unfortunately this has unflagged all the data, including any off-source scans, and even the channels that are always flagged by the correlator. So we'll need to flag all that bad data again.

Luckily, the scan on 0537 in this file has no off-source scans. To flag all the always bad channels, do:

uvflag vis=test.uv flagval=flag select=amplitude(0,1e-6)

This works because any channel that is flagged by the correlator will be written out as having an amplitude of 0, and this uvflag command selects for this. If you used the correlator to flag the birdies, then this command will have reflagged them, and you don't need to do the next step manually. Otherwise, you should flag the channels 641, 257, 769, 1409, 1281, 1921, 1793, 1177, 157, 129 and 1153 using uvflag. After this, your data should be usable again

This brings up a very important consideration though, so it's good that it's on the forum. This problem will happen to you if you track on a pointing scan, so wherever possible, please use a dwell scan to observe your reference calibrators; it will make your life easier!

[emahony]

Thanks Jamie - that's probably what I did! I didn't realise that the correlator cared about what type of scan it was when tracking. Next time will use a dwell scan (why do I always learn these things the hard way??)

[emahony]

For anyone else who happens to make the same silly mistake that I did, we worked out that you need to use "options=unflag" in atlod. This saves all the flagged visibilities but keeps them flagged. Using "options=caldata" removes the flags.