CABB Zoom problem
Posted: Fri Apr 11, 2014 3:51 pm
It has recently been observed that CABB zooms occasionally suffer what appears to be a corrupt first correlator cycle. This post will illustrate the problem, describe how to figure out whether your data may have been affected, and how to use Miriad to flag out affected data.
An illustration of the effect comes from Shari Breen, who observed maser sources strong enough to clearly see in each 10 second correlator cycle. She found that in the first cycle of most scans on these masers the emission seemed to be shifted in frequency compared to where it was in the other cycles. We can see this by looking at a waterfall plot of the data, as generated by pgflag:
In this plot the channel number is increasing along the x-axis, while the time is increasing along the y-axis. The plot also makes clear that for two scans the problem doesn't appear, and in the last scan the problem has a different magnitude to the earlier scans.
Our correlator engineers are investigating this issue, and as yet a fix is not available. It has been found that this bug has existed since zooms were implemented, but obviously it requires a strong source and fast slews to be readily visible. Anyone who has used CABB zooms may be affected however.
The corruption is isolated to the first cycle of any particular scan, and the correlator engineers have told us that it looks like 50% of first cycles are affected.
Your observations may be affected if:
To flag out the affected cycles most likely requires that you redo your data reduction from scratch. When you load your data with atlod, all the off-source cycles will be discarded; this is done to save space on the disk. However this also means that Miriad cannot later determine if the first cycle in a scan is actually the first correlator cycle, or if it is simply the first cycle while tracking the source. For example, we can look at the partial output of uvindex for a random observation loaded with atlod in the normal way:
As you can see, the first cycle on 1921-293 present in the data file starts at 22:10:00 (these are 10 second cycles), while the first cycle labelled as 1934-638 is at 22:22:40. We could easily flag out both of these first cycles, but do we need to? There is no way to tell in this case.
The "unflag" option in atlod makes it keep all the flagged cycles, although confusingly it won't actually unflag those cycles (which is good!); it simply means that all the data will be preserved in the output of atlod. Let's look at the same observation as before, loaded with atlod's "unflag" option:
Now you can see that 1921-293 would be affected by this problem: we now know the first cycle on 1921-293 is also the first correlator cycle in the scan. since the time is the same as before at 22:10:00. But the first correlator cycle for the 1934-638 scan was at 22:20:20, which is 2 minutes and 20 seconds before the first cycle while actually tracking the source (ie. the telescope slewed for 2:40 seconds between 1921-293 and 1934-638; the correlator starts cycling after the equivalent of 2 cycles time after the start of a scan). Therefore we don't need to flag out any further data for this 1934-638 scan.
So how do we do the flagging? After using the "unflag" option in atlod, the Miriad task "qvack" can be used to flag the first cycle of every scan.
For example, the atlod step might look like:
Then to flag every first correlator cycle:
This tells qvack to always flag the first cycle of every scan. If the first cycle is already flagged, most likely because the telescope wasn't yet tracking, then you won't lose any further data. You are now assured however that your data will not longer contain corrupt first cycles from CABB. From this point your reduction should be as per usual.
More information will be provided when we know if this issue can be fixed. Please feel free to reply to this post if you have questions.
An illustration of the effect comes from Shari Breen, who observed maser sources strong enough to clearly see in each 10 second correlator cycle. She found that in the first cycle of most scans on these masers the emission seemed to be shifted in frequency compared to where it was in the other cycles. We can see this by looking at a waterfall plot of the data, as generated by pgflag:
In this plot the channel number is increasing along the x-axis, while the time is increasing along the y-axis. The plot also makes clear that for two scans the problem doesn't appear, and in the last scan the problem has a different magnitude to the earlier scans.
Our correlator engineers are investigating this issue, and as yet a fix is not available. It has been found that this bug has existed since zooms were implemented, but obviously it requires a strong source and fast slews to be readily visible. Anyone who has used CABB zooms may be affected however.
The corruption is isolated to the first cycle of any particular scan, and the correlator engineers have told us that it looks like 50% of first cycles are affected.
Your observations may be affected if:
- You are using CABB zooms, either 1 MHz or 64 MHz (although only 1 MHz has shown definitive evidence of the corruption), AND
- Your first cycle of a scan contains on-source visibilities. If it takes more than 30 seconds to slew to your source, then this potentially corrupt first cycle will automatically flagged as off-source and you won't be affected.
To flag out the affected cycles most likely requires that you redo your data reduction from scratch. When you load your data with atlod, all the off-source cycles will be discarded; this is done to save space on the disk. However this also means that Miriad cannot later determine if the first cycle in a scan is actually the first correlator cycle, or if it is simply the first cycle while tracking the source. For example, we can look at the partial output of uvindex for a random observation loaded with atlod in the normal way:
Code: Select all
Time Source Antennas Spectral Wideband Freq Record
Name Calcode Channels Channels Config No.
14MAR14:22:10:04.9 1921-293 c 6 8196 0 1 1
14MAR14:22:22:44.9 1934-638 c 6 8196 0 1 3601
The "unflag" option in atlod makes it keep all the flagged cycles, although confusingly it won't actually unflag those cycles (which is good!); it simply means that all the data will be preserved in the output of atlod. Let's look at the same observation as before, loaded with atlod's "unflag" option:
Code: Select all
Time Source Antennas Spectral Wideband Freq Record
Name Calcode Channels Channels Config No.
14MAR14:22:10:04.9 1921-293 c 6 8196 0 1 1
14MAR14:22:20:24.9 1934-638 c 6 8196 0 1 3601
So how do we do the flagging? After using the "unflag" option in atlod, the Miriad task "qvack" can be used to flag the first cycle of every scan.
For example, the atlod step might look like:
Code: Select all
atlod "in=*.C007" out=c007.uv options=birdie,rfiflag,xycorr,opcorr,noauto,unflag
Code: Select all
qvack vis=c007.uv interval=0.1 force=0 mode=source
More information will be provided when we know if this issue can be fixed. Please feel free to reply to this post if you have questions.