Problem selecting frequency ranges with CABB data

[JamesAllison]

Hi All,

I am trying to select specific frequency ranges in my CABB Zoom-mode data in order
to flag out bad channels due to RFI. The RFI spikes appear at different times
and in specific baseline-channels.

Since I have a large data set I would like to construct a flag file that contains all of the
specified flagged data, then read it into the select option in uvflag.

However when I set the following in Miriad:

> select = frequency(1.34,1.36)

for data in the band 1.31 to 1.37 GHz (64MHz Zoom band), the data are not correctly selected.
For example if I were to set this option in uvspec, I would get the following error:

### Fatal Error: Nothing to plot

I'm not sure where I am going wrong here. If I set the frequency range to encompass the whole band (e.g.
select = frequency(1.3,1.4)) then no error appears and the whole band is plotted.

I would be very grateful if someone knows a trick around this. Unfortunately I cannot use the line option
since I need to read in a large select file to flag specific data.

Cheers,

James

[ste616]

Hi James,

I'm afraid the select keyword "frequency" can only ever select the entire band. What it does is find the data that has its first frequency (ie. the frequency of channel 1) between the two values you pass to it. So

Code: Select all

select=frequency(1.34,1.36)

would only work if your data had a frequency in channel 1 of between 1.34 and 1.36 GHz. And if it does find it, it will return the entire spectrum, as select can only be used to select a visibility, not a portion of a visibility.

So you will have to use the line keyword to select specific frequency ranges. I don't understand exactly why you can't use line - can you explain your situation a bit more?

[JamesAllison]

Hi Jamie,

Thanks for your reply,

So to clarify, I am looking to construct a file of flags that I can store and re-run on the same
data set. I am developing an automated reduction pipeline for my data, during which "uvflag"
is called and the "select" option used to read the flag file.

I have already constructed flags that remove particular time and source combinations, and the
next step was to remove channels containing RFI. Previously I have been using mirflag, however
I was uncomfortable using this task blindly on a large data set, especially since I am working
on spectral line data.

If you have any suggestions that would be great!

James

[ste616]

Hi James,

Yes, this is a tricky problem to solve using standard Miriad tasks.

In my experience with making pipelines, I can recommend two possibilities:

• Make a script that will run uvflag repeatedly for each separate group of channels that need to be flagged. This method has the advantage that it will work independent of how the data was loaded.
• Do all the flagging manually (or with a script if you want), and then copy the 'flags' file from the dataset directory. Later if you want to replicate the flagging, simply copy the 'flags' file back to the dataset directory. The flags file is very small compared to the size of the data, and should be heavily compressible with something like gzip. It would therefore be very easy to archive and distribute to other users. The disadvantage is that if the dataset varies at all from the original set (due to different loading options for example), then the flagging will likely fail.

I hope this helps. Let us know how you go!

[JamesAllison]

Hi Jamie,

Thanks for the summary - since I'm relatively new to spectral line reduction with Miriad
its good to know the options available.

Will let you know how things go!