Issue with 'Divide & Conquer', then UVCAT

Is MIRIAD being a pain? Let us know your experience.

Moderator: Mark.Wieringa

Post Reply
and460
Posts: 79
Joined: Thu Sep 01, 2011 12:23 pm

Issue with 'Divide & Conquer', then UVCAT

Post by and460 »

Hi guys,

I'll just preface this by saying that for some reason, the ATCA forum refuses to let me search for anything anymore, telling me that 'search is not working at this time, please try again in a few minutes' (but subsequent attempts are always met with the same response). Hopefully then this topic has not been addressed before - I had a scan through but could not find anything on it.

Anyway, I'm trying to image a calibrator in 32MHz multi-freq chunks through the 16cm CABB band after using the 'divide and conquer' approach to calibrating - UVSPLITing into 128MHz chunks and calibrating those individually. Since file handing is made far easier by using the 'line' parameter in INVERT to image channels of interest through the band rather than further splitting each of the 128MHz chunks down to 32MHz chunks, I thought I would just UVCAT the 128 MHz uv files back into one large uv file so I could do the imaging in this manner.

However, now when I use the line parameter to select the channels that I want for invert, the task fails whenever I try to select channels beyond #128. I.e. INVERT with line=channel,32,97,1 works without a problem, but line=channel,32,129,1 fails saying

### Fatal Error: No data to process

I thought that perhaps it was a flagging issue, so I ran UVFSTATS, but it returns:

### Fatal Error: Illegal channel range specified, in UVREAD

It seems like UVCAT is not behaving how I thought it would. Any idea why it is going wrong, and what the best way forward is?
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: Issue with 'Divide & Conquer', then UVCAT

Post by ste616 »

Hi Craig,

Yes, you've misunderstood what uvcat does.

uvcat simply takes one file and pastes it on to the end of another file. It does not check if times are in order or duplicated, so does not combine data together based on the time it was obtained.

In your case, now you just have a set of data with 128 channels that spans the same time range over and over again. This, incidentally, will cause troubles with a number of tasks that expect that all data is in time order.

To stitch data back together side-by-side, you will want to use the uvglue task. It takes a bit more setting up (you will probably need to do some symbolic linking or renaming), but the documentation should make it clear how to proceed (or ask for help here!).

There is an alternative though. It is no longer our recommendation that "divide and conquer" be taken so literally as to actually break up the data into smaller bandwidth chunks. Instead, you can use things like "nfbin" in gpcal, and (as you mention) the "line" parameter to do the splitting while keeping all the data available. In terms of calibration, I have not personally seen a single instance where calibration of individual chunks has been more effective than calibration of the entire bandwidth at once.
cheers
Jamie Stevens
ATCA Senior System Scientist
and460
Posts: 79
Joined: Thu Sep 01, 2011 12:23 pm

Re: Issue with 'Divide & Conquer', then UVCAT

Post by and460 »

Thanks Jamie - much appreciated.

You said that "In terms of calibration, I have not personally seen a single instance where calibration of individual chunks has been more effective than calibration of the entire bandwidth at once". Just to clarify - you mean that you've never seen an instance where literally splitting the band into individual chunks has provided superior calibration solutions to using 'nfbin' to split the band internally to calibrate, right? I assume you don't mean that splitting the band for the calibration at all is entirely useless in terms of the end result?

Also - not sure if you saw my other post in the 'calibration' section of the forum on the wobbles I'm getting on the amplitudes of individual baselines after bandpass cal of 1934 when examined with UVSPEC, but I'm finding it a bit disturbing that they are there, yet impossible to avoid no matter what I do. Any advice?

Thanks for everything...

Cheers,

Craig.
and460
Posts: 79
Joined: Thu Sep 01, 2011 12:23 pm

Re: Issue with 'Divide & Conquer', then UVCAT

Post by and460 »

Also - I want to combine UV data from two different epochs (that has been separately calibrated) for imaging. UVCAT is the correct task for this purpose, is it not?
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: Issue with 'Divide & Conquer', then UVCAT

Post by ste616 »

Hi Craig,

Clarification: yes, splitting the band for calibration can be useful, but it is no longer necessary to actually split the band into separate files for calibration.

I have seen your 'wobbles' post, and am investigating it with some of my own data. I'll let you know.

Regarding your latest post: yes, uvcat is ideal for pasting together separately-calibrated epochs into a single dataset. For most purposes however, this is not necessary, as the Miriad tasks which are normally used to combine multiple epochs (eg. invert, uvspec) can all accept multiple files for their 'vis' argument.
cheers
Jamie Stevens
ATCA Senior System Scientist
and460
Posts: 79
Joined: Thu Sep 01, 2011 12:23 pm

Re: Issue with 'Divide & Conquer', then UVCAT

Post by and460 »

Thanks for the help Jamie, and thanks for looking into the 'wobbles' issue for me! I owe you multiple beers next time I'm up...
Post Reply