How do you find the total time on source?

Is MIRIAD being a pain? Let us know your experience.

Moderator: Mark.Wieringa

Post Reply
nickill
Posts: 2
Joined: Wed Feb 12, 2020 12:37 pm

How do you find the total time on source?

Post by nickill »

I have a large dataset that is the result of merging many uv datasets collected over the course of a few years. The information i want is quite basic but has proved difficult to get consistent accurate results from. I want to know how much time we have on the source over this period of time. We expect a value in the region of ~50h.
  • When i tried taking the value of tau from uvspec with options=nobase,avall and an interval covering the whole period i got 4578.1min (572h)

    When i did the same but with options=nobase i got 4578.1min (76.3h)

    When i check the file with uvindex (as suggested here) I get (2295min) 38.25h. @Mark.Wieringa mentions that "You need to run this on a single source file too.". Is this to say that uvindex won't give an accurate result for a merged dataset and i should run it on individual datasets and sum the results?

    When i log uvplt time vs amplitude and take the number of timestamps and multiply by the 10s integration length i get (140103min) 2335h (obviously incorrect)

    When i log uvplt time vs amplitude and take the number of unique timestamps and multiply by the 10s integration length i get (1123.8min) 18.73h

    When i take the number of correlations (5169911400) and multiply by the 10s integration, divide out the number of baselines (doesn’t account for antenna dropouts) and the number of channels (6145) i get 6677min (111.28h)

    I also attempted running gethd on the data set with in=PATH_TO_DATASET/inttime but it returned no errors nor values.
Does anyone have any ideas on why these all return different values? I assume they are all measuring slightly different things but without knowing what the differences are I'm not sure which value i should be using.

Thanks!
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: How do you find the total time on source?

Post by ste616 »

Hi Nick,

My recommendation is to use the task uvfstats. What you are after is the number of unflagged correlations in a single IF, on a single baseline, for a single pol, in a single channel. Multiplying this by the cycle time (assuming it's constant over the duration of all the correlations) will result in the actual integration time.

The uvfstats task can help you obtain this number. You will want to split the dataset into IF and source-specific chunks (so you have something like source.5500). Then find the channel with the most number of correlations using:

Code: Select all

uvfstats vis=source.5500 mode=channel options=absolute,unflagged
Once you have that channel number (let's say channel 150), work out which baseline has the highest number of correlations:

Code: Select all

uvfstats vis=source.5500 mode=baseline line=channel,1,150 options=absolute,unflagged
Assuming that baseline 1-2 has the most number of correlations, then display the correlations per pol:

Code: Select all

uvfstats vis=source.5500 mode=stokes line=channel,1,150 "select=ant(1)(2)" options=absolute,unflagged
You'll probably want to make sure that XX and YY are the same (or choose the lowest value) since you'll need both to form Stokes I. This is the number of integration cycles you have for this source in this IF.
cheers
Jamie Stevens
ATCA Senior System Scientist
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: How do you find the total time on source?

Post by Mark.Wieringa »

Hi Nick,

Jamie's method will give the precise answer and uvindex will give you a quick estimate of total time on source (unflagged and flagged).
Uvindex also gives you the number of 'records' per polarisation - each record here is a spectrum for a single polarisation. If you know the integration time is the same (usually 10s) across all data, you can cross check the number you get by taking # records * 10s / # baselines (usually 15). You can also run uvfstats to see how much of this data is flagged (in %).

It looks like the first uvspec number is about 15 times too high, so it must be counting each baseline separately.
With uvplt, make sure you select a single polarisation to avoid double counting, the number of 'visibilities read' should then be the same as the number of records in uvindex. Small differences my be due to flagging.

Cheers,

Mark
nickill
Posts: 2
Joined: Wed Feb 12, 2020 12:37 pm

Re: How do you find the total time on source?

Post by nickill »

Thank you Jamie for the method it worked perfectly! Thank you Mark for the explanation!

:D

Cheers,
Nick
Post Reply