CABBScheduler

Anything to do with the online software, eg. caobs, vis, spd etc.

Moderator: Mark.Wieringa

Post Reply
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

CABBScheduler

Post by Mark.Wieringa »

Max emailed around some suggestions for the scheduler - I'll repeat those here and follow up with a reply

On 31/05/2010, at 11:44 AM, Maxim Voronkov wrote:
All other comments are related to CABB scheduler. They are not
problems as such, but just improvement suggestions.

I managed to stuff up frequency a couple of times during my green
time observations (but fortunately noticed that something was wrong
quickly enough). Partly this is
because the green time observations were unexpected and I had to
create schedules in rush. However, the following changes to the
scheduler could reduce the
chance of a similar mistake made again.

1. It may be good to give a warning in the velocity to frequency
conversion dialogue window if the current scan does not look like a
program source
(i.e. marked as a calibrator by calcode or scan type is point, etc).

2. It may also be nice to give a warning if the date in the scheduler
is notably different from the current date. This is how I stuffed up
the conversion second time -
I edited the old schedule by adding zooms and the scan happened to
have an old observation date set.

3. Just as a thought, may be we want a more structured approach to
define zoom windows, i.e. not per scan which is the most general way,
but introduce a frequency
configuration page and give some kind of FREQ_ID for each scan in the
schedule. In most cases I expect that people would use 1 setup in
most cases (and
probably up to 3 frequency setups anyway given the overheads). This
approach would help to ensure that calibrators and sources are
observed using the same
setup. Otherwise there is too much scope for error, especially if one
changes the existing schedule.

Cheers,
Max
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: CABBScheduler

Post by Mark.Wieringa »

Hi Max,

Thanks for the scheduler feedback - item 1 and 2 look straightforward and will probably appear at some point. Point 3 needs some more thought, as I'm not quite sure what the current/best observing practice is here.

The current idea behind the scheduler is to pick a continuum frequency that will fit all the lines, then set the rest frequencies of the lines of interest and calculate the velocities and zoom channel numbers (for the current source and schedule date)

To calibrate this source you might want a calibrator with the same zoom channel setup. You could achieve this by copying the scan and changing the source name to a cal name. However if you now recalculate the velocities (either for this cal, or globally) it would use the cal coordinates for the conversion, which may shift the channels.

To observe another source with the same lines, you'd copy the scan, fill in the new source name and position and probably redo the velocity to zoom channel calculation to ensure the lines still fall within the zoom channels.

Because there is no longer significant bandpass structure in the zoom bands (corrected on-line), having small shifts in channel numbers between cal and source may not matter. I wonder if we even need the zoom bands for the calibrators?

I'm not sure how to define the freq_id - the conversion depends on rest frequencies, source position and source velocity (and date). Being able to copy a block of rest frequencies and zoom widths could be helpful, but you might still need to adjust the source velocities for each line and recalculate the channels. There is a lot to specify for an 8 line zoom observation...

Which of the following get stored in a freq_id: rest freq, velocity, zoom_channel,continuum freq?
Does a freq_id cover both IFs or just one?

I assume you would like freq_id's to be saved/restored with the schedule?

Cheers,

Mark
MaximVoronkov
ATCA Expert
Posts: 14
Joined: Wed Jun 02, 2010 1:11 pm

Re: CABBScheduler

Post by MaximVoronkov »

Hi Mark,
Mark.Wieringa wrote:Because there is no longer significant bandpass structure in the zoom bands (corrected on-line), having small shifts in channel numbers between cal and source may not matter. I wonder if we even need the zoom bands for the calibrators?
You're probably right. I am just trying to be conservative for the first experiments.
Mark.Wieringa wrote: I'm not sure how to define the freq_id - the conversion depends on rest frequencies, source position and source velocity (and date). Being able to copy a block of rest frequencies and zoom widths could be helpful, but you might still need to adjust the source velocities for each line and recalculate the channels. There is a lot to specify for an 8 line zoom observation...
I think the basic editing should still be done the same way, but freq_id should provide a way to lock the setup between sources. It is up to user
which source they use to define their velocity to frequency conversion (btw, it is probably practical to define velocity per source by default. In most cases people will have the same (or similar) velocity for each zoom for both IFs).
Mark.Wieringa wrote: Which of the following get stored in a freq_id: rest freq, velocity, zoom_channel,continuum freq?
Does a freq_id cover both IFs or just one?

I assume you would like freq_id's to be saved/restored with the schedule?
I guess this could be just a presentation layer. The current schedule format can be used. Just upon read, you do matching between different
scans and generate a minimum number of frequency IDs (labelled 1,2,3,4) describing the setup. It is then straightforward what happens when
you edit. There could be an additional page (to Sched and Listing) called frequencies which allows one to edit all frequency information for
every setup. By default, when one copies the scan, it should inherit the original scan's freq_id index. If the user then changes any parameter, a
new freq_id can automatically be created (and of course the user should be given an opportunity to override this link and associate the scan
with any existing freq_id). There could be an additional option to choose between old and new view as this approach may be an overkill for continuum observers. And yes, I think the setup should comprise the whole frequency setup, i.e. both IFs + all their zooms.

The usage of velocity to frequency converter is also slightly different for big surveys (what exists now pretty much replicates the web velocity
to frequency calculator, but is probably a bit easier to use). I'd say that for large surveys with zooms observers would try to design the
minimum number of setups covering all their sources. I always used my own velocity to frequency calculator which I could run in a batch
mode directly on my source list and get the sky frequency for each source. Then I can group sources in sky frequency (with CABB there
definitely would be a decision whether one uses an extra zoom to increase velocity coverage or to observe something else). Some other people
I know just calculate frequencies for a few sources known to be outliers in velocity. This approach works fine for cm-wavelength observations
where the velocity coverage is not too bad, and/or for sources away from the Galactic centre where there is a large number of sources
with similar velocities. So we may think about some way to visualise velocity coverage for different sources (e.g. in a way similar to mops_vis.)

BTW, it seems to me I spotted another bug/feature of the scheduler.
Look at my test schedule cx110g343m25.sch I used last night, scan 7. If you remove one zoom by erasing the frequency and typing enter -
the velocity coverage disappears, so I assumed that this zoom had been removed. However, if you then look at another scan and come
back, it reappeared again. If you remove both frequency and fliterbank channel, the zoom is removed. It may be good to add check box
"enable" and don't rely on the emptyness or otherwise of a particular field.

Cheers,
Max
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: CABBScheduler

Post by Mark.Wieringa »

Hi Max,

I've implemented the easy bits - warnings for calculating velocities on calibrators or for schedules with old dates. Also fixed the zoom erase bug - if you erase either the zoom freq or the zoom channel the zoom will be removed. Still mulling over the best way to implement freq_id's.

I've installed this version as cabb_next, i.e., http://www.narrabri.atnf.csiro.au/obser ... cabb_next/" onclick="window.open(this.href);return false; for now, will move it to current early next week.

Cheers,

Mark
MaximVoronkov
ATCA Expert
Posts: 14
Joined: Wed Jun 02, 2010 1:11 pm

Re: CABBScheduler

Post by MaximVoronkov »

Great. This freq_id issue can wait until we get more experience with zooms. I suspect the first projects will be relatively simple from scheduling point of view.

As a simple version, there could be a global option (and I'd suggest to make it default) which enforces the same frequency setup for all scans (i.e. just one setup). And just to
avoid confusion when this option is on, frequency pages can be moved a level up (i.e. to a separate Tab in addition to Sched and Listing). But there is no rush with this.
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: CABBScheduler

Post by Mark.Wieringa »

Hi Max,

the cabb_next version ( http://www.narrabri.atnf.csiro.au/obser ... /cabb_next" onclick="window.open(this.href);return false; ) now has two new features.

1. Master/Slave frequency settings - select Advanced view in the Tools menu to display a field called FreqConfig. This lets you select a scan as the 'master' for frequency settings, other scans can select the corresponding 'slave' setting to get the same setup. 'null' leaves things as they were - independent scans.

2. The velocity calculation box now has a 'fix' tickbox to lock the velocity for all zooms and both frequencies to be the same as the first zoom. Note that you still need to hit the apply button to make this stick (like any other changes made on the velocity panel).

The two features can be combined to e.g., set the first scan as the 'Master1' template freq config with a single source velocity. All subsequent scans made with newscan will be 'Slave1' and have the same setup.

The FreqConfig also appears in the listing now.

I'll leave this in the 'cabb_next' slot for a little while so it can be tested before inflicting it on everyone.

Cheers,

Mark
MaximVoronkov
ATCA Expert
Posts: 14
Joined: Wed Jun 02, 2010 1:11 pm

Re: CABBScheduler

Post by MaximVoronkov »

Great! Thanks Mark.
MaximVoronkov
ATCA Expert
Posts: 14
Joined: Wed Jun 02, 2010 1:11 pm

Re: CABBScheduler

Post by MaximVoronkov »

Hi Mark,

Just to follow the discussion over the morning tea about zoom setups. I thought a bit more about this and have the following suggestions which should be relatively straightforward to implement.

1. Allow the user to specify the velocity in the catalogue file (i.e. if there are extra columns after J2000 they could be treated as radial velocity + frame for this particular source). Currently these extra columns in the catalogue are ignored. The velocity given in the catalogue can be the default velocity for all zoom configurations, unless the user edits it manually via the appropriate page.

2. For all "slave" configurations show the radial velocity of a particular source if it is set and the velocities corresponding to the edge of each zoom band. It can be done either graphically or just by showing numbers.

I think these two features would automate planning a lot as users can play with their zoom setups and see how sources fit into this. One complication I see is that the velocity is currently a part of the frequency configuration (if I am right), so we need to track source velocities separately even if it is a slave configuration.

Cheers,
Max
MaximVoronkov
ATCA Expert
Posts: 14
Joined: Wed Jun 02, 2010 1:11 pm

Re: CABBScheduler

Post by MaximVoronkov »

Hi Mark,

First, thanks again for the Master/Slave feature. Without it, we probably wouldn't be able to do our yesterday's C1820 run successfully given the amount of other issues.

I think something funny is still going on with the zoom config page. One thing which seems certain (because I've seen it
reappearing several times) is that the state of the "fixed" flag for the first configuration does not seem to be reloaded
from the file correctly. Another issue may be partially because of my misunderstanding of the "fixed" flag. Our use case
was as follows.

The frequencies of all zooms were chosen to cover the lines of interest and had to stay fixed. However, to address some of the
problems found during observations (uneven distribution of the computing load between DFBs) we had to shift the whole zoom
arrangement for a few DFB channels by altering central frequency for the offending IF by 1-2 MHz. With the "fixed" flag set to fix
the zoom frequency, only first zoom of this IF seems to be recalculated. And I am not sure whether CABB channel number
changed following the change of the central frequency. At least when I changed zoom frequency slightly, the channel number
was not changed (if I recall correctly). Because I wasn't sure what is actually used by caobs zoom frequency or CABB channel,
I had to clear up everything and retype all values just in case. But this approach seems to be working as we saw a few strong
lines where they were supposed to be at the time of observation.

Another use case I've seen/had before is to fix the arrangement of CABB channels and change the central frequency to get right
zoom frequencies. I guess it is related to another position of the "fixed" flag. Probably we can have this page further improved by
graying out those input fields which are slaved to other input fields and calculated if those fields change. The "fixed" flag can be
a three position switch in this case choosing whether central frequency, CABB channel or zoom frequencies are calculated.

Cheers,
Max
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: CABBScheduler

Post by Mark.Wieringa »

Hi Max,

An update on you queries.
I've implemented the feature to read velocities from a catalog (not yet available though).
I'm looking at adding support for calculating slave config velocities.

Thanks for the feedback on Master/Slave configs.

The "fixed" button does indeed not get saved. It keeps the value you've last set it to in a CABBScheduler instance (for each Freq).
The default state for the fixed button is to keep the Cont freq fixed when calculating zoom frequencies and channels. In this mode it only changes when you type in a different number.
The "fixed line1" state fixes the channel the first spectral line appears in and shifts the continuum frequency around to achieve this. If you change the velocity the Cont Freq will shift to keep the line in the same channel, all other channels should update too. The feature was requested by Jim Caswell. I notice that it currently doesn't work well if you change the zoom width. The implementation adds a lot of complexity to the code with much potential for bugs. Maybe we're better of without it?

If you manually enter the zoom frequencies, then these are saved in the schedule as SkyFrequencies and never changed, if you change the Cont freq, the channels will be recalculated to match the SkyFrequencies.
If you enter the velocity and spectral line/rest freq in the velo panel and press Apply, the SkyFrequency is calculated and the channel calculated from that. This route lets you update the schedule for another day, by setting the new date in the scan parameter panel and then choosing Tools - recalc freq/velo.

Which fields are calculated depends on where you type: if you type in the zoom freq field the channel field will be computed and v.v., if you type in the Cont freq field, the zoom channels are updated.

Caobs and CABB just look at the the Cont frequencies and the channel numbers for the zooms in each band. The rest frequencies are passed along too, but rpfits only has room to record one at present.

Cheers,

Mark
Post Reply