Hi guys
the link at 64-bit-mirflag-memory-hogging-t29.html doesn't appear to be working(?)
Also can you confirm that you are running mirflag AFTER doing a quick calibration? Is that required? I am trying to use mirflag and its flagging 99% of the data (20% flagged from birdie,rfiflag). That's alot of flagging!
thanks
Ilana
Flagging with mirflag
Moderator: Mark.Wieringa
-
ste616
- Site Admin
- Posts: 220
- Joined: Thu Feb 04, 2010 3:27 pm
- Location: Paul Wild Observatory Narrabri NSW
Re: Flagging with mirflag
Hi Ilana,
Yes, I only run mirflag after doing an initial calibration, at least with mfcal. Sometimes I have also found it will flag pretty much the entire dataset. In cases like these, I end up flagging manually, as I don't have a quick explanation of why it does this. Perhaps Emil can shed some more light on it.
Yes, I only run mirflag after doing an initial calibration, at least with mfcal. Sometimes I have also found it will flag pretty much the entire dataset. In cases like these, I end up flagging manually, as I don't have a quick explanation of why it does this. Perhaps Emil can shed some more light on it.
cheers
Jamie Stevens
ATCA Senior System Scientist
Jamie Stevens
ATCA Senior System Scientist
Re: Flagging with mirflag
Hi Ilana/Jamie,
Sorry, I have not been getting notifications for some reason so I wasn't aware of this posting.
With regards to your question, it depends a lot on the nature of the source. For bright, uncalibrated sources, mirflag will do a terrible job if you attempt to do any amplitude-based flagging because the band will look horrendous and the time-variations will also cause problems. In general though, you can get away with a lot by always adding the medsed and medtcal options - these attempt to remove any frequency-based or time-based variations based on median statistics per channel (for frequency-based) and per time interval (for time-based). Note that you should always use medsed if you are using amp-based flagging because this will remove any spectral curvature as well (without do this amp-based flagging becomes meaningless because it will flag all channels that are above or below some median value in the band). Note that these options will generally only be useful for unresolved calibrator sources. For the target, you will need to apply the calibration and then depending on the nature of the target field use rms-based flagging (if it is a detection experiment or the target is complex) and/or amp-based if the target is compact and bright.
Cheers,
Emil.
Sorry, I have not been getting notifications for some reason so I wasn't aware of this posting.
With regards to your question, it depends a lot on the nature of the source. For bright, uncalibrated sources, mirflag will do a terrible job if you attempt to do any amplitude-based flagging because the band will look horrendous and the time-variations will also cause problems. In general though, you can get away with a lot by always adding the medsed and medtcal options - these attempt to remove any frequency-based or time-based variations based on median statistics per channel (for frequency-based) and per time interval (for time-based). Note that you should always use medsed if you are using amp-based flagging because this will remove any spectral curvature as well (without do this amp-based flagging becomes meaningless because it will flag all channels that are above or below some median value in the band). Note that these options will generally only be useful for unresolved calibrator sources. For the target, you will need to apply the calibration and then depending on the nature of the target field use rms-based flagging (if it is a detection experiment or the target is complex) and/or amp-based if the target is compact and bright.
Cheers,
Emil.