Superuniform weighting and CABB - avoid!

Is MIRIAD being a pain? Let us know your experience.

Moderator: Mark.Wieringa

Post Reply
len067
ATCA Expert
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

Superuniform weighting and CABB - avoid!

Post by len067 »

Hi All,

I've just been running some tests with ATLAS data taken with CABB and have found (well, apparently rediscovered) a problem with superuniform weighting in the task invert.

I took a 600 MHz band at 20 cm and tried to image this with superuniform weighting and mfclean. Using this data I reached an RMS image noise of ~105 uJy/beam for ~52 minutes of on-source data. We were getting much better than this pre-CABB with only 2x13x8 = 208 MHz of bandwidth so I was rather underwhelmed with the result.

As a quick test I made the CABB data look like my pre-CABB data i.e. 26 x 8 MHz channels (instead of 600 x 1 MHz channels) and at the same frequency as my pre-CABB data. After imaging using exactly the same settings the resulting RMS image noise was ~75 uJy/beam - a 25% improvement in sensitivity with ~60% less data ;-)

Not knowing if it was the restricted bandwidth or the averaging that improved my sensitivity I thought I'd try to image 3 x 200 MHz bands (1200-1400, 1400-1600, 1600-1800) averaged to 8 MHz channels, the same 3 x 200 MHz bands unaveraged, the full band (1200-1800) averaged to 8 MHz channels and also the full band unaveraged. In ALL cases, the averaged data resulted in a sensitivity that was 25-50% better than the unaveraged data of the same band. Interestingly, averaging the entire 600 MHz band into one 600 MHz channel improved the sensitivity by a further 25% suggesting that even our pre-CABB data was probably afflicted with this problem! Note that in each case the theoretical rms noise reported by invert also reflected the same behaviour, so it had nothing to do with the actual quality of the visibility data.

Using robustness weighting (trying both -2 and +2), the noise reported by invert (and that measured in the final image map) was more closely associated with the actual amount of data rather than how much it was averaged. So this seemed to work fine.

After discussing this particular odd behaviour with Tim Cornwell, he pointed out that there was an issue with the way super-uniform weighting was implemented in Miriad (ie. the way it grouped nearby visibilities during gridding) and that this is probably what is causing the unusual noise characteristics ... and why it seemed to be worse with unaveraged data (because with mfs the points would be close to each other in the u-v plane) than with averaged data.

Cheers,

Emil.
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: Superuniform weighting and CABB - avoid!

Post by Mark.Wieringa »

Hi Emil,

so just to confirm.. Does this only happen if you specify a non zero number for the sup parameter?
Sup = 0 is natural, Sup unset is uniform, Sup between 0 and field size is super-uniform.

Cheers,

Mark
len067
ATCA Expert
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

Re: Superuniform weighting and CABB - avoid!

Post by len067 »

Hi Mark,

I probably should've been a bit more clear with this - sorry. I haven't fully tested various values of sup. For our ATLAS processing we were using sup=4500 with an image size of 7995x7995 pixels and a cell size of 2.0. For the two robust weighting tests I left sup unset and used robust=-2 and robust=+2.

Cheers,

Emil.
len067
ATCA Expert
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

Re: Superuniform weighting and CABB - avoid!

Post by len067 »

Hi Mark,

If it helps, I have just tried a number of different sup settings for the unaveraged (1 MHz channels over 600 MHz) and averaged (8 Mhz channels over 600 MHz) data and here is the rms reported by invert in each case:

Code: Select all

sup    unaveraged  averaged
       Jy/beam     Jy/beam
0      6.60E-05    6.15E-05
100    3.10E-04    1.53E-04
200    2.82E-04    2.18E-04
500    3.04E-04    1.86E-04
1000   2.54E-04    1.54E-04
2000   2.05E-04    1.30E-04
Cheers,

Emil.
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: Superuniform weighting and CABB - avoid!

Post by Mark.Wieringa »

Hi Emil,

I (finally) had a look at the invert code last week. The way uniform weighting in Miriad works is as follows:
1. Use the value of sup to decide on a grid spacing in the uv plane ~(1/sup)
2. Grid all the weights into this grid
3. Weigh the data in each grid cell with 1/sum(weights)

Leaving sup unset is equivalent to setting it to the field size.
Setting sup > fieldsize gives smaller uv grid cells, i.e., fewer points per cell, in the extreme case this leads to one point per cell, which is equivalent to natural weighting. This explains why just increasing the images size (with fixed cellsize) tends to give lower noise images - we're tending towards natural weighting as the image gets larger.
Setting sup< fieldsize gives bigger uv grid cells, i.e., more points per cell. This smooths out the small scale weights distribution, but as cells get bigger, less structure is taken into account, again making it more like natural weighting.

Each of these options makes the uv grid more uniform in some sense. However, the drawback is increased noise - single points in the uv plane get the same weight as a large clump of points falling in a single cell. The higher the contrast in the uv plane, the larger the noise increase. This explains why mfs images of CABB data suffer much greater noise increase than mfs images of preCABB (or frequency averaged) data - the contrast is 64 times bigger. If a data point happens to fall in a single cell in CABB data it gets 64 times more weight than in the preCABB data, greatly increasing its contribution to the noise (and stripe artefacts in the image).

The solution to all this is to add robust weighting whenever uniform (sup>0) is used. It was designed to solve exactly this problem - excessive weight of isolated data points. Instead of just using the inverse sum of weights in each cell, it adds a constant term : 1/(c+sum(weights)). This gives isolated points less weight, depending on the value of the constant.
The robust parameter lets you vary the constant in factors of 100 (between e.g. robust=0 and robust=1). Robust=-2 is essentially uniform, Robust=2 nearly natural weighting.

For CABB mfs data with uniform weighting the most useful values are 0, 0.5 and 1. Robust=0 still has slightly increased noise compared to preCABB data, from 0.5 upwards the effect is gone. A bit of experimenting will let you choose the value that gives you the beam size and noise level compromise you want.

I've added a warning message to invert, to alert people to the dangers of using uniform weighting without adding the robust modification.
len067
ATCA Expert
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

Re: Superuniform weighting and CABB - avoid!

Post by len067 »

Hi Mark,

Thanks for looking into this. I may not get a chance to test this out in the short-term but I'll let others in ATLAS know about this.

Cheers,

Emil.
cAh
Posts: 5
Joined: Tue May 04, 2010 12:57 pm

Re: Superuniform weighting and CABB - avoid!

Post by cAh »

Hi all,

Apologies for not posting to this forum when I first spoke with Tim Cornwell (who in turn spoke with Bob Sault) about what is going on with this issue (ahem...6 months ago).

Below I reproduce some text from the ATLAS Data Release 2 draft manuscript (yet to be submitted to a journal), which is mostly in the form of a footnote:

"We note that we were limited in our ability to make full use of the available sensitivity of our CDF-S data because of an error in MIRIAD's implementation of superuniform weighting, which caused visibilities to be incorrectly gridded in the uv-plane. --> FOOTNOTE: Because of the lack of documentation on this issue, we briefly describe it here. In uniform weighting, pixels in the uv-plane are treated as cells, where visibilities assigned to each cell are weighted by the summed weights of the visibility samples gathered in that cell. This will minimise sidelobes over the entire field of view. In superuniform weighting, the cell size is allowed to decouple from the pixel size in order to minimise sidelobes over a fraction of the field of view, f, where the cell size is now 1/f. Importantly, there should now be only **one** cell placed in the uv-plane. Thus in superuniform weighting, the cell size in which uniform weighting is applied is increased, such that all visibilities located within 1/f of the nominal pixel are weighted by their summed weights. This approach is implemented in AIPS and CASA. The approach taken in MIRIAD is to form **multiple** cells of size 1/f and uniformly weight within each of them, resulting in both the production of diffraction peaks in the dirty beam and artificially decreased sensitivity in the output images. Given sufficient cleaning, the former effect will be marginalized. (T. Cornwell 2011, private communication)."

I think the solution described by Mark to incorporate robustness sounds right, because the new weighting factors mitigate against the incorrect super-uniform implementation in MIRIAD.

If anyone has any concerns with the explanation above then please let me know before I submit the ATLAS DR2 manuscript (~mid Jan 2012).

Chris Hales
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: Superuniform weighting and CABB - avoid!

Post by Mark.Wieringa »

Hi Chris,

thanks for that contribution - this encouraged me to have a peek at the AIPS code.
It appears to me that what it actually does (using the uvbox parameter) is the following:
- like Miriad it uses a weights grid with a size that can be decoupled from the image grid size (using uvsize in AIPS and sup in Miriad)
- Miriad then just sums the weights in the (usually larger) cells and uses the inverse to weigh the data, AIPS can do the same but also lets you specify a uvbox size (and weight function) that adds in the counts from surrounding cells (with optional reduction in weight). Thus it smooths out the extreme density variations in the weights grid, reducing striping and noise increase much like the robust scheme.

I might have another look at invert and see if I can make the sup parameter do similar smoothing instead of increasing the weights grid cell size. I am a bit reluctant to change the image that invert produces when running an old script though.

Cheers,

MarkW
Post Reply