Page 1 of 1

ATLOD returning "File too large"

Posted: Mon Jun 12, 2017 10:50 am
by Stefan
Hi,

I've been trying to load multiple RPFITS files (4) through atlod with a total of four sources, pasted below from the ATOA.

▶ 2011-12-12_1700.C2457 1714.43 no C2457 2011-12-13 6A A Baker 0214-522 0302-623 ACT0245-5302
▶ 2011-12-12_1335.C2457 4094.90 no C2457 2011-12-13 6A A Baker 0214-522 ACT0245-5302
▶ 2011-12-12_0643.C2457 4094.93 no C2457 2011-12-12 6A A Baker 0214-522 1934-638 ACT0245-5302
▶ 2011-12-12_1010.C2457 4094.92 no C2457 2011-12-12 6A A Baker 0214-522 ACT0245-5302

Upon attempting to load via atlod I receive "Error writing variable-value for corr, in UVPUTVR: File too large" with the following settings:

Code: Select all

atlod in=*.C2457 out=c2457.uv ifsel=1 options=birdie,rfiflag,xycorr
I've also tried simply loading each file in separately and then using uvsplit and uvcat to get single source miriad files but I am greeted with the same error. Will this simply require me to split the main source (ACT0245-5302) into multiple miriad files or is there a way around this?

Cheers,
Stefan

Re: ATLOD returning "File too large"

Posted: Tue Jun 13, 2017 10:17 am
by Mark.Wieringa
Hi Stefan,

What version of Miriad are you using?
Please check that you have the current ATNF version of Miriad on a 64 bit linux or MacOS machine.
If you find that you cannot create files larger than 4 GB, you must have an old OS or Miriad version.

My atlod prints:
atlod: Revision 1.53, 2016/08/25 23:02:21 UTC

I tried to replicate your problem and successfully loaded the 4 files you indicated into a single uv file.

You may also want to add the flag noauto to the options, to throw away the autocorrelations.

Cheers,

Mark

Re: ATLOD returning "File too large"

Posted: Tue Jun 13, 2017 11:51 am
by Stefan
Hi Mark,

I have the latest darwin_x86_64 miriad version which has atlod version "Revision 1.53, 2016/08/25 23:02:21 UTC".

I've just tried again, including the noauto option (which I had forgotten about), and atlod has completed successfully. If I leave noauto off it returns that error.

This solves my issue, but what is the maximum size for a uv dataset that miriad can use?

Cheers,
Stefan

Re: ATLOD returning "File too large"

Posted: Tue Jun 13, 2017 12:46 pm
by Mark.Wieringa
Hi Stefan,

that is rather mysterious. I just tried again, now on my Mac, and it happily created an 11GB uv file. There should be no file size limit in the GB range. There are other limitations (# visibilities) that you may run into before reaching a file size limit.
So on both the linux64 and darwin_x86_64 version of Miriad I can't reproduce the problem.

The only thing I can think of now is that you are trying to write to an external disk formatted with a FAT file system that doesn't support files larger than 4 GB.
Could that be the case?

Cheers,

Mark

Re: ATLOD returning "File too large"

Posted: Tue Jun 13, 2017 2:06 pm
by Stefan
Hi Mark,

I appreciate the help with this - it turns out it is indeed the FAT32 formatting on my external hard drive that is causing the issue. Having tried again with the data stored on the Mac's HDD everything is working as it should!

Thanks,
Stefan