ATLOD returning "File too large"

Is MIRIAD being a pain? Let us know your experience.

Moderator: Mark.Wieringa

Post Reply
Stefan
Posts: 3
Joined: Sun Jun 11, 2017 8:51 pm

ATLOD returning "File too large"

Post by Stefan »

Hi,

I've been trying to load multiple RPFITS files (4) through atlod with a total of four sources, pasted below from the ATOA.

▶ 2011-12-12_1700.C2457 1714.43 no C2457 2011-12-13 6A A Baker 0214-522 0302-623 ACT0245-5302
▶ 2011-12-12_1335.C2457 4094.90 no C2457 2011-12-13 6A A Baker 0214-522 ACT0245-5302
▶ 2011-12-12_0643.C2457 4094.93 no C2457 2011-12-12 6A A Baker 0214-522 1934-638 ACT0245-5302
▶ 2011-12-12_1010.C2457 4094.92 no C2457 2011-12-12 6A A Baker 0214-522 ACT0245-5302

Upon attempting to load via atlod I receive "Error writing variable-value for corr, in UVPUTVR: File too large" with the following settings:

Code: Select all

atlod in=*.C2457 out=c2457.uv ifsel=1 options=birdie,rfiflag,xycorr
I've also tried simply loading each file in separately and then using uvsplit and uvcat to get single source miriad files but I am greeted with the same error. Will this simply require me to split the main source (ACT0245-5302) into multiple miriad files or is there a way around this?

Cheers,
Stefan
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: ATLOD returning "File too large"

Post by Mark.Wieringa »

Hi Stefan,

What version of Miriad are you using?
Please check that you have the current ATNF version of Miriad on a 64 bit linux or MacOS machine.
If you find that you cannot create files larger than 4 GB, you must have an old OS or Miriad version.

My atlod prints:
atlod: Revision 1.53, 2016/08/25 23:02:21 UTC

I tried to replicate your problem and successfully loaded the 4 files you indicated into a single uv file.

You may also want to add the flag noauto to the options, to throw away the autocorrelations.

Cheers,

Mark
Stefan
Posts: 3
Joined: Sun Jun 11, 2017 8:51 pm

Re: ATLOD returning "File too large"

Post by Stefan »

Hi Mark,

I have the latest darwin_x86_64 miriad version which has atlod version "Revision 1.53, 2016/08/25 23:02:21 UTC".

I've just tried again, including the noauto option (which I had forgotten about), and atlod has completed successfully. If I leave noauto off it returns that error.

This solves my issue, but what is the maximum size for a uv dataset that miriad can use?

Cheers,
Stefan
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: ATLOD returning "File too large"

Post by Mark.Wieringa »

Hi Stefan,

that is rather mysterious. I just tried again, now on my Mac, and it happily created an 11GB uv file. There should be no file size limit in the GB range. There are other limitations (# visibilities) that you may run into before reaching a file size limit.
So on both the linux64 and darwin_x86_64 version of Miriad I can't reproduce the problem.

The only thing I can think of now is that you are trying to write to an external disk formatted with a FAT file system that doesn't support files larger than 4 GB.
Could that be the case?

Cheers,

Mark
Stefan
Posts: 3
Joined: Sun Jun 11, 2017 8:51 pm

Re: ATLOD returning "File too large"

Post by Stefan »

Hi Mark,

I appreciate the help with this - it turns out it is indeed the FAT32 formatting on my external hard drive that is causing the issue. Having tried again with the data stored on the Mac's HDD everything is working as it should!

Thanks,
Stefan
Post Reply