mirpy

Do you use (or want to use) other tools to reduce ATCA data?

Moderator: Mark.Wieringa

len067
ATCA Expert
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

mirpy

Post by len067 »

Hi All,

Malte Marquarding has written a Python wrapper for miriad (mirpy) which can greatly simplify scripting with miriad. For further information, please refer to:

http://pypi.python.org/pypi/mirpy/

Cheers,

Emil.
len067
ATCA Expert
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

Re: mirpy

Post by len067 »

Hi All,

BTW, if anyone has their own filter functions (i.e. to parse the output from miriad tasks into something more usable) could you please post them here. Malte would like to collect these and perhaps incorporate them into mirpy.

Cheers,

Emil.
Mark.Wieringa
ATCA Expert
Posts: 297
Joined: Mon Feb 08, 2010 1:37 pm

Re: mirpy

Post by Mark.Wieringa »

Hi,

and if you have trouble parsing task output and/or have suggestions for output formatting for miriad tasks that would make that easier please post those here too.
We don't want to change the formats too often, because it might break existing scripts, but some tasks do need improved output or updates for the CABB era.

Cheers,

Mark
bnorfolk
Posts: 45
Joined: Fri May 17, 2019 3:56 pm

Re: mirpy

Post by bnorfolk »

Hi,

I'm attempting to use atlod in mirpy and I'm running into many issues. My current execution is detailed in the figure.

Now I've tried the filter:
def atload_filter(output):
return output.decode()

miriad.set_filter(‘atload’, atload_filter)

But I get an AssertionError with not details.

Any help would be greatly appreciated.
Cheers,
Brodie
Attachments
Screen Shot 2019-12-03 at 2.08.12 pm.png
Screen Shot 2019-12-03 at 2.08.12 pm.png (255.28 KiB) Viewed 10339 times
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: mirpy

Post by ste616 »

Hi Brodie,

Does mirpy actually support Python 3? I've never tried. Could you try your exact same steps in 2.7 and see if the problem is the same?
cheers
Jamie Stevens
ATCA Senior System Scientist
bnorfolk
Posts: 45
Joined: Fri May 17, 2019 3:56 pm

Re: mirpy

Post by bnorfolk »

Hi,

The github https://github.com/radio-astro/mirpy states py3 compatibility.

When setting up a conda environment with python 2.7 and using pip to install mirpy I get the the error that the module doesn't have the attribute atlod or set_filter.

Cheers,
Attachments
Screen Shot 2019-12-03 at 3.17.56 pm.png
Screen Shot 2019-12-03 at 3.17.56 pm.png (391.51 KiB) Viewed 10339 times
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: mirpy

Post by ste616 »

You are misspelling "atlod" as "atload" in your call to set_filter. I don't know if this is the problem that you were experiencing originally, but given we should assume that mirpy does support Python 3, it's a possibility.

Could you please show the entire script?
cheers
Jamie Stevens
ATCA Senior System Scientist
bnorfolk
Posts: 45
Joined: Fri May 17, 2019 3:56 pm

Re: mirpy

Post by bnorfolk »

ah yes - an embarrassing mistake.

If I run this code below:
from mirpy import miriad

ddir='/Users/brodienorfolk/obs/c1794/*.C1794'


def atload_filter(output):
return output.decode()


miriad.set_filter('atlod', atload_filter)
miriad.atlod(In=ddir,out='uvdata',options='birdie,rfiflag,xycorr,opcorr,noauto')

In py3 I receive:
TypeError: a bytes-like object is required, not 'str'
In py2.7 I receive:
AssertionError:
(The blank assertion error)
ste616
Site Admin
Posts: 220
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: mirpy

Post by ste616 »

Hi Brodie,

What exactly are you trying to get from the "atload_filter" routine? The "decode" method seems to attach to a byte object, not a string, which would explain why you're getting the errors you are.

If all you're trying to do is get a list of the lines output from atlod, I would suggest you instead change your routine to:

Code: Select all

def atload_filter(output):
  return output.split('\n')
To get access to that output data though, you would also need to assign the miriad.atlod call, like so:

Code: Select all

atlodLines = miriad.atlod(In=ddir, out='uvdata', options='birdie,rfiflag,xycorr,opcorr,noauto')
cheers
Jamie Stevens
ATCA Senior System Scientist
bnorfolk
Posts: 45
Joined: Fri May 17, 2019 3:56 pm

Re: mirpy

Post by bnorfolk »

Hi Jamie,

I was using the decode line because Malte suggest it.

Currently I'm trying to run the python wrapper in an attempt to write an automated reduction pipeline - hence trying to output in a similar method as I would in the miriad bash.

I've implemented your new atload_filter but I still get the type error.

from mirpy import miriad

ddir='/Users/brodienorfolk/obs/c1794/*.C1794'


# def atload_filter(output):
# return output.decode()

def atload_filter(output):
return output.split('\n')


miriad.set_filter('atlod', atload_filter)
atlodLines = miriad.atlod(In=ddir,out='uvdata',options='birdie,rfiflag,xycorr,opcorr,noauto')
Attachments
Screen Shot 2019-12-03 at 4.11.45 pm.png
Screen Shot 2019-12-03 at 4.11.45 pm.png (429.41 KiB) Viewed 10338 times
Post Reply