mirpy

Do you use (or want to use) other tools to reduce ATCA data?

Moderator: Mark.Wieringa

mirpy

Postby len067 » Thu Apr 22, 2010 10:42 am

Hi All,

Malte Marquarding has written a Python wrapper for miriad (mirpy) which can greatly simplify scripting with miriad. For further information, please refer to:

http://pypi.python.org/pypi/mirpy/

Cheers,

Emil.
len067
ATCA Expert
 
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

Re: mirpy

Postby len067 » Thu Apr 22, 2010 10:46 am

Hi All,

BTW, if anyone has their own filter functions (i.e. to parse the output from miriad tasks into something more usable) could you please post them here. Malte would like to collect these and perhaps incorporate them into mirpy.

Cheers,

Emil.
len067
ATCA Expert
 
Posts: 66
Joined: Mon Feb 08, 2010 2:35 pm

Re: mirpy

Postby Mark.Wieringa » Fri Apr 23, 2010 12:54 pm

Hi,

and if you have trouble parsing task output and/or have suggestions for output formatting for miriad tasks that would make that easier please post those here too.
We don't want to change the formats too often, because it might break existing scripts, but some tasks do need improved output or updates for the CABB era.

Cheers,

Mark
Mark.Wieringa
ATCA Expert
 
Posts: 284
Joined: Mon Feb 08, 2010 1:37 pm

Re: mirpy

Postby bnorfolk » Tue Dec 03, 2019 2:09 pm

Hi,

I'm attempting to use atlod in mirpy and I'm running into many issues. My current execution is detailed in the figure.

Now I've tried the filter:
def atload_filter(output):
return output.decode()

miriad.set_filter(‘atload’, atload_filter)

But I get an AssertionError with not details.

Any help would be greatly appreciated.
Cheers,
Brodie
Attachments
Screen Shot 2019-12-03 at 2.08.12 pm.png
Screen Shot 2019-12-03 at 2.08.12 pm.png (255.28 KiB) Viewed 778 times
bnorfolk
 
Posts: 41
Joined: Fri May 17, 2019 3:56 pm

Re: mirpy

Postby ste616 » Tue Dec 03, 2019 3:02 pm

Hi Brodie,

Does mirpy actually support Python 3? I've never tried. Could you try your exact same steps in 2.7 and see if the problem is the same?
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
 
Posts: 217
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: mirpy

Postby bnorfolk » Tue Dec 03, 2019 3:18 pm

Hi,

The github https://github.com/radio-astro/mirpy states py3 compatibility.

When setting up a conda environment with python 2.7 and using pip to install mirpy I get the the error that the module doesn't have the attribute atlod or set_filter.

Cheers,
Attachments
Screen Shot 2019-12-03 at 3.17.56 pm.png
Screen Shot 2019-12-03 at 3.17.56 pm.png (391.51 KiB) Viewed 778 times
bnorfolk
 
Posts: 41
Joined: Fri May 17, 2019 3:56 pm

Re: mirpy

Postby ste616 » Tue Dec 03, 2019 3:33 pm

You are misspelling "atlod" as "atload" in your call to set_filter. I don't know if this is the problem that you were experiencing originally, but given we should assume that mirpy does support Python 3, it's a possibility.

Could you please show the entire script?
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
 
Posts: 217
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: mirpy

Postby bnorfolk » Tue Dec 03, 2019 3:49 pm

ah yes - an embarrassing mistake.

If I run this code below:
from mirpy import miriad

ddir='/Users/brodienorfolk/obs/c1794/*.C1794'


def atload_filter(output):
return output.decode()


miriad.set_filter('atlod', atload_filter)
miriad.atlod(In=ddir,out='uvdata',options='birdie,rfiflag,xycorr,opcorr,noauto')

In py3 I receive:
TypeError: a bytes-like object is required, not 'str'
In py2.7 I receive:
AssertionError:
(The blank assertion error)
bnorfolk
 
Posts: 41
Joined: Fri May 17, 2019 3:56 pm

Re: mirpy

Postby ste616 » Tue Dec 03, 2019 4:03 pm

Hi Brodie,

What exactly are you trying to get from the "atload_filter" routine? The "decode" method seems to attach to a byte object, not a string, which would explain why you're getting the errors you are.

If all you're trying to do is get a list of the lines output from atlod, I would suggest you instead change your routine to:

Code: Select all
def atload_filter(output):
  return output.split('\n')


To get access to that output data though, you would also need to assign the miriad.atlod call, like so:

Code: Select all
atlodLines = miriad.atlod(In=ddir, out='uvdata', options='birdie,rfiflag,xycorr,opcorr,noauto')
cheers
Jamie Stevens
ATCA Senior System Scientist
ste616
Site Admin
 
Posts: 217
Joined: Thu Feb 04, 2010 3:27 pm
Location: Paul Wild Observatory Narrabri NSW

Re: mirpy

Postby bnorfolk » Tue Dec 03, 2019 4:13 pm

Hi Jamie,

I was using the decode line because Malte suggest it.

Currently I'm trying to run the python wrapper in an attempt to write an automated reduction pipeline - hence trying to output in a similar method as I would in the miriad bash.

I've implemented your new atload_filter but I still get the type error.

from mirpy import miriad

ddir='/Users/brodienorfolk/obs/c1794/*.C1794'


# def atload_filter(output):
# return output.decode()

def atload_filter(output):
return output.split('\n')


miriad.set_filter('atlod', atload_filter)
atlodLines = miriad.atlod(In=ddir,out='uvdata',options='birdie,rfiflag,xycorr,opcorr,noauto')
Attachments
Screen Shot 2019-12-03 at 4.11.45 pm.png
Screen Shot 2019-12-03 at 4.11.45 pm.png (429.41 KiB) Viewed 777 times
bnorfolk
 
Posts: 41
Joined: Fri May 17, 2019 3:56 pm

Next

Return to Other Tools

Who is online

Users browsing this forum: No registered users and 0 guests

cron