Transcript Slide 1
WfMS and external systems Katarzyna Bylec PSNC Agenda • Introduction • Pre-corelation ▫ ▫ ▫ ▫ ▫ North Star NRAO SCHED Vlbeer FTP Log2vex drudg • Correlation ▫ Field system ▫ Telescope visualisation ▫ Software correlator • Post-correlation ▫ ▫ ▫ ▫ ▫ ▫ j2ms2 2bitVVx Diagnostic tools tConvert Crude images creation EVN data archive WP7 Meeting – Poznań- 19-04-2010 Introduction • External systems ▫ ▫ ▫ ▫ Existing or to-be-implemented APIs Defined by Use Cases Included directly in KIWI Portal Accessed via WfMS Server • Goals of this presentation ▫ ▫ ▫ ▫ Decide if KIWI/NEXPReS will use those services What tools can be used? Are there APIs defined? Are those services remotelly available? • Glossary ▫ WfMS – Workflow Manager Service WP7 Meeting – Poznań- 19-04-2010 Pre-correlation: North Star • Web-based Proposal Submission Tool • Usage: ▫ By Prinicipal Investigator ▫ Creation and submission of proposal ▫ Information of proposal status • Interactivity/automatization ▫ Human interaction needed • WfMS integration ▫ Should it be integrated in KIWI Portal? ▫ Web-page redirecting WP7 Meeting – Poznań- 19-04-2010 Pre-correlation: NRAO SCHED • Usage: ▫ By Prinicipal Investigator ▫ Produce VEX file for observation • Interactivity/automatization ▫ Have to be reviewed by JIVE staff • WfMS integration ▫ What’s the input? Can the creation be automated? ▫ Is it accessible as a remote service? Can it be integrated in KIWI Portal (directly)? ▫ Should it be installed at WfMS? WP7 Meeting – Poznań- 19-04-2010 Pre-correlation: vlbeer FTP • FTP repository of schedules • Usage: ▫ ▫ ▫ ▫ By all stores VEX schedules password-based upload anonymous download • Interactivity/automatization ▫ Access - automated ▫ Files - have to be reviewed by JIVE staff • WfMS integration ▫ Should it be integrated in KIWI Portal (directly FTP access)? ▫ Import from Vlbeer to KIWI Portal ▫ An idea: alternative repository managed by WfMS? WP7 Meeting – Poznań- 19-04-2010 Pre-correlation: log2vex • Usage: ▫ By Correlation Operator ▫ adds infromation to VEX file: clock errors recorded media Earth Orientation Parameters (EOPs) • Interactivity/automatization ▫ It can be automated - ? • WfMS integration ▫ WfMS can run log2vex using input data from user (or can it be done without user basing on some const. data?) WP7 Meeting – Poznań- 19-04-2010 Pre-correlation: drudg • Usage: ▫ ▫ ▫ ▫ By Telescope Operator Input: VEX file Output: Procedure and Snap files can be run interactively or in a batch mode • Interactivity/automatization ▫ Human interaction needed – to review the files ▫ Automated – with set of oscilator voltage values known • WfMS integration ▫ WfMS integration – run by WfMS and results presented for acceptation? WP7 Meeting – Poznań- 19-04-2010 Correlation: Field system • Station Control Software • Usage: ▫ By Telescope Operator ▫ Mainly monitoring ▫ Telescope control? • Interactivity/automatization ▫ Human interaction needed • WfMS integration ▫ ▫ ▫ ▫ DO WE NEED THAT? (Security, time) Integration of Remote Tool (WP5’s Task 3) into KIWI Portal? RPC call to FS functionalities What’s the difference between FS and Station Control Software? WP7 Meeting – Poznań- 19-04-2010 Field System - remote • Remote Access – Web Services ▫ RPC-style C/C++ ONC RPC implementation UNIX only client generation tool ▫ If FS will expose telescope/observation controll API WfMS can forward it to KIWI Portal Will TOs acctually use it when they have full FS access? WfMS can start/stop observation - ??? • Remote Access – Rich Client Application ▫ C++ wxWidgets ▫ very hard/impossible to integrate it into KIWI Portal WP7 Meeting – Poznań- 19-04-2010 Correlation: Telescopes visualisation • Usage: ▫ Functionality in KIWI Portal ▫ Visualisation of resources (parsing VEX) ▫ Monitoring • Interactivity/automatization ▫ automated • WfMS integration ▫ Tool needed (the same as used in EXPReS? JIVE?) Event – Location - Date Correlation: Correlator … will be discussed tomorrow. WP7 Meeting – Poznań- 19-04-2010 Post-correlation: j2ms2 & 2bitVVx • Usage ▫ j2ms2: translates correlation output to APIS++/CASA MeasurementSet ▫ 2bitVVx: Correction of amplitudes (Vleck correction) • Interactivity/automatization ▫ Can be automated - ? • WfMS integration ▫ Can be run by WfMS after correlation ▫ Are those programs available as remote services? ▫ Documentation and programs needed WP7 Meeting – Poznań- 19-04-2010 Post-correlation: diagnostic tools • Usage ▫ Suite of programs run on the MeasurementSet ▫ Assures that correlation was done correctly ▫ In e-VLBI re-correlation is not possible • Interactivity/automatization ▫ Can be automated - ? • WfMS integration ▫ What tools are in this suite? ▫ Can they be run by WfMS? ▫ Are they accessible as a remote services? WP7 Meeting – Poznań- 19-04-2010 Post-correlation: tConvert • Usage ▫ Converts MeasurementSet into the FITS-IDI data format ▫ FITS-IDI is the de facto standard for VLBI data reduction • Interactivity/automatization ▫ Can be automated - ? • WfMS integration ▫ Can tConvert be run by WfMS? ▫ Is it accessible as a remote services? WP7 Meeting – Poznań- 19-04-2010 Post-correlation: crude images • Usage ▫ Pipeline script (Python/ParselTongue) ▫ Creates crude images of observed sources ▫ Generates diagnostic plots • Interactivity/automatization ▫ Can be automated - ? ▫ Can diagnostic plots be handled by WfMS (user needed?) • WfMS integration ▫ Can it be run by WfMS? ▫ Is accessible as remote services? WP7 Meeting – Poznań- 19-04-2010 Post-correlation: EVN data archive • Usage ▫ http://www.jive.nl/select-experiment ▫ Upload of correlation results ▫ Access to data • Interactivity/automatization ▫ Can be automated - ? • WfMS integration ▫ ▫ ▫ ▫ Is there API available? Are they accessible as remote services? What is taken as an input? What parameters describe uploaded data? WP7 Meeting – Poznań- 19-04-2010