Transcript B.Stabler-Development of an Open Matrix Emme

Development of an Open
Matrix Emme Integration Tool
INRO Model City Portland
September 21, 2016
Today’s Talk
• What is OMX and why should
I care?
• How do EMME and OMX
work together?
• Example integrations for
– LCOG’s mode choice model
– Oregon Metro’s new multicriterion evaluation toolkit
OMX Integration, September 2016
RSG
2
What is OMX and why should I care?
Purpose of OMX
Is a simple standard matrix format for storing and
transferring matrix data that can be easily adopted
For
Anyone managing transportation data with a matrix structure
Who
Wants a portable and open format for storing and exchanging that data
That
Is easy to understand, can be used in custom programs/scripts, and
supports commercial products
Unlike
Existing solutions that are either proprietary, cumbersome, slow, and/or
too large
OMX Integration, September 2016
RSG
4
Who We Are
• A diverse group of agency, consultant, and
research modelers
• Formed a working group to define and
implement the file format for use in modeling
• Similar to the HTML working group for example
• Formed in early 2013, shortly after TRB
• Periodically meet to update the standard/project
OMX Integration, September 2016
RSG
5
We Believe OMX will:
• make model development and application
easier across models and software packages
• help spur industry research and innovation by
having a shared data structure (think
shapefile, GTFS)
• enable researchers, students, agencies, and
practitioners to collaborate in ways that are
currently difficult, cumbersome and errorprone
OMX Integration, September 2016
RSG
6
What is OMX
•
•
•
•
Open matrix is a published file format (i.e. skims.omx)
Based on open-source HDF-5 file storage technology
Binary data storage with compression
Stores multiple matrices in one file and includes zone
indexes/lookups
• Can contain additional attributes for matrices and indexes
• Python, R, Java, C#, C++, Ruby interfaces available
• Built-in support in EMME, VISUM, and TransCAD
• Supported by CT-RAMP and DaySim ABM platforms
OMX Integration, September 2016
RSG
7
How do EMME and OMX work together?
inro.emme.data.matrix.export_to_omx
OMX Integration, September 2016
RSG
9
inro.emme.data.matrix.import_from_omx
OMX Integration, September 2016
RSG
10
Example integrations
EMXtoOMX Command Line Tool
• EMME tools work well for basic matrix I/O
• However, we needed something that easily fits within our
typical DOS-based overall model workflow
• Thus, we developed a little command line tool for matrix
I/O
Export
python EMXtoOMX.py proj.emp 9999 mats.omx -e mf1 mf2
Import
python EMXtoOMX.py proj.emp 9999 mats.omx -i mf1 mf2
OMX Integration, September 2016
RSG
12
EMXtoOMX Command Line Tool Steps
• Starts EMME
• Loads the specified project and scenario
• If export:
– Exports the matrices using inro.emme.data.matrix.export_to_omx
– Removes appended matrix names in order to keep tool
interface simple
• mf1_timau  mf1
• If import:
– Imports the matrices using inro.emme.data.matrix.import_from_omx
OMX Integration, September 2016
RSG
13
LCOG Model Choice Model Example
• Eugene MPO C++ mode choice model previously did binary
databank I/O, which is no longer supported
• So we revised the C++ program by linking in the OMX/HDF5
libraries and recompiling the program
• Updated the overall model run script as well:
Export matrices
python EMXtoOMX.py LCOG.emp 9999 mos.omx -e mo10 mo11 mo12 mo13 …
python EMXtoOMX.py LCOG.emp 9999 mds.omx -e md02 md03 md34
python EMXtoOMX.py LCOG.emp 9999 mfs.omx -e mf294 mf295 …
Run model
lcogmc.exe hbw.ctl report.rpt
Read back in updated matrices
python EMXtoOMX.py LCOG\LCOG.emp 9999 mfs.omx -i mf81 mf82 mf83 …
OMX Integration, September 2016
RSG
14
Metro MCE Toolkit Example
• Portland Oregon MPO multi-criterion evaluation (MCE) / benefitscost analysis (BCA) social return on investment (SROI)
Python/pandas-based calculator currently being development
• Requires zone, OD, and link-level model outputs for a base and
build scenario
• Monetizes the differences and compares the benefits to scenario
costs
• Model also run via cmd line so data export looks similar to LCOG:
Export base scenario skims and trips
python EMXtoOMX.py nb_skimsbank.emp 4 mfs.omx -e mf1 mf2 …
python EMXtoOMX.py nb_assignbank.emp 4 mfs.omx -e mf8 mf7 …
Export build scenario skims and trips
python EMXtoOMX.py b_skimsbank.emp 4 mfs.omx -e mf1 mf2 …
python EMXtoOMX.py b_assignbank.emp 4 mfs.omx -e mf8 mf7 …
OMX Integration, September 2016
RSG
15
Resources
Resources
• OMX project, code, packages, examples, etc.
– https://github.com/osPlanning/omx
• EMME OMX cmd line tool
– https://github.com/bstabler/EMXtoOMX
• HDF5
– https://www.hdfgroup.org/hdf5
• pandas (basically Python’s version of R)
– http://pandas.pydata.org
OMX Integration, September 2016
RSG
17
Contacts
Ben Stabler
Portland, OR
[email protected]
www.rsginc.com