Transcript ESMF Infrastructure

Earth System Modeling Framework
Python Interface (ESMP)
October 2011
Ryan O’Kuinghttons
Robert Oehmke
Cecelia DeLuca
Architecture
• ESMF provides a
superstructure for assembling
geophysical components into
applications.
• ESMF also provides an
infrastructure that modelers
use to
– Generate and apply
interpolation weights
– Handle metadata, time
management, data I/O and
communications, and other
functions
– Access third party libraries
Components Layer
Gridded Components
Coupler Components
ESMF Superstructure
Model Layer
User Code
Fields and Grids Layer
ESMF Infrastructure
Low Level Utilities
External Libraries
MPI, NetCDF, …
The purpose of the ESMF Python
interface (ESMP) is to provide
access to ESMF Field Regridding.
Python Interface Features
ESMF related features:
•
Unstructured Mesh
•
Field built on Mesh
•
Parallel regridding (bilinear, higher order, or conservative)
Python related features:
•
Numpy arrays for data representation in Python
•
Pythonic approach to error handling and multiple return values
•
Automatic documentation via Pydoc
Other neat features:
•
Open source and freely available!
Limitations
Data structure related:
- No logically rectangular grid representation
- No FieldBundle class, only single Fields
- No access to Mesh coordinates
- No access to Field bounds
Testing related:
- Only tested on Darwin/gfortran platform
- Parallel implementation is not fully tested
- Does not undergo nightly ESMF regression
testing
ESMP is in an early BETA stage!
The Python interface is currently not on
the ESMF task list. If you would like to
see more functionality included, please
contact ESMF support so this can be
made a priority with the ESMF Change
Review Board.
Data Structures
Fortran
C
Component
x
x
State
x
x
FieldBundle
x
x
Field
x
x
ArrayBundle
x
x
Array
x
x
Grid
x
Mesh
x
Attributes
x
Log
x
VM
Time
Manager
Python
x
x
x
x
x
x
x
x
Data Representation – Numpy
Numpy – scientific computing with Python, a piece of Scipy
-
Used to represent data arrays on the Python front-end
Numpy arrays are passed in and out of ESMP interfaces
Input parameters in Numpy arrays are copied into internal ESMP data
structures, but referenced out of internal ESMP data structures.
Sample Code:
Declare a Numpy array
ungriddedLBoundN = Numpy.array([1,1])
Create an ESMP_Field, copy data from Numpy array
field = ESMP_FieldCreate(...ungriddedLBoundN, ...)
Get a pointer to Field data as a Numpy array
fieldPtr = ESMP_FieldGetPtr(field, 0)
Access Field data as a Numpy array, referencing the internal data
for i in range(num_node):
fieldPtr[i] = 10
ESMP Field regridding
Functions in much the same way as Field regridding in ESMF
routehandle = ESMP_FieldRegridStore
(srcfield, dstfield,
ESMP_REGRIDMETHOD_BILINEAR,
ESMP_UNMAPPEDACTION_ERROR)
ESMP_FieldRegrid(srcfield, dstfield,
routehandle)
ESMP_FieldRegridRelease(routehandle)
Features:
File based regridding:
-
Fully parallel, and highly scalable
-‘remap’ module (from ESMP import remap)
-
Bilinear, higher order, or conservative
-Drives the ESMF_RegridWeightGen application
-
2D or 3D unstructured Meshes
-Inputs: NetCDF grid files
-
Optional error handling of unmapped
destination points
-Output: NetCDF weights file
-Supports: ESMF and SCRIP file format
http://climate.lanl.gov/Software/SCRIP/
-Limitations: Serial NetCDF file IO
Automatic documentation
Documentation can be generated with the ‘pydoc’ command at terminal prompt:
$: pydoc ESMP
Help on module ESMP:
NAME
ESMP - The ESMF Python API
...
FUNCTIONS
ESMP_FieldCreate(mesh, typekind, gridToFieldMap,
ungriddedLBound, ungriddedUBound, name)
RETURN ESMP_Field :: field
ESMP_TypeKind
:: typekind
Numpy.array
:: gridToFieldMap
Numpy.array
:: ungriddedLBound
Numpy.array
:: ungriddedUBound
string
:: name
...
Software access and
dependencies
The software is available for download from the esmfcontrib Sourceforge cvs repository:
cvs -z3 -d:pserver:[email protected]:/cvsroot/esmfcontrib co -P -rESMF_5_3_0_beta_snapshot_09 python
Dependencies:
-Python 2.6 or higher – add top level ESMP directory to PYTHONPATH
-Ctypes
-
http://docs.python.org/library/ctypes.html
-
Not supported on AIX
-Numpy
-
http://numpy.scipy.org/
-ESMF installation
-
ESMF_5_3_0_beta_snapshot_09 or higher!
-
Shared libraries not supported on some platforms (notably Darwin), see the supported
platforms table from ESMF for more information:
-
http://www.earthsystemmodeling.org/download/platforms/
-ESMFMKFILE environment variable pointing to installation esmf.mk file
Optional dependencies:
-NetCDF - 'remap' interface to ESMF_RegridWeightGen
-LAPACK - higher order patch regridding
-GNUMake - terminal helper commands
Specific Questions on ESMP
Style and Conventions
-
Argument returns in ESMP_XXXGet() routines:
- Could receive arguments in a tuple of multiple return values
- Could receive each piece of info in a single Get call
- Could also use Python class structures to pass allocated memory into
a Get routine to fill with data
-
Type forcing of Numpy array input arguments
- Could force a specific type for these array inputs
- Could cast input array to type needed by ESMF
-
Error handling
- No return codes in ESMP -> Abort on bad ESMF return code
- Backtrace from Python -> Send to stdout or ESMF logs?
-
Any other issues of style or convention?
Future work proposal
-
Add support for the following:
- Get bounds information from a Field
- Get coordinate information from a Mesh
- Structured grids
-
Output of numerical results
- Mesh and Field information in vtk file format using
ESMP_MeshWrite (plot with Visit or Paraview)
- NetCDF format?
-
Support for NetCDF IO of Field data? Serial or Parallel?
-
Add support for Components and States?
-
Anything else?