Transcript seacoos_mum_2004_v2 - University of South Carolina

Near Real Time
Ocean Observations
Online
the Escape of SEACOOS
(Southeastern Atlantic Coastal Ocean Observing System)
Data Management and Visualization Secrets
The (tiny) big picture
Near Real Time Ocean Observations
Online : SEACOOS
Near Real Time Ocean Observations
Online : SEACOOS
The nuts & bolts




Data is aggregated and stored
Data is normalized
Data is visualized
Data is disseminated
Near Real Time Ocean Observations
Online : SEACOOS
Data aggregation & storage

Format flavors

netCDF: coverage may vary;
format may not vary

PNG: coverage may not vary;
color scale may not vary
Near Real Time Ocean Observations
Online : SEACOOS

Technology
 Perl
Data aggregation & storage

Relational database

One table per in-situ variable type
E.g. a wind_table, sst_table
 Contains the real data


One table per raster variable
E.g. a sst_raster_table
 Contains pointers to PNG files + boundaries


One table per model variable per hour

E.g. a 12_00_00_current_forecast_table
Near Real Time Ocean Observations
Online : SEACOOS
Normalization

Reporting time varies

In-situ data


Remotely-sensed data


E.g. daily, hourly, half-hourly, every 10 minutes
E.g. twice daily
Reporting area varies

Remotely-sensed
data passes
Near Real Time Ocean Observations
Online : SEACOOS
Visualization

Main engines




PHP, PHP-MapScript
Perl
MapServer, of course
Icing on the cake





Dynamic shapefile
creation for colorbars
AnIS and GIFSicle for
animations
ImageMagick for image
manipulation and raster
queries
Mouseovers
Time-series graphs using
Gnuplot
Near Real Time Ocean Observations
Online : SEACOOS
Icing on the cake : dynamic
shapefiles
Near Real Time Ocean Observations
Online : SEACOOS
Icing on the cake : animations

AnIS and GIFsicle

Produce several
images and glue them
together
Near Real Time Ocean Observations
Online : SEACOOS
Icing on the cake : raster queries
Let MapServer take care of projection issues
How to get an SST, for example, from a raster PNG


1)
2)
3)
4)
Dump the current extent view of just the raster to PNG
Get (x,y) clicked pixel RGB of the dumped image with
Perl::ImageMagick
Lookup pixel RGB in database to find out SST
Return SST in the normal query data train
Near Real Time Ocean Observations
Online : SEACOOS
Icing on the cake : mouseovers


Use MapServer 4.x searchmap
Produce two images for an HTML
page


One is static; with the querymap
The other is an input image


Use javascript to swap the two
images based on user activity
The querymap is produced by
making a simple LWP::simple
request for a querymap, and then it
is merged with the HTML page
Near Real Time Ocean Observations
Online : SEACOOS
Icing on the cake : time series graphs

Pass a query result set to
Gnuplot for a time-series
graph
Near Real Time Ocean Observations
Online : SEACOOS
Dissemination



Maps and charts, of course
OPeNDAP (DODS) data access
OGC-friendly: WMS, WFS


SEACOOS pushed a lot of data to the IOOS
interoperability project
Down the pipeline

Ad-hoc CSV and/or netCDF data extraction
Near Real Time Ocean Observations
Online : SEACOOS
Magic 8 ball

Everyone wants speed
‘Near real time’ implies constantly updated databases
 Recently introduced round-robin-ing of database
requests

As machine A is populated with new data, send all DB
requests to machine B
 When machine A is done populating, have it resume DB
handling, and go update machine B


Want to round-robin MapServer requests, too, as
well as pull application server to its own box
Near Real Time Ocean Observations
Online : SEACOOS
A friend in the business



Development site: http://nautilus.baruch.sc.edu/rs
Production site (new interface to be released soon):
http://seacoos.org
Very active listserv dealing with mainly remote-sensing
issues: [email protected]



mailto: [email protected]
message text: subscribe remotesensing
Who am I?



Charlton Purvis, University of South Carolina, SEACOOS
[email protected]
Happy to help and share.
Near Real Time Ocean Observations
Online : SEACOOS