Transcript - ePrints Soton

Instruments and Sensors as Grid
Services
Donald F. (Rick) McMullen1
Kenneth Chiu2
John C. Huffman1
Kia Huffman1
Randall Bramley1
1Indiana
2SUNY
University
Binghamton
Motivations
• Instruments and sensors are not well integrated
into grids
• Data is acquired, processed and stored before it
“hits the grid”.
• Need methodology for interacting with
instruments and sensors in real time from grid
applications
• Some abstraction of sensor and instrument
functionality is needed to make grid applications
that use them more robust and flexible.
2
Goals
• Integrate instruments and sensors (e.g. real-time data
sources) into a Grid computing environment with Grid
services interfaces.
• Abstract instrument capabilities and functions to reduce data
acquisition and analysis applications’ dependence on
specialized knowledge about particular instruments.
• Move production of metadata as close to instruments as
possible and facilitate the automatic production of metadata.
• Develop a standard, reusable methodology for “grid
enabling” instruments.
• Collaborate with scientists in academia and industry in a
broad range of disciplines who either develop instruments or
whose work depends on the details of using them.
3
Increasing
bandwidth
per sensor
Electron Microscope
X-Ray Crystallography
Ground motion
sensor array
Traffic sensors
Radio Telescope
Wireless ‘mote’
sensor network
Increasing number of sensors
Increasing real-time application
• Simple 2-D analysis of instrument taxonomy
• At lease five dimensions identified in more detailed analysis
• Project must address enough points (classes) to assure breadth of applicability
4
Initial Applications
• High brilliance X-ray crystallography
– Large instrument application
– Deeply integrated into bio and medical discovery research methodology
– Mature analysis software and large user community
• Robotic telescopes
– Small numbers of sensors: CCD, environmental; some control aspects:
filters, aiming, dome.
– Global coordination needed for scheduling
– Aggregation of disparate sensors into a “composite”
instrument
• Small sensors
– Minimal memory and CPU
– Wireless connectivity
– Developing parallel project to use ad hoc/swarm networks for data
collection for real-time simulation and prediction
– Updating of data flows in response to sensor/network reconfiguration.
5
CIMA implementation targets
Large scientific
instruments
Embedded sensors
and controllers
very
large
systems,
few
elements
very
small
systems,
many
elements
PC104 industrial
controller board
Synchrotron
beamline
(APS/ALS)
MICA Mote wireless
6
sensor/controller board
MMSF Automated Telescope
• Typical remote access
automated observatory
• System has 33 distinct
“sensors”, 12 controllers
– Open/close of roof based on a
Polaris transparency monitor
and rain detector – simple grid
of wires detecting rain drops
– Telescope direction and dome
control
– Filter selection and telescope
focus
– Liquid nitrogen fills of the CCD
dewar jars ….
7
MMSF Observatory Features
• Instruments producing data without units
– Temperature, humidity cutoffs determined empirically
as resistances, not degrees or %
• Hierarchical and co-located instruments
– Single platform holds three instruments, so orienting
one changes orientation of all
• Updates to equipment occurs frequently
• Data transferred via 28k modem line –
middleware needs to work locally, directly
between instruments and sensors
8
Observatory Data Architecture
• Control Data
– object control
– instrument control
• Object Data (i.e., object of scientific interest)
– Full spectrum from raw unitless data to derived data
artifacts
• Instrument package
–
–
–
–
–
system package data (multiple attributes output)
system sensor data (single attribute output)
nonsystem sensor data (weather data from NOAA)
calibration data
access protocols
9
X-Ray Crystallography
Proxy Box
Instrument
Services
WAN
Instrument
Manager
LAN
Portal
LAN
Non-grid service
Grid service
Persistent
Data
Archive
Non-persistent
10
LTER
University of
Wisconsin Campus
Sensors
Buoy
Env.
Agent
Oracle
Datalogger
ORB
Config.
Agent
QA
Agent
Config. Event (CIMA)
Other Locations
Trout Lake Station
Config.
Agent
QA
Agent
2
ARTS Connection
4
3
Web
Server
Config.
Agent
3
Env. Event (CIMA)
Web
Browser
Config.
Agent
Web
Browser
1
JDBC/ODBC Connection
Other Connection
•
•
•
Automatic updating of flows
Provenance
SOAP/ARTS (Antelope Real-Time System)
11
Further applications
currently being
explored
Electron Microscopy
-U. Queensland EM group – regional scale,
multiple instruments
- KISTI 2nd largest EM (after Osaka)
Robotic telescopes
- Bradford robotic telescope,
Oxenhope observatory,
Faulkes telescope project
Environmental monitoring
- Water use
- contaminants
Industrial monitoring and control, e.g. Train axles
- Ore trains – Km long, derailment is very expensive to fix
- Temperature sensors on axles monitor bearing status, anticipate wheel failure
12
Architecture
13
Common Instrument Middleware
Architecture (CIMA) Elements
• Schema for instrument functionality (and ontology for schema
attributes);
• Data model for representing instrument metrics and calibration;
• A small, high performance, embeddable Web Services stack, initially in
Java, including Proteus support for multi-protocol, multimodal
transport;
• Service implementation for accessing the instrument’s functionality and
metrics via the Proteus-mediated interface;
– Ability to dynamically insert new protocols into running instances
• OGSA and WS-RF compliant functions to register with a location
service, authenticate users, provide access control to instrument
controls and data, send and receive events, and co-schedule the
instrument into a Grid computing and storage context.
14
Overview
Data Pipeline
Instrument
Sensor 2
Acquisition
Component
Analysis
Component
Curation
Component
Controller
Acquisition
Code
Analysis
Code
Curation
Code
Instrument
Presentation
Instrument
Access
Instrument
Access
Instrument
Access
Sensor 1
Physical Network Transport
Instrument
Access
Device-Dependent Virtualization Module
Remote
Access
Device-Independent Application Module
Shared Implementation
GUI
Scientist
15
Instrument
Access
Instrument
Presentation
Instrument-Specific CIMA
Instrument-Independent CIMA
Existing Instrument Code
Service
Implementation
Controller
OGSI
SOAP
Binary
Format
Instrument
Hardware
Proteus
TCP/IP
Other Network
Protocols
16
Example CIMA minimal knowledge bootstrap
procedure
CIMA instrument
Application
Globus init, user
authentication, and
instrument lookup
Application parses
description for ports to
read for calibration
and voltage
Proteus/SOAP calls
“send description”
“RDF description”
Service
Implementation (SI)
returns
description of itself
and instrument
“read calibration port”
“calibration”
SI returns
stored calibration
“read thermocouple port”
“thermocouple voltage”
Application reads
thermocouple voltage
then computes and
displays a
temperature
SI calls controller
function to read
and return voltage
17
Services
• Instrument
Instrument
– get
– set
Sensor
• Sensor
– get
– set
Channel
Sink
Channel
Source
Channel
Source
• ChannelSource
– get
– set
– register
• ChannelSink
Sensor
Sensor
Channel
Source
Channel
Source
– receive
– get
– set
18
Parcels
• Wish to unify our data models, etc.
• Toolkit must be application-independent as
much as possible.
• Attributes
– Type (string)
– Globally unique ID (string)
– Encoding
• CDATA, Binary, ASCII, Base64
– Location
• Inline, URL, Other
• Parcel Sets
– Special data used for connectivity information.
19
Technologies
20
Technologies
• Web services
– XML, SOAP, WSDL, binary XML
• Grid services
– OGSA/OGSI, WS-RF, DAIS
•
•
•
•
Axis C++ gSOAP
Proteus (SC 2002)
XBS (HPC 2004)
Schema-specific parsing
21
Proteus Motivation
• Web Services for scientific computing
– SOAP performs well as a lingua franca
• But suffers from performance problems for
scientific data
– Solution: establish initial communication with
SOAP, and then switch to a faster protocol.
• Grid intermediaries
22
Proteus Overview
• Provides multiprotocol RMI system to applications
– Can wrap existing protocol implementations with dynamic
invocation
• Facilitates use of SOAP as common language
– Switch to faster protocols if supported by both sides.
• Mediates between protocol providers and applications
– Applications use Proteus client API
– Providers use Proteus provider API
• Allows a new provider to be added (at run-time) without
changing application
• Generic serialization/deserialization allows marshalling
code to be reused for multiple protocols
23
Multiprotocol
Process 1
Process 2
Client
Client
Proteus
Proteus
ProviderA
ProviderA
ProviderB
ProviderB
Protocol A
Network
Protocol B
24
Proteus APIs
Application
Client API
Proteus
Provider API
Protocol A Provider Protocol B Provider
25
Schema-Specific Parsing
• XML processing stages (conceptual)
– Well-formedness
• Lexical and syntactic, defined by core XML
specification.
– Validity
• Conformance to a schema, mainly structural
– Application
26
Merging Stages
Application
Application
Application
Validation
Validation
Validation
Wellformedness
Wellformedness
Wellformedness
Fully articulated
Implicitly validated
Schema-specific parsing
User written
Merged
27
Compiler-Based Approach
C (fast)
XML
Schema
IR
RELAX
NG
Java
C
(low-pow)
• Front-end parses schema into intermediate
representation.
• Back-end generates code from intermediate
representation.
• Intermediate representation is a generalized
automata.
28
Summary
• Create standards for accessing broad
spectrum instruments and sensors.
• Incompatible components should still have
some base level of interoperability.
29