Transcript Machine-To-Machine Interface Strategy

Machine-to-Machine Interface Strategy
December 28, 2006
DRAFT
1
External Interface Strategy
Deliverables
•
•
Overall Strategy
• All interactions outside of user interfaces
• Alternative to screen-scraping
• Security approach
• Delivery approaches
• Technical interoperability
• Service Level Agreements (SLA)
• Auditing, Monitoring and Management
• Versioning
• Governance to manage technical issues
•
Interface Specification (per interface)
• Interface Approach (based on the strategy)
• Transport and Data Format e.g. XSD
• Signature & Capabilities
• Interoperability
•
Market Participant Sandbox Environment
• Stubbed Applications
• Validate the Approach
• Validate the Security
• Validate Interoperability
• Identify Refactoring Early
List of Interfaces (services)
• Prioritization set with TPTF
•
•
•
•
•
•
•
•
Importance of an interface
Volatility of an interface
Existence in underlying systems
•
Support to Market testing for example EDS3
Uses the SoSA & Domain Model
Constrained by System Capabilities
Sufficient granularity to initiate MP work
Represents requirements
2
Plan
• Schedule to deliver all interface Specifications
in iterations
• Solicitation of feedback
External Interface Strategy
Plan for Delivering the Complete Set of Interface Specifications
12/30/06
1/31/07
2/28/07
3/31/07
4/30/07
•Draft Strategy
•Draft List of Interfaces
•Draft Interface
Specifications
•Definition of Nodal Sandbox
•Plan to Deliver Complete Set
of Interface Specifications
•Operating Nodal Sandbox
•Definition of Interfaces for Bidding
•Interfaces for Notifications
•Definition of Interfaces for Awards and Obligations
•Partial Set of Interfaces for Market Information Requests
•Updated Sandbox to include some Bidding and Notification
Interfaces
•Final Strategy
•Final List of Interfaces
•Complete Set of Interface Specifications
•Updated Nodal Sandbox to Include Some Market
Information Request Interfaces
•Plan for Iterative
Implementation of
the Interfaces
3
External Interface Strategy
High Level Requirements
• Provision simple, reliable and secure Market Participant machineto-machine interfaces by
– Providing SIMPLICITY through a consistent industry standard
approach across Nodal. Need to insulate Market Participants from
variations in approach across Nodal systems (NMMS, MMS, CRR,
CS, EDW etc..)
– Providing RELIABILITY through a robust and reliable web services
infrastructure. This infrastructure includes a common approach to
message handling, auditing, exception handling, monitoring,
notification, disaster recovery and process orchestration.
– Providing SECURITY through an enterprise approach to
authentication, authorization, data confidentiality and data integrity
4
External Interface Strategy
Functional Requirements
•
•
Need to provide support for servicing requests from Market Participant
software:
– A variety of Web services would be provided to enable the processing of
requests
– Need to continue existing functionality as well as support new Nodal
functionality
Web Service Patterns
– Market Participant initiated requests associated with bid, offers, trades…
which must be received by the Market Participant
– Awards, obligations, trades and bid submission results, which are
published to the Market Participant(s)
– Alerts which must be acknowledged
– Trade confirmation or rejection
– Notifications, which may be sent via web services or e-mail without
acknowledgement
– Large document download/upload request reply (settlement statements,
extracts, models etc..)
5
External Interface Strategy
High Level Architecture
Market
Participant
Simple & Consistent Web services interface
Web
Service
Interface
Comprehensive security
Enterprise
Identity
Management
& Security
Reliable Web services infrastructure
Enterprise Integration Layer
MMS
EMS
CRR
6
NMMS
Settlements &
Billing
EDW
External Interface Strategy
etc
Implementation Approach
• Define Web services needed for Market Participants to request
market information and initiate transactions
– Leverage IEC 61968 for definition of message
– Use the IEC Common Information Model (CIM) where appropriate
• Provide the means for Market Participant Systems to receive
notification messages from ERCOT systems
– Leverage OASIS WS-Notifications specification
– Participants can decide to pull notifications or have notifications be
pushed to them
– Requires only two Web service interfaces, due to the nature of use
by ERCOT (e.g. subscriptions are predefined and persistent, where
there is no need for subscription requests)
7
External Interface Strategy
Application of IEC 61968 to Web Services
•
•
•
•
•
•
•
Message structures based upon IEC 61968-1 defined for the input and
output of operations (i.e. request and response)
Header used for information needed for all messages, includes key
control information such as verb, noun and source
Request structure has parameters (some required, some optional) that
are appropriate for the specific operation
Reply structure indicates success, failure and errors as appropriate
Payloads can be strongly typed or loosely coupled from message
structures, where payloads can be compressed and encoded if needed
Message structures used within WSDL
Messages form the body of the underlying SOAP message
8
External Interface Strategy
Web Service Message Payloads
• Message payloads carry the data
that is needed for a specific
request or reply (e.g. a set of bids
and offers)
• The noun in the message header
identifies the type of the payload
• Payloads can be supplied in one of
the following forms:
– Any type of XML element of any
namespace (with structure
typically defined by an XSD)
– An XML document that is encoded
as a string, compressed and
base64 encoded
– Other base64 encoded content
9
External Interface Strategy
Asynchronous Messages
Notification
Consumer
Pull
Notification
Consumer
Notification
Consumer
Notify
Notify
GetMessages
Notification Broker
Integration Layer
Participant URLs,
PullPoints
Notify
Notify
•
•
•
•
•
WS-Notifications is new OASIS standard for publishing
messages using Web services
Market Participants (notification consumers) can choose to
pull (i.e. poll for) notifications, or have them pushed (push is
recommended for all Market Participants that can support
the Web service receiver)
Requires only two Web service interfaces (Notify,
GetMessage), due to the nature of use by ERCOT (e.g.
subscriptions are persistent)
Push consumers must expose a Web service
Many different types of notifications can be supported, the
consumer can choose how to handle each type
10
Notification
Publisher
Notification
Publisher
External Interface Strategy
Security Requirements
•
•
Industry good practices
– Authentication
– Confidentiality-protection
– Integrity-protection
– Prevent Replay Attacks
– Access control
– Auditing and logging
– Consistent availability and performance
Business requirements with security implications
– Design a solution that is simple and readily implementable
– Use industry standards
– Enable leveraging of existing security infrastructure
– Provide origin authentication at the transaction level
11
External Interface Strategy
Basic Message Flow
Web Services
Client
1. Signed SOAP
message over HTTP/SSL
with
mutual authentication
Web
Services
Server
Signature
verification
3. Signed SOAP
message
Web
Service
Message
Processor
2. SSL Certificate validation and
authorization check
4. SOAP Message
Certificate validation and
authorization check
Security
Infrastructure
12
External Interface Strategy
Security Standards
•
•
Transport level security via SSL/TLS
Message level security via signed SOAP messages according to the Web
Services Security (WSS) standards
13
External Interface Strategy
Trust Model
• Use Verisign Trust Network (VTN) Class 3 certificates
• Provide two levels of authentication
– Transport
– Message
• Implementations must ensure that :
VTN Public Key Infrastructure (PKI)
– Only ERCOT brand of certificate is used
– Certificate has not been revoked
Market
Participant
End Users and/or
Processes
Signed
Message
Market
Participant
MP-Specific Authentication
14
ERCOT
ERCOT-Specific Authentication
MP-Specific Authentication
End Users and/or
Processes
Signed
Message
End Users and/or
Processes
External Interface Strategy
Authentication
• Transport-level
– Implement mutual authentication via SSL/TLS
• Message-level
– Use OASIS Web Services Security (WSS) 1.1 standards
• SOAP Message Security
• X.509 Token Profile
– Signing entities are expected to be applications/systems with
appropriate certificates
15
External Interface Strategy
Transport Level Confidentiality and Integrity Protection
• SSL/TLS is required while data travels on the Internet
• Deployments may implement additional data protection, based on
their own security infrastructure
• Minimum security settings for SSL/TLS
– SSL/TLS version MUST be at least 3.0
– The SSL/TLS cipher suite MUST include AES 128 bit for encryption
– The SSL/TLS cipher suite MUST include SHA-1 for integrity
protection
16
External Interface Strategy
Preventing Replay Attacks
• Each message must include:
– A timestamp as specified by WSS <wsu:timestamp>
– A random number as specified by WSS <wsu:nonce>
• Clocks need not be synchronized
• Typical implementation
– Reject expired messages
– Cache the nonce for the expected amount of clock skew and reject
messages whose nonce is in the cache
17
External Interface Strategy
Implementation-Specific Security Functions
• Access Control and Auditing
– Use authenticated identity of message producer, as determined by
the signing certificate of the SOAP message
• Consistent performance and availability
18
External Interface Strategy
What does this mean to Market Participants?
• Market Participants MUST
– Deploy SSL/TLS
• Obtain client side certificate
– Certificate is issued by Verisign under the ERCOT brand
• Implement mutual authentication
• Ensure minimum SSL/TLS security settings
– Secure SOAP messages
• Obtain application/system signing certificate
– Certificate is issued by Verisign under the ERCOT brand
• Sign all SOAP messages
• Use Web Services Security Standards and its X.509 Certificate Token Profile
• Message headers MUST include a timestamp and a nonce
• Validate all SOAP messages
– Signature
– Certificates
– Revocation status of certificates
– Timestamp and nonce (to prevent replay attacks)
19
External Interface Strategy
Security Summary
• Web Services Client will need at most two certificates
– SSL/TLS
– SOAP Message Signing
• Security rules are equally applicable to request, reply and
notification messages
• Some security functions are deployment specific and each MP
must determine the best method of implementation
• Implementation of industry standards promotes interoperability
and flexibility
20
External Interface Strategy
Delivery Approach
• In advance of a Nodal go live and in accordance with agreed
schedule and dependencies, ERCOT will provide the following:
– Interface specifications for web services
– Design artifacts, including XML schemas and WSDLs
– Source code examples
– A sand box environment for testing and validation of the interfaces
• The interface specifications, artifacts and implementation of the
sand box environment would be staged
• An iterative implementation approach will be used, where
feedback from each stage would be used to adjust subsequent
stages
21
External Interface Strategy
Technical Interoperability
• Strategy is to achieve technical interoperability through the use of
open standards, including:
– W3C standards
– OASIS WS-* standards
– IEC Common Information Model and related standards (e.g. IEC
61968-1)
• The implementation of Web service interfaces must not be
dependent upon any proprietary, third party products
• Key requirement is that implementation must be possible using
tools that support SOAP-based Web Services (e.g. Java and .Net
development tools)
• More details on technical interoperability will be provided as a
consequence of existing Web services provided by ERCOT,
detailed design and experience with the Sand Box environment
22
External Interface Strategy
Service Level Agreements
•
•
Different categories of Web services will have different service level agreements
The SLAs for some Web services are directly impacted by the variability in the amount of data
that can be transferred (e.g. large bid sets)
•
Market Participants will also have SLA obligations as related to Web services for notifications
•
Web Services for bidding:
– Must be over X% available for any given trading day
– Validation of bid sets will be processed within M1 minutes
– Bid set inquiries will be processed within S1 seconds, with an additional S2 seconds per bid
•
Web services for obtaining real-time market information and providing acknowledgments and
confirmations:
– Must be over X% available for any given trading day
– Requests involving less than 10KB data should be processed within S3 seconds, unless
otherwise specified for a specific interface
•
Web services for other than real time transactions and information requests:
– Must be over Y% available
– Requests involving less than 10KB data should be processed within S4 seconds, unless
otherwise specified for a specific interface
•
Notification interfaces:
– ERCOT will post notifications to a Market Participant within S5 seconds of the time of internal
posting
– Notification service interface provided by Market Participant should be minimally Y% available for
any given trading day. Any ‘downtime’ or periods of inaccessibility will directly impact the
timeliness of notifications (e.g. validation of bid sets, alerts, etc.) from ERCOT
Note: These values and additional SLAs will be finalized at some point in the future, after the
finalization of vendor requirements
23
External Interface Strategy
Auditing, Monitoring and Management
• ERCOT will perform auditing, monitoring and management for all
services it provides
• Internal auditing by ERCOT will be used to track and insure that
SLAs are met by ERCOT
• All Web service requests will be logged by ERCOT in order to
permit calculations related to SLAs
• The signatures supplied on SOAP messages will be recorded with
transactions as a means of non-repudiation
• ERCOT is not responsible for the monitoring and management of
Market Participant software and network connectivity, and
therefore can not guarantee that notification interfaces provided
by Market Participants are accessible as needed for timely
delivery of notifications
• Delivery of Alerts are guaranteed based on Market Participant
acknowledgements
24
External Interface Strategy
Versioning
•
•
•
•
•
•
It is important to recognize that new versions of interfaces may be
provided over time, largely as a consequence of:
– Staging of initial implementation
– New requirements
– Upgrades to vendor products
New versions of interfaces will be manifested by:
– Changes to WSDLs
– Changes to XML Schemas
– Changes to software implementations
New versions will be deployed within a Sand Box environment for a
testing/trial period
WSDL and XML schemas namespaces will include a date reference
Messages will use the Header/Revision field to identify a specific
revision, in order to enable ERCOT software to handle the desired
version of a message definition for a given interface
A detailed versioning strategy will be developed and provided to Market
Participants as a part of the External Interface Specification
25
External Interface Strategy
Governance
• The Web service interfaces will be critical to the operations of
both ERCOT and Market Participants
• The Web services will evolve for many reasons, especially as the
needs of the market evolve
• Governance policies and processes will need to be defined for the
Web service lifecycle that provide strict guidelines related to:
– Design
– Implementation
– Testing
– Deployment
• A comprehensive governance strategy will need to be developed
and implemented by ERCOT with input from the TPTF and the API
Subgroup
26
External Interface Strategy
Web Service Configuration Standards
• ERCOT will configure its Web servers with specific parameters
that may be of consequence to use of Web services by Market
Participants (e.g. for security)
• Market Participants will need to set up Web services to handle
notifications from ERCOT
• ERCOT will define specific configuration details and parameters
to be used by Market Participants
• Detailed Web service configuration standards will be provided to
Market Participants as identified through detailed design efforts
and experience with the Sand Box environment
27
External Interface Strategy
Public Forum
• A public forum will need to be provided within an ERCOT web site
• This would provide for:
– Posting of interface specifications
– Posting of design artifacts (e.g. XML Schemas, WSDLs)
– Pages for frequently asked questions (FAQs)
– Code examples
• Some information may not be publicly posted, and would only be
available to QSEs (e.g. Security Detailed Design document)
• Consideration can be given to the use of a UDDI server to support
the development efforts of the Market Participants
28
External Interface Strategy
Sand Box Environment
• Sand Box environment will:
– Allow for the testing of machine-to-machine interaction between
Market Participants and ERCOT
– Provide a platform where interfaces can be incrementally deployed
and validated
– Mimic the machine-to-machine interaction of the future production
environment
– Be used for interoperability testing between the Market Participant
and ERCOT
– Provide a Pre-Market Trial Environment for Texas Nodal
• Staging of the Sand Box:
– First deployment will provide for a simple ‘Who am I?’ service
– Next deployment will provide a small set of Web services with a
loop back stub
– Subsequent deployments would provide incrementally greater
functionality
29
External Interface Strategy
Sand Box: ‘Who Am I?’
Market
Participant
F
i
r
e
w
a
l
l
Proxy
Server
SiteMinder
F
i
r
e
w
a
l
l
HTTP Server
TIBCO BUS
Test Mode
EIF
“Who Am I”
Web Service
Oracle 10g
Policy Server
SiteMinder 6
LDAP
Sandbox initially implements a simple ‘Who am I?’
interface for initial connectivity testing
30
External Interface Strategy
Sand Box: ‘Loopback’
Market
Participant
F
i
r
e
w
a
l
l
Proxy
Server
SiteMinder
F
i
r
e
w
a
l
l
HTTP Server
TIBCO BUS
Loopback Mode
MMS POC
Web Service
EIF
Oracle 10g
Policy Server
SiteMinder 6
LDAP
Sandbox implements several interfaces as described by the
External Interfaces Specification, with mimic response messages
31
External Interface Strategy
Dependencies and Assumptions
• All development of external interfaces is dependent upon access
to the design documentation related to interfaces provided by
ERCOT systems
– Need interface specifications from MMS
– Need interface specifications from EMS
– Need interface specifications from CRR
• The set of specifications developed in the first quarter of 2007
may need to be modified as project teams update their
requirements and design
32
External Interface Strategy