Transcript CCET Discovery Across Texas Security Fabric Demonstration

CCET Discovery Across Texas
Data Quality Plan - Overview
November 15, 2012
John Ballance – EPG
Data Quality Plan


10.11.12
In order for ERCOT to achieve a production quality phasor monitoring that can be
relied upon in real-time operations, three conditions must be met:
―
The data must be flowing reliably from the PMU to the operator’s console,
―
The data must be valid, and
―
The data must be monitoring the critical locations (right places).
The Data Quality Plan will address the first condition
Page 2
Dropout is Observable
 RTDMS Reports show significant dropout data on some
signals (60-99%)
 Observation of ePDC Management Tool shows major
dropouts (60-99%) in real-time data streams
 Data availability testing performed on the RTDMS DB
confirms the presence of data dropouts
 Examination of one-second and one-minute tables in RTDMS
database confirms down sampled data is missing on signals
as well
10.11.12
Page 3
Goals
 Identify nodes in phasor network affecting data
availability (i.e., data dropouts)
 Classify identified dropout issues by severity and
frequency
 Determine likely causes of data dropouts at
identified locations
 Propose solutions to help eliminate identified data
availability problems
10.11.12
Page 4
Phasor Data Network
RTDMS Server
AEP
ePDC
PMU/PDC
PMU/PDC
PMU/PDC
C37.118
Data Stream
C37.118
Data Stream
ODBC
Data Archive
ODBC
Data Archive
RTDMS
Database
Phasor
Archiver
Phasor
Archiver
ADO.NET
Data
Retrieval
RTDMS ISG
Oncor
PDC
RTDMS Reports
Phasor
Database
RTDMS
Client
10.11.12
Page 5
RTDMS
Client
RTDMS
Client
Possible Locations of Data Dropout
 RTDMS Reports (and/or RTDMS ISG) to RTDMS database
interface
 RTDMS Server to RTDMS Database interface
 ePDC to RTDMS Server interface problems
 ePDC input and/or output configuration problems
 Problems on the inbound ePDC stream (e.g. T.O. PDC/PMU
problems, PMU installation problems, etc.)
10.11.12
Page 6
Review Process
 Dropout Location Identification
―
From end-point back to source
 Severity Classification
―
Minor – less than 1% loss of data
―
Moderate – 1-5% loss of data
―
Severe – greater than 5% loss of data
―
Issues may be recurring or non-recurring
 Likely cause determination
10.11.12
Page 7
Data Sources for Investigation
Extracts from the following sources for a 24 hour
period will be used to analyze data quality:
 ERCOT RTDMS Data Base
 ERCOT Phasor Archiver Data Base (from ePDC)
 AEP Phasor Archiver Data Base
―
this represents the stream being sent to ERCOT
 Oncor recorded PDC Output Stream
―
10.11.12
Page 8
this represents the stream being sent to ERCOT
Likely Causes of Data Loss




10.11.12
Application configurations, including
―
ePDC I/O settings
―
RTDMS Server settings
Database
―
Database scripting errors
―
Database to server configuration problems
Hardware
―
Improper host machine configurations
―
Physical network or firewall problems
Source Measurements/Incoming Data
―
Improper or undocumented PMU/PDC configurations
―
Unavailable or dead PMU signals
―
Communication link between TO PMU/PDC to ERCOT ePDC
―
Firewall, Router, IP address issues
Page 9
Likely Solutions
The difficulty of identifying a cause, and the time and resources required to
implement a solution may vary considerably for different problems. In
general,
 Configuration issues will be fairly easy to identify and eliminate
 Database and hardware-level problems may be straightforward to
identify at a high level, but difficult to pinpoint if the cause is an
individual piece of network hardware or a corrupt database.
 Incoming data dropouts will be relatively easy to identify after local
problems have been eliminated, but solutions will require cooperation
with the entities that manage and document the remote PMU or PDC
installations.
10.11.12
Page 10
Study Plan
 Select date for data base extract
―
AEP-ERCOT data stream intermittency (mid-September) has now
been resolved
 Extract the data and send to EPG
 EPG will perform analysis, recommend configuration tuning
and potential network/server interface adjustments
 EPG will prepare a summary report and recommendations
10.11.12
Page 11
Questions
10.11.12
Page 12