Transcript panel 7
Speaker: Doug McGinnis Company: Exelon Smart Grid Communication Architecture Smart Grid at Exelon 2 Smart Home / Business Smart Meters (AMI) Smart Distribution System Smart Utility • Real-time usage and pricing statistics • A method to enable two-way information flow • Real-time reporting of status and outages • Home Area Network (HAN) composed of smart devices and appliances that know the price of energy • System status, customer outage status, usage and pricing signals delivered to and from location • Automated controls of relays and reclosers. Efficient field force management • More efficient data collection, processing and back office functions • Asset Monitoring • Effective interconnection of renewable energy sources Leveraging integrated communication systems and information processing is critical Customer End-Use Advanced Pricing & Billing In-home Devices PowerCo PowerCo 1234567 1234 Vehicle Electrification Plug-in Hybrid Electric Vehicles Customer Gateway Advanced Metering Infrastructure (AMI) Grid Modernization Distribution Automation Smart Substations Renewable Interconnection 1. 2. 3. 4. 5. 6. 7. 8. Security – Robust end-to-end, aligned with NISTIR 7628 Deterministic – Smart Grid applications will share a logically isolated deterministic communications infrastructure. Interoperable – Industry standard protocols will be utilized with a focus on migrating to IP/Ethernet consistent with industry direction Privately Owned – Privately owned communications is preferred. Eliminate telecom circuits, O&M cost savings. No Unanalyzed Single Points of Failure (Self Healing) – The communication architecture will be designed with no unanalyzed single points of failure. Communications Maintenance Management & Monitoring – Inherent to the communications Architecture will be Communications Maintenance Management & Monitoring, i.e. the ability to maintain, monitor and control network devices. Tiered Architecture – Unit/Distribution substations are linked to larger Transmission substations in a hierarchical design through new wireless technologies Relay Protection Communications – Highest level of reliability and availability including diverse backup paths. Substation Application Portfolio – 6 application groups exist in the substation and are logically partitioned over common transport ◦ Telemetry – RTU/IED – EMS/DMS communications (encrypted) ◦ CIP Telemetry – NERC CCA sites (encrypted) ◦ DA – Field Distribution Automation traffic aggregated by TGB (encrypted) ◦ Enterprise – Business applications (email, VoIP, surveillance video) ◦ Security – Card readers ◦ AMI – TGB backhaul handoff to Core PoP Tier 1 – Substation will be a point-of-presence to the SONET backbone. ◦ SONET infrastructure provides Ethernet and TDM provisioning ◦ Routing capability to permit application provisioning over Ethernet (Layer 3) ◦ Quality of service management, data prioritization Tier 2 – RF bridge between TGB’s and non-PoP substations ◦ WiMax/Ethernet transport ◦ VLAN partitioning per application for separation Relay Protection Communications Parallel dual SONET ring configuration (OC3 & OC48) with PoP’s at each substation No relay channel failure with loss of a ring Path diversity – loop topology with fast switching to protected path OC3 ring – Primary Relay communications No other applications will use OC3 Static environment OC48 ring – Backup Relay, TDM & Ethernet service GigE provisioned to support Ethernet services TDM circuits for relay protection, voice & serial communications as required Substation LAN Access switch – VLAN provisioned (Layer 2) No inter-application routing will be permitted Telemetry network access/authentication will be through core SCADA Firewall – NERC CIP compliant TGB’s and other substation IP devices will be connected to switch partitioned in their respective VLAN’s Substation WAN Router (layer 3) will interface with switch and will provision Virtual Route Forwarding Tunnels (VRF) 6 VRF tunnels will be created for logical separation VRF tunnels will be encrypted using Dynamic Multipoint VPN (DMVPN) IP addressing schema will be defined for entire substation population based on application requirements Substation WAN Communication Architecture Application Telemetry CIP Tele DA (Field) Enterprise Security AMI Encrypt Yes Yes Yes No Yes No Plymouth ` DFR PC Substation SONET Ring IP ECS DMACS RS-232 1200/2400 Baud Substation Serial RS-232 or DNP/IP DFR IP IED SONET OC48 JMux Ring ` SEL 2020 SI PC Serial TDM RFL 9660 RTU MOB ` TGB DA DFR PC TGB AMI Tier 2 AMI TGB ECS Substation DA Recloser DMACS RS-232 1200/2400 Baud ` Work Station Network Core VRF Tunnels Substation Telemetry VoIP VLAN extended to switch per Application Camera CardReader Switch RTU SCADA CIP Telemetry Firewall Router Gigabit Ethernet Field DA Enterprise Core Router Switch Security AMI Security DA TGB Incorporates Layer 3 VRF Tunneling and Dynamic Multipoint VPN AMI TGB Ethernet based devices Enterprise Firewall AMI/RNI Transport the Smart Grid application portfolio ◦ AMI backhaul – (200kbps/TGB @ 70 Aggregation pts) ◦ Distribution Automation (100kbps/TGB @ Aggregation pts) ◦ Substation Telemetry (56kbps/substation) ◦ Voice/Video (~1Mbps per video stream) Application Traffic Considerations ◦ ◦ ◦ ◦ Bandwidth consumption (5-20Mbps) Latency sensitivity (QoS tagging) Security (PKI) Logical provisioning of applications (VLAN tagging) Spectrum Options – Licensed vs. Unlicensed Unlicensed (ISM) Licensed • Free • High cost – Spectrum Market • Limited Power • Strict FCC deployment rules • Uncontrolled Noise Floor • High Power Operation • Uncontrolled Interference • Low/controlled Noise Floor • 900 MHz – Hi Noise • Interference Remedy • 2.4 GHz – Hi Noise •700 MHz Public Safety • 3.65 GHz (Lite License) • 900 MHz • 5.X GHz • 2.3 GHz Free-space path loss is proportional to the square of the distance between the transmitter and receiver, and proportional to the square of the frequency of the radio signal. ◦ Higher the frequency – Lower the signal propagation ◦ What type of coverage are you planning for? Blanket umbrella (target lower frequencies with minimal interference characteristics) Surgical microcell (target higher frequencies) Urban (target lower frequencies with minimal interference characteristics) Suburban/rural (depends on the type of coverage, Blanket vs. Surgical) Frequencies Requirements 700Mhz 900Mhz 2.3Ghz 3.65GHZ 5.8Ghz 6-11Ghz Risk High High High Medium Low Low Cost Low Low High Low Low High Excellent Adequat e Good Good Good Excellent Limited Good Growing Growing Good Good √ √ √ No No √ Unlicensed No √ No √ √ No Lightly No No No √ No No √ √ √ √ √ √ No No No No √ √ Point-to-Multi Point √ √ √ √ No No Overall Ranking 2 6 5 1 3 4 Coverage Equipment Availability Licensed Availability – PECO area Point-to-Point Ranking: 1 high - 6 low UNIT SUBS MOB Tier II Point-to-Point Point-to-Multipoint RTU’s Corporate SONET RING Plymouth Stimulus Substations Selections DAX Switching Substation SONET Ring x # of Substations Berwyn Skyscraper AMI Meters Point-to-Point TGB Collector to Tier II Interface Point-to-Point Tier II AMI Collector Point-to-Point UNIT SUB UNIT SUB Tier II Devices and wireless connections labeled in RED Two prevailing backhaul Standards WiMax 802.16 WiFi 802.11 • Long Distance 5-10 Miles • Shorter range 1-2 Miles • P2P & P2MP • Meshing capability • Time Slotted Access • Contention Access DSSS • Mobility (802.16.e) • 22 MHz Channels • 5-10 MHz Channels • 802.1Q (VLAN, QoS) • 802.1Q (VLAN, QoS) • Security EAP/TLS/WPA 2 • Security EAP/TLS/AES Backhaul 100 AMI/DA collector sites Provide backhaul of non-fiber substations Requires surgical deployment, not blanket coverage ◦ Point-to-point links ◦ Microcell canopies Decided WiMax using 3.65 GHz Lite-License ◦ Minimal noise at this time ◦ Do not require full territory coverage ◦ Point-to-point links where necessary C Tier II Wireless Backhaul Simple Network Design Security Telemetry (undecided) C Point-to-Multipoint Point-to-Point Unit Substation (undecided) AMI TGB Collection Points Tier I Core Network T CIP RTU Serial RTU RTU (BSTUN) (Sage 2300) RTU Converter Security Devices Enterpise Devices AMI DA (Card Readers, Cameras) CGS 2510 Ethernet Switch ` Telemetry VLAN CIP Telemetry VLAN Field DA VLAN Security VLAN Enterprise VLAN AMI VLAN Cisco 2010 Grid Router SONET GigE Ring MOB Plymouth Cisco 3945 Cisco 3945 (RTU Termination via BSTUN) (RTU Termination via BSTUN) Cisco 3945 Cisco 3945 Switch Switch Firewall Firewall BSTUN BSTUN SCADA VRF Security VRF Enterprise VRF AMI VRF Enterprise Application Services Enterprise Application Services (Voice, Video, email, IM, CRM, AD, DNS) (Voice, Video, email, IM, CRM, AD, DNS) Netflow Sy slo CSM RADIUS (ACS) g SNMP Traps Syslog Ne w tflo SDEE, RDEP Retrieve IPS Alerts