Transcript 20050920-abilene-cotter

Abilene Update
Fall Member Meeting ’05
Philadelphia, PA
Steve Cotter
Director, Network Services
[email protected]
Welcome
•
•
•
•
•
•
•
The Abilene Network
Hurricane Katrina
Advanced Services Across Abilene
Network Research Across Abilene
Other Network Services
Network Security
Abilene Network Futures
2
The Abilene Network
Abilene Partnerships
•
•
•
•
•
Indiana University
Juniper Networks
Nortel Networks
Qwest Communications
ITECs
•
•
•
•
NC ITEC
Ohio ITEC
San Diego ITEC
Texas ITEC
• Internet2 Staff
4
Abilene Network Topology
5
Abilene Network Topology
6
Abilene Network Topology
7
Abilene Network Topology
8
Abilene Network Topology
9
Abilene Network Topology
10
Abilene Network Topology
11
Abilene Community
• 38 direct connections (OC-3c  10 Gbps)
• 3 10 GE connections (OC-192c SONET also supported)
• 7 OC-48c connections & 3 GE connectors
• 26 connected at OC-12c (622 Mbps) or higher
• 240 Primary Participants – research universities
and labs
• Claremont Colleges, New World Symphony, Manhattan
School of Music, Cleveland Museum of Art, Cleveland
Institute of Music, Los Alamos National Lab and Qwest are
the most recent additions
• 130 Sponsored Participants - Individual institutions,
K-12 schools, museums, libraries, research
institutes
• 34 Sponsored Educational Group Participants state-based education networks
See: http://abilene.internet2.edu/
12
Abilene R&E Peerings
13
Abilene International
Peerings
September 2005
14
Abilene Connector Fees
OC-3c
(155 Mbps)
OC-12c
(622 Mpbs)
Gig E
(1 Gbps)
OC-48c
(2.5 Gbps)
10 Gbps
(SONET/
Ethernet)
Original
2003
Fee
$110k ($110k)
(1998)
$320k
$270k
(1998)
$325k
$325k
(2001)
$495k
$430k
(2000)
$490k
2004
2005
($110k)
($110k)
$240k
$220k
$280k
$250k
$360k
$340k
$480k
$480k
15
Abilene Participation Fees
Effective January 1, 2006:
• Abilene Primary Participation - $21,000
Effective January 1, 2007:
• Abilene Primary Participation - $22,000
First increase since Abilene was launched
in 1998
16
Hurricane Katrina
Hurricane Katrina
18
Hurricane Katrina
• Hurricane Katrina strikes the Gulf Coast on August 29th, 2005.
• Abilene’s unprotected lambda network link from Houston to
Atlanta goes down. The IGP (IS-IS) automatically reroutes
around the fault.
• On September 1st, 2005 the damage to the carrier network was
fully assessed and estimated to take days to repair.
• During this time, Abilene was operating with the risk of network
isolation if there is a loss of the Chicago to Kansas City link.
• A redundancy plan was formulated and approved by Internet2 to
have Abilene traffic route over the HOPI wave from Chicago to
Seattle in the event that Chicago to Kansas link fails. The
Abilene NOC engineers implement the redundancy plan.
• Service is restored to the Houston to Atlanta link on September
8th, 2005. No Abilene outages occurred during this period.
19
Hurricane Katrina
H
8
8
8
8
8H
8
8 8
8 8
H
H
20
Hurricane Katrina
We would like to thank our partner
Qwest for the extraordinary efforts
they made to repair the network.
Great job!
We also appreciate the support we
received from the Abilene NOC and
NLR. Thanks!
21
Abilene Redundancy
• Responding to requests of our members, Internet2
has pursued redundancy options with our partner
Qwest Communications.
• Qwest has agreed to provide on a per port basis,
redundant connections to the Abilene router, at the
node, for a cost of $400 per month regardless of
speed as long as the redundant circuit speed is equal
to or less than the primary circuit.
• This option is available to any active Abilene
Connector who delivers their redundant circuit to the
Abilene node. SONET and Ethernet framing methods
would be supported under this option.
22
Abilene Redundancy
Most Abilene Connectors Today:
What We Can Offer:
23
Redundancy Offering
• We can make the following redundant connections available to
our members who bring their circuits to an Abilene node:
• VLAN connections through an existing exchange point:
• 1 GE
$50,000.00
• Physical connections to the router:
•
•
•
•
•
•
OC3
OC12
OC48
OC192
1 GE
10GE
$70,000.00
$75,000.00
$90,000.00
$125,000.00
$80,000.00
$125,000.00
• A redundant circuit must be equal to or less than the primary
circuit in speed and will not carry traffic unless the primary circuit
fails.
• Each request will be evaluated on a case basis. The above
figures are for budgetary purposes and are subject to change.
24
Redundancy Offering
• Member’s requests for redundant circuits
carried back to an Abilene node over the
Qwest network will have their requests
evaluated on a case-by-case basis for
available capacity and pricing.
• These types of connections currently must be
SONET.
25
Advanced Services Across Abilene
IPv6 Peerings
• IPv6 Deployment
• Significant number of peers and connectors now have native
connections:
• Roughly 2/3 of the connectors are IPv6 enabled
• Roughly 1/2 of the peers are IPv6 enabled
• Connected to Palo Alto PAIX peering fabric at 333
Mbps for IPv6 and IPv4-Multicast experimental, nonproduction peering
• 10 new experimental, non-production IPv6 peerings at the
PAIX so far in 2005
• Connected to MCI MAE-West at OC-3 for IPv6-only
experimental, non-production peering
• Qwest and MCI collaborated in providing the connection
27
IPv6 Addressing
• Abilene has /32 that it can distribute to its
members
• However, a number of connectors and
members have or are acquiring their own
address space:
•
•
•
•
2001:4e0::/32 Wiscnet
2001:5e8::/32 Pittsburgh Supercomputing Center
2001:1860::/32 Pacific Northwest Gigapop
2001:18e8::/32 Indiana University
28
IPv6 Security
• Abilene NOC activities:
• Limiting the v6 prefixes connectors send us
(as we do for IPv4)
• Limited filtering for peer networks
• [email protected] is a mailing
list for v6 security topics
29
Internet2 Involvement
with the NAv6TF
• Internet2 is active in the North American
IPv6 Task Force (NAv6TF).
• Rick Summerhill is on NAv6TF advisory
committee
• Abilene is key network component of
the NAv6TF's Moonv6 national test
network
30
Internet2 IPv6
Member Activities
• North Carolina State University and
Centaur Labs -- IPv6 streaming audio
feeds from radio stations WCPE and
WZYC
• IPv6 H.323 at Georgia Tech
• Abilene IPv6-enabled hosts
• http://ipv6.internet2.edu/ipv6hosts.shtml
31
Internet2 Member
Multicast Activities
• DVGuide - http://db.arts.usf.edu/dvguide/listings.asp
• Several campus radio stations multicasting across
Abilene
• ConferenceXP, a Microsoft Research initiative, relies
on multicast and has been deployed at several
schools
• Access Grid continues to grow
• More activity requiring "bridging" to multicast in
challenged environments, using the rcBridge
software from ANU
• NYSERnet, Abilene and Internet2 deploying native
IPv6 multicast
• IPv6 Multicast demo live at Fall Member Meeting
32
Multicast Security
• Basic measures on Abilene:
• Not allowing multicast streams with RFC1918 source
addresses
• Not allowing multicast streams to "site local" group
addresses (239.0.0.0/8) which is a similar idea to RFC1918
addresses, but for group addresses.
• Blocking group addresses which are used for application
which only have local significance. A good example of this
is Norton Ghost.
• Other measures are under consideration, such as:
• Blocking all IANA reserved multicast group addresses
• Place a limit on the number of MSDP SAs each Abilene
Connector/Peer can originate
33
Internet2 Hands-on
Multicast Workshops
Upcoming Workshops:
• Hartford, CT – 4-6 October 2005
• Ann Arbor, Michigan - 17-19 October 2005
• Albuquerque, New Mexico - 2-4 February
2006
http://multicast.internet2.edu/workshops/
34
Other Advanced Services
• MPLS VPN testing – NC-ITEC experimenting
(with ITEC Ohio) with creating a multipoint
Layer 2 VPN using inter-domain MPLS
tunnels and Virtual Private LAN Service
(VPLS). Working in both a lab environment
and between the ITECs using Abilene.
• The goal is to examine multipoint alternatives
for a possible Abilene private network service
offering.
35
Network Research Across
Abilene
Network Research
Philosophy
• Internet2 today does not do network research
per se, but seeks to facilitate and support
research projects led by faculty at member
institutions
• Make accessible network resources readily
available to this community
• Participate in research collaborations and provide
support for proposals
• Integrate research findings into the evolution of
Internet2 network initiatives and services
37
Network Research
Resources
• Resources available to researchers:
•
•
•
•
Abilene Observatory
MAN LAN Exchange Point
HOPI testbed
Collaboration with NLR, Regional Optical
Networks and other testbeds
38
Abilene Observatory
The Abilene Observatory is a program that supports the
collection and dissemination of network data
associated with the Abilene Network.
Provides researchers:
•
•
Operational data associated with a large-scale network
Data associated with the fundamental properties of basic
network protocols.
Two components of the Observatory:
•
•
Data collected by Abilene engineers using equipment
located in the router nodes and operated by the Abilene
NOC
Data collected by separate research projects using
equipment collocated in the Abilene racks
39
Abilene Observatory
There are more than 30 research projects currently using
Observatory data. Some of the more recent additions are:
• Flow Sampling and Anomaly Detection , Paul Barford,
University of Wisconsin
• Assess the Presence and Incidence of Alpha Flows in
Backbone Links, Vincenzo Liberatore, Case Western Reserve
University
• Traffic Management and QoS Provisioning in IP Networks,
Hassan Peyravi, Kent State University
• Spatio-Temporal Network Analysis, Mark Crovella and Eric
Kolaczyk, Boston University.
• MINDS Project, Vipin Kumar, University of Minnesota
• Study of the Temporal-spatial Correlations in Network
Traffic, Don Towsley, University of Massechusetts
For a more comprehensive list, see:
http://abilene.internet2.edu/observatory/research-projects.html
40
Project Highlight:
PlanetLab
• PlanetLab Upgrade
• PlanetLab nodes currently located at all Abilene router
nodes, connected to the IP network
• Upgrade will add connection to an MPLS L2VPN
configuration forming a layer2 network where the PlanetLab
nodes will provide the routing engines
• Abilene becomes the layer2 circuit provider for PlanetLab
• Normal users on Abilene don't have direct access to this new
"backbone network"
• The PlanetLab network can peer with the commodity
network
• Provides an infrastructure for network research that has
national scope
41
Other Network Services:
FiberCo & MAN LAN
FiberCo Overview
• Tool designed to support optical initiatives in the regions or
nationally
• Spun off from NLR governance discussions
• Internet2 took responsibility for forming the LLC
• Operates on behalf of U.S. higher education and affiliates –
Internet2 and NLR membership
• Not an operating entity
• Will not light the fiber – only a holding company
• Functions
• Market maker
• Assignment vehicle for both national & regional optical initiatives
• Dark fiber provider: Level3 Communications
• 3 year pricing agreement ends March 06
• Intercity and metro fiber, new builds, consulting services
• Exploring more formal relationships with other providers
43
State and Regional
Optical Networks
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Alabama*
Arizona (CENIC)
Arkansas*
California (CALREN)
Colorado (FRGP/BRAN)
Connecticut (Conn. Education
Network)
Florida (Florida LambdaRail)
Georgia (Southern Light Rail)
Great Plains Network* (MIDnet)
Indiana (I-LIGHT)
Illinois (I-WIRE)
Louisiana* (LONI)
Massachusetts*
Maryland, D.C. & northern Virginia
(MAX)
Michigan (MiLR)
Minnesota* (BOREAS)
(*RONs with RFx’s issued or in
process of acquiring fiber)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
National LambdaRail
New England Region (NEREN)
New Mexico (NMSU, UNM)
New York (NYSERNet*, Cornell)
North Carolina (NC LambdaRail)
Ohio (Third Frontier Network)
Oklahoma (OneNet)
Oregon
Pacific Northwest (Lariat – NIH BRIN,
PNNL)
Rhode Island (OSHEAN)
SRON* (southeastern U.S.)
Tennessee* (OneTN)
Texas (LEARN)
Virginia (MATP)
Wisconsin (WiscNet)
Wyoming
(RONs in red have made dark fiber
acquisitions through FiberCo)
44
States with Regional
Optical Networks
States with a RON
45
Dark Fiber Placement
• Aggregate dark fiber assets acquired by U.S. R&E optical
initiatives
• CENIC (for CalREN & NLR)
• FiberCo (via Level 3 for NLR & RONs)
• SURA (via AT&T)
6,200+
8,600
6,000
• Plus 2,000 route-miles for research
•
•
•
•
•
NLR Phase 2 (WilTel & Level3)
OARnet
ORNL (via Qwest)
NEREN
Other projects (IN,IL,OR,CT…)
Total (conservative estimate)
5,000
1,500
900
670
2,200+
30,000+
• Over 60% of these assets are now held by RONs
• Remainder held by NLR (~11,250 route-miles)
46
MAN LAN Exchange Point
• Manhattan Landing in New York City - partnership
with NYSERNet, Indiana University, and the IEEAF
• Provides a high performance exchange facility for
research and education networks
• Located at 32 AoA in NYC - easy interconnection to
many national and international carriers and other
research and education networks
• Peering model is open and bilateral
• Cost recovery model - minimal connection charges
for layer 2 facility, none for layer 1 connections
• Working with AtlanticWave on future distributed
exchange point along U.S. East Coast (NYC↔Miami)
47
MAN LAN Services
• Layer 2 - Ethernet switch for IPv4/v6 peering
with 1GigE and 10 GigE interfaces
• Layer 1 - TDM based optical equipment
(SONET / Ethernet interfaces)
• Cisco 15454
• Nortel OME 6500
• Nortel HDXc
• Layer 0 – Glimmerglass optical cross connect
to facilitate changes
48
Network Security
Network Security
Basic Premise: Abilene Security Policy is
determined by the properties of an IP network
• Control is at the edge
• Hosts determine when and where to send packets and
initiate flows
• This control often leads to vulnerabilities
• Hosts can become compromised
• Hosts may be used to compromise other hosts
• Can lead to large amounts of traffic sent to other hosts
As a backbone network, we view Abilene as a
‘pipe’ and not a controlling entity
50
Network Control
The Abilene backbone does have the means to apply
some control across the network:
• It is possible to block traffic on some ports
• It is possible to block all traffic from a particular IP address
Abilene does not unilaterally filter traffic on a
network wide basis unless the network itself is
under attack.
51
Filtering Traffic
Abilene will filter traffic in some situations:
• If one or more hosts on a connector or peer were under
attack
• If requested by an institution, peer, or connector
([email protected], 317-278-6622)
Abilene will filter traffic to a connector or peer if
requested by that particular connector or peer
network, filtering the appropriate traffic through
the connection in question.
• Abilene’s method for blocking this traffic is our BGP
Discard Routing procedure
52
Filtering Traffic
Abilene reserves the right to protect itself and its
connectors / peers from other connectors and peers.
• If a threat to the network exists through a particular
connector, Abilene reserves the right to filter that traffic
• Ultimately, Abilene could disconnect the offending connector
or peer
Abilene reserves the right to filter all traffic or
terminate any connection if it is under attack.
• Note: Every attempt will be made to contact the network in
question to discuss various options and alternatives.
53
Research and Education Information
Sharing Analysis Center (REN-ISAC)
The REN-ISAC supports higher education and the
research community by:
• Provides advanced security services to national supporting
networks
• Supports efforts to protect the national cyberinfrastructure by
participating in the formal sector ISAC infrastructure
Abilene will report all known incidents of security
threats to the REN-ISAC.
54
Data Collection
Abilene collects flow statistics on a sampling basis that
potentially could identify source and destination
addresses and ports
• This data is anonomyzed (11 lower order bits of all IP
addresses are zeroed out) before it is saved to disk
• For privacy reasons: Abilene does not collect data pertaining
to communications between identifiable hosts
• However, this information could identify compromised hosts
During times of security attacks, the REN-ISAC can
unanonomyze data, but only that data related to
the attack itself. The resulting data is
anonomyzed as soon as possible after the attack
is understood.
55
Data Analysis
Information derived from analysis of the flow data that
identifies specific institutions or hosts is treated as
confidential information.
Institutions may request specific sources of cyber
security attacks located on their respective
networks. Only security related information will
be reported to the institutions.
Abilene data is meant to supplement, not replace, data
collected by individual institutions or connectors.
Internet2 strongly encourages institutions to collect
their own data, potentially providing a greater degree
of specificity to particular security problems.
56
BGP Discard Routing
Connectors can advertise routes to Abilene via BGP for which all
traffic to those routes will be discarded by the Abilene routers.
This is useful during a DoS attack because the traffic can be
dropped before it crosses the link to the connector.
Here are a few important points:
• Discard routes will NOT be accepted for routes larger than a /24
• There is no way to place a limit on the number of discard routes
a connector can advertise. The limit on the total number of
routes a Connector can advertise is currently 3,000.
• Abilene's default policy is to not accept routes smaller than a
/27. There have been some exceptions made to this policy. For
those /28 and smaller routes, it will not be possible to announce
more specific discard routes.
57
Abilene Network Security
Actions underway/planned:
• Updated the Abilene Transit Security Policy.
• Planning periodic Operational Security Assessment Excercises
• Work more closely with the REN-ISAC on investigating, validating,
and resolving ongoing security issues.
• Work with the REN-ISAC, Abilene NOC, Advanced Network
Management Lab (ANML) and Arbor Networks to enhance our
security capabilities.
• Deploying Arbor Networks Peekflow SP equipment
• Work with industry and researchers to gather information about
threats and attacks and disseminate this information to the
community.
•
•
•
•
Developing portal views for Abilene Connectors and Peers
Web publish traffic statistics
Fingerprint detection and sharing with other networks
Disseminate alerts when worms and anomalies detected
58
Abilene Network Futures
Next Generation Abilene
Mission of Internet2: To build leadingedge R&E networking capabilities.
This mission rests on belief that evolving
new technologies will drive new network
architectures with a broader set of
services and capabilities.
60
Next Generation Abilene
• Internet2 is focused on integrating and rapidly
deploying innovative new capabilities
• Working to understand how the next generation
architecture will evolve over the next 5-7 year
timeframe
• Numerous discussions with researchers, carriers and
equipment vendors
• Examining how a hybrid of shared IP packet switching and
dynamically provisioned optical lambdas can meet the needs
of the community.
• Continue to engage the GigaPoPs, state/regional
networks and campus environments
61
Next Generation Abilene Design
Considerations
Architectural Design Considerations
• NLR, RON and international integration
• Advanced service support - Multicast, v6, High Performance
Throughput, Measurement
• Enhanced network research facilitation
• Network and end-user security
• The applications that will ride across the network
• Options for increased reliability and additional services
Process
• Hybrid architecture evaluation (HOPI)
• Production IP core network
• Dedicated point-to-point capabilities (’s, MPLS tunnels)
• Evaluation of optical transport capabilities • NLR, commercial providers & RONs
• Design & planning collaboration
• U.S. & int’l partners (ESNet, TeraGrid, SURFnet, GEANT-2)
62
HOPI Resources
Resources available to the HOPI team:
• Abilene Network – 10 Gbps IPv4/IPv6 + MPLS
tunnels
• 10-Gbps  on the NLR footprint
• MAN LAN Exchange Facility
• 10-Gbps λ NYC – London to provide connectivity to the
European testbeds
• Layers 1 and 2 switching gear
• Collaborations with Regional Optical Networks
(RONs) and other related efforts (GLIF, UltraLight,
DRAGON, etc.)
63
Next Generation
Abilene Timeline
• October 2007 - End of recent 1-year Abilene
transport MoU extension
• Sets next-generation network planning timeline
•
•
•
•
•
Architecture definition: 1/1/2006
Transport selection: 4/1/2006
Equipment selection: 7/1/2006
Backbone deployed: 1/1/2007
Connector transition: 2007
• Concurrently, review overall business plan and management
model
• Network design time frame: 2007-2012
• HOPI testbed is expected to be in place for 2-3 years,
to experiment with future protocols
• Refine and evolve next generation architecture
64
Next Generation
Network Roadmap
• 2005-2007
• ‘WaveCo’ – complementary relationship for carrier provided
wavelengths to augment backbone
• Collocation and dark fiber services via FiberCo
• Layer 1 measurement / monitoring
• Interdomain control plane & AAA
• 2008
• Wavelength services
• Static ‘Core’ wavelengths for IP backbone
• Point-to-point unprotected & protected variable duration waves
• GMPLS dynamic provisioning: dynamic set up on the order of minutes
• 40G transport / switching on selected routes
• Optical layer security
• 2009-2010
• GMPLS dynamic provisioning: near real-time dynamic set up
• Alien / transparent wave service
65
Many Thanks to the
Abilene Team
• Heather Bruning – Program Manager, Business
Operations
• Andrea Blome – Asst. Prog. Manager, Business
Operations
• Bill Cerveny – Internet Engineer
• Christian Todorov – Network Engineer
• Ana Preston – Program Manager, International
Relations and RONs
• Members of the Indiana NOC, Abilene Planning
Team, Abilene TAC
And other Internet2 staff and member volunteers who
help make Abilene run.
66
Abilene Information
• For more Information:
•
•
•
•
http://abilene.internet2.edu
http://abilene.internet2.edu/observatory/
http://www.nationallambdarail.org
http://hopi.internet2.edu
• Or contact us at:
• [email protected]
• [email protected]
• [email protected]
67
Questions / Comments?
Thank you for coming.
68