Transcript CA*net II

Optical Networks for the Rest of Us
“Customer Empowered Networking”
NANOG 17 – Montreal 1999
http://www.canarie.ca
Background Papers on Gigabit to
The Home and Optical Internet
Architecture Design Available
Optical Internet News list:
Send e-mail to [email protected]
[email protected]
http://Tweetie.canarie.ca/~bstarn
Tel: +1.613.785.0426
CANARIE Inc
 Mission: To facilitate the development of Canada’s
communications infrastructure and stimulate next generation
products, applications and services
 Canadian equivalent to Internet 2 and NGI
 private-sector led, not-for-profit consortium
 consortium formed 1993
 federal funding of $220m (1993-99)
 total project costs estimated over $500 M
 currently over 140 members; 21 Board members
 Phase III funding to be announced 1998-2001
 $55 million announced for Optical Internet -March 1998
CA*net 3 National Optical Internet
Consortium Partners:
Bell Nexxia
Nortel
Cisco
JDS Uniphase
Newbridge
CA*net 3 Primary Route
CA*net 3 Diverse Route
GigaPOP
ORAN
Deploying a 4
channel CWDM
Gigabit Ethernet
network – 700 km
Netera
SRnet
MRnet
Condo Dark Fiber
Networks
connecting
universities and
schools
BCnet
Calgary
Regina
Winnipeg
ONet
Vancouver
Seattle
Los Angeles
16 channel DWDM
-8 wavelengths @OC-192
reserved for CANARIE
-8 wavelengths for carrier
and other customers
Chicago
Multiple Customer
Owned Dark Fiber
Networks
connecting
universities and
schools
Deploying a 4
channel Gigabit
Ethernet transparent
optical DWDM–
1500 km
ACORN
St. John’s
Charlottetown
Fredericton
RISQ
Montreal
Halifax
Ottawa
STAR TAP
Toronto
New York
What is an Optical Internet?
 WDM fibers where individual wavelengths are the link layer
interconnect directly connected to routers via Optical ADM
(Add Drop Mux) or WDM coupler
 High Performance Router acts as the main switching routing
device
 Bypass or cut-thru connections via dedicated wavelengths
 SONET or Gigabit Ethernet framing (also 10xGbE or SDL)
 Use intrinsic self healing nature of Internet for redundancy and
protection (don’t require SONET/SDH layer)
 Traffic engineering and network management done via MPLS
 Network design optimized for unique characteristics of Internet
traffic – fractal traffic, asymmetric traffic and congestion at the
edge
Lessons Learned
 Carrier transport people now must learn to deal with customers directly
 Require network management tools that give customer a view of
“their” wavelengths
 A whole new set of operating procedures required
 OAM&P issues between router vendors and DWDM remain a challenge
 SONET management systems expect to see a contiguous network
 CA*net 3 required DCC work arounds
 Need network tools to measure end to end performance and throughput at
OC-48 or greater speeds –
 HP is about to release a couple of beta products
 MPLS is proving a lot more difficult in practice to implement




Need tools for management of tunnels
Need Inter-domain MPLS-TE
Mythology of 50msec “fast restoral” still not understood
OSPF with very short hold down timers and GRE tunnels or policy routing
may be an adequate alternative
10xGbE & CWDM
 Several companies have announced long haul GbE and
CWDM with transceivers at 50km spacing
 10GbE coming shortly
 Costs are as little as $12K US per node (or transceiver)
 Future versions will allow rate adaptive clocking for use
with “gopher bait” fiber, auto discovery, CPE self manage
 Excellent jitter specification
 Most network management and signaling done at IP layer
 Anybody with LAN experience can build a long haul WAN
– all you need is dark fiber
 With CWDM, no EDFA power disbursement
 Repeater distance independent number of wavelengths
Market drivers for GbE in the WAN
 Many ISPs, regional networks, municipalities, school districts are
purchasing dark fiber or building dark fiber networks up to 1000 km
rather than managed bandwidth
 With dark fiber increased bandwidth only entails upgraded equipment
costs and no additional monthly charges
 Significant savings in relocating servers to central site and using VoIP
 Also many carriers willing to sell “gopher bait” fiber (fiber that does not
meet stringent SONET/DWDM requirements) at a discount
 As such, cost of transmission equipment is becoming a significant factor
versus cost of fiber
 SONET and ATM networks require specialized engineering knowledge
and skills
 Customers want a technology in the WAN they are familiar with and that
is easily extensible from the LAN e.g. Ethernet
 Don’t require the same reliability as telco systems
Major Long Haul Components
for IP/DWDM network
SONET Transport
Terminal
WDM Coupler
$20K
50 km
Transponder
SONET
Regen
$250k per Tx/Rx
Wideband
Optical Repeater
$250K
250 km
Terabit Router
$400K
Approximate Distances for OC-192 system
Typical Cost $6000 per km (not counting cost of fiber
router, and transponder) for one OC-192 channel
Advantage – can support multi-services and well known
technology
For transponder currently
using regen box
$125K
Major Long Haul Components
for 10xGbE CWDM network
50 km
CWDM Coupler
$5K
G
G
G
G
10xGbE Switch
$20K
10x Transceiver
$20K
Approximate Distances for 10xGbE system
Typical Cost $400 per km (not counting cost of fiber
Or 10xGbE switches) for 10 Gbps
Advantage – very low cost 1/10 cost of SONET & DWDM
Disadvantage – requires 2 fibers and can only carry IP traffic
Optical Networks for the Rest of Us
 With customer owned dark fiber, 10GbE and 4 channel CWDM
anybody can build a 40Gbps network up to 1000km or greater
at a fraction of cost of traditional telco network
 May not be suitable for mission critical traffic (at least not yet)
 But ideal for high bandwidth Internet to the school, small
business and home
 Ring structures are a customer option – not a mandatory
requirement
 The driver is NOT new applications, but cost savings –1 year
payback
 Typical cost is one time $20K US per school for a 20 year IRU
 In Ottawa we are deploying a 60km- 96 strand network
connecting 22 institutions – cost $500K US
Where are we going?
 Today the Internet is “virtual” network riding on top of a traditional
“connection oriented” network of cooper and fiber
 With optical technology such as customer owned dark fiber,
customer owned wavelengths, 10GbE etc we can extend the model
of the Internet as tool to empower the user to build networks in the
physical domain as well as the virtual domain
 The future telecommunication’s world may be dominated by
thousands of customer owned networks that peer at the physical as
well as at the virtual level, “Optical Networks for the Rest of Us”
 A national or provincial K-12 network with its own wavelengths and dark
fiber
 A national banking network with its own wavelengths and dark
fiber
 A national auto network with its own wavelengths and dark fiber
 A radical departure from the “carrier centric” view of the universe
2 Different Views
Telco Network
“Ring of Rings”
99.999% reliability only in the SONET
Ring for the telco, no guarantees for the
customer
SONET ring
CO
ISP B
Customer Empowered Network
“Warp and Weave”
Customer responsible for building rings
99.999% reliability to the customer but
no guarantees for the ISP
ISP A
3 separate dark fiber builds
ISP C
Customer Empowered Networks
Dark fiber Network
City C
Dark fiber or CWDM Network
City A
ISP B
ISP D
ISP C
ISP A
Dim Wavelength
Long Haul DWDM
Optical Label
Switched Router
ISP C
ISP A
First Dark fiber Network
City B
Customer achieves 99.999
reliability by multi-home
to different ISPs
ISP B
Second Dark Fiber Network
New Challenges and Opportunities
 “Customer empowered networks” present a whole new set
of research challenges:
 Peering and topology protocols in the optical domain –
what will be the equivalent to BGP and OSPF in the optical
domain
 Multi Protocol Lambda Switching?
 Defining LSP attributes such as power level, wavelength,
encoding, etc?
 Interdomain optical MPLS?
 Customer controlled establishment of wavelengths, routing
and service delivery
 Auto discovery of wavelengths?
 Management and interface systems, etc, etc