Transcript Gaylord-VaTech-CaseStudy-Atlanta

Virginia Tech Campus
Network Overview
Internet2 Campus Focused Workshop
On Advanced Networks
Clark Gaylord
Virginia Tech
[email protected]
http://rdweb.cns.vt.edu/
30 May 2001
Basic Services
• Basic Ethernet portal service (~25,000 ports)
– Switched 10Base-T (10 Mbit/s)
– Some switched 100Base-TX (100 Mbit/s)
• Dial-in modem pool with ~1,200 56k modems
• Wireless LAN (currently in pilot phase, beginning
second round deployment)
• ATM portal service (25 or 155 Mbps), mostly for
specialized applications (e.g. distance learning
programs)
• Local network access point provides campus
backbone access to local ISPs. Ethernet in
apartment complexes
• Telephone (~13,000 phones) and cable
television (~5,000 connections)
30 May 2001
Gaylord: Virginia Tech Campus
Overview
2
Some Network Management Tools
• Largely home-grown
–
–
–
–
ARP/Forwarding table/Cable plant databases (HC++ ™)
Device status reports
TACACS+ database query tool
nemisys - Network Engineering Management Information
SYStem ™
– Various ad hoc tools
– People
• Commercial
– Smarts InCharge
– Concord Network Health
– InterMapper, What’s Up “inventory” tools (“Your switch is still
alive.”)
• Freeware, etc
– Usual suspects: MRTG, Big Brother, Netsaint
30 May 2001
Gaylord: Virginia Tech Campus
Overview
3
ARP/MAC mappings 1
30 May 2001
Gaylord: Virginia Tech Campus
Overview
4
ARP/MAC mappings 2
30 May 2001
Gaylord: Virginia Tech Campus
Overview
5
Roll your own
30 May 2001
Gaylord: Virginia Tech Campus
Overview
6
End-to-End Management
•
•
•
•
Largely ad hoc, informal measurements
ttcp, netperf, treno, Ganymede
nlanr AMP project
How to you obtain measurements from the far
end?
• Usual SNMP tools cannot/do not address
burstiness
• Configuration management/production quality
30 May 2001
Gaylord: Virginia Tech Campus
Overview
7
ISM
30 May 2001
Gaylord: Virginia Tech Campus
Overview
8
AMP
30 May 2001
Gaylord: Virginia Tech Campus
Overview
9
Standard Building Network
• Switched 10 Mbps to the
desktop.
• Switched 100 Mbps where
desired.
• Switched 100 Mbps building
backbone.
• N x 100 Mbps scalable
building feed.
• Simple model
• ISL virtual LAN capability
Catalyst
1900/1924
Catalyst
1900/1924
100Base-FX downlinks
Catalyst
1900/1924
Catalyst
1900/1924
Catalyst
1900/1924
10Base-T
to wall portal
n x 100Base-FX
Cisco
Catalyst
5000
30 May 2001
Gaylord: Virginia Tech Campus
Overview
Cisco
Catalyst
6500
Core
10
Advanced Building Network
Catalyst 3500
Catalyst 3500
1000Base-TX
inter-switch links
Catalyst 3500
Catalyst 3500
Catalyst 3500
Catalyst 3500
Wireless LAN
Access point
• 10/100 available to the desktop
• Scalable Gigabit Ethernet
building backbone and building
feed
• 802.1Q/ISL virtual LAN
capability
• 802.1p packet prioritization
capability
• 802.11b Wireless LAN
• Deployed in Torgersen Hall
Catalyst 3500
Catalyst 3500
Cisco
Catalyst
6500
30 May 2001
10/100Base-TX
to wall portal
n x 1000Base-LX
Gaylord: Virginia Tech Campus
Overview
Cisco
Catalyst
6500
Core
11
Wireless LAN Deployment Plan
(2001-2002)
•
•
•
30 May 2001
Gaylord: Virginia Tech Campus
Overview
Current/pilot:
– Torgersen (ACITC)
– Information
Systems
Summer 2001:
– Library (partial)
– Student Centers
(partial)
– Randolph
(Engineering)
– Pamplin (Business)
2001-2002:
– Veterinary Medicine
– Seitz (Agriculture)
– McBryde (Math, CS)
12
Wireless LAN Deployment
• Two-three year deployment schedule
– Instructional space
– Student centers
– Other space as needed
• Facilities to support portable computing
• Based on 802.11b
– Economics of 802.11a dubious
• Keep it “standards” based
• Large perimeter
– WEP/ESSID not supportable, negligible benefit
• 802.1X/EAP
– a definite maybe
• Airspace internal policy
• Custom antenna design?
30 May 2001
Gaylord: Virginia Tech Campus
Overview
13
VT Gigabit Backbone
2 x 1000Base-LX
1 x 1000Base-LX
1 x 1000Base-SX
1 x 100Base-FX/TX
ATM OC3c
Future POS/DPT/ATM OC12c
or 1000Base-ZX
BUR
Catalyst
6500
HIL
Catalyst
6500
SHA
Catalyst
6500
ISB
Catalyst
6500
OWE
Catalyst
6500
ISB
Catalyst
6000
Modem
Pool
ISB
Cisco
7500
Internet
N.W.V
Internet2
ISB
Cisco
7500
VT
ATM
Backbone
CAS
Catalyst
6500
CAS
Cisco
7500
MSAP
DSL
T1/FR
LMDS
30 May 2001
Gaylord: Virginia Tech Campus
Overview
14
ATM Architecture
• ATM provides key benefits, not yet feasible or
cost-effective using other technologies
– Integrated, multi-service WAN access
• Voice transport (toll-bypass)
• Videoconferencing (distance learning)
• IP data transport
– Layer 2 virtual private networks with explicit resource
provisioning; vital to network layer research.
• Also provides last-resort backup for the gigabit
backbone.
– When all else fails, we haul it over ATM.
30 May 2001
Gaylord: Virginia Tech Campus
Overview
15
VT ATM Backbone
ATM OC3c multi-mode
ATM OC3c single-mode
1000Base-SX
BUR
ASX-200BX
HIL
ASX-200BX
SHA
ASX-200WG
OWE
ASX-200WG
ISB
ASX-1000
CAS
ASX-200BX
ISB
Cisco
7507
N.W.V
Internet
ISB
Cisco
7507
VT
Gigabit
Backbone
Internet2
30 May 2001
Gaylord: Virginia Tech Campus
Overview
CAS
Cisco
7507
16
Net.Work.Virginia
• Provides ATM services throughout Virginia to K-12
schools, institutions of higher education, and state
agencies
• Low cost access with level pricing statewide
– OC3c (155 Mbit/s), ~ $134,000/annum
– DS3 (45 Mbit/s), ~$53,000/annum
– DS1 (1.5 Mbit/s), ~$12,000/annum
• Provides Internet access services, as well as
supporting a wealth of ATM-based services (intranet,
videoconferencing, PSTN toll bypass, etc)
• Provides access to research backbones (e.g. Abilene)
• Currently 595 sites
30 May 2001
Gaylord: Virginia Tech Campus
Overview
17
Net.Work.Virginia
OC-12c POS
OC-12c POS
OC-12c ATM
N x OC-3c/DS-3 ATM
Regional
LEC ATM Regional
LEC ATM
WTN
ASX-1000
Sprint
Public ATM
WTN
GSR
12016
SprintLink
(Internet)
ESnet
WTN
GSR
12016
Abilene
(Internet2)
ROA
GSR
12016
Regional
LEC ATM
Regional
LEC ATM
RIC
GSR
12016
ROA
ASX-1000
RIC
ASX-1000
Regional
LEC ATM
30 May 2001
Regional
LEC ATM
Regional
LEC ATM
Regional
LEC ATM
Gaylord: Virginia Tech Campus
Overview
18
Quality of Service
• Traffic Classification
• Congestion Avoidance
– Policing
– WRED
• Congestion Management
– CBWFQ
– Shaping/Conditioning
30 May 2001
Gaylord: Virginia Tech Campus
Overview
19
Traffic Classification
• DSCP/IP Precedence marking
• Classify different classes of “Best Effort”
to protect each from each other
• Classes can be based on users or
applications
• Use netflow to determine major
applications
• Use business rules to determine major
user bases
30 May 2001
Gaylord: Virginia Tech Campus
Overview
20
Traffic Classification
• Only bits 0-2 are used (DiffServLite?)
– Control Plane (DSCP=56(?) set by routers)
– IP Telephony (DSCP=40(?) set by end-station)
– On-campus any-to-any (DSCP=24)
[Following are destined off-campus]
– Non-residential (aaanet) TCP (DSCP=16)
– Residential (resnet) TCP (DSCP=8)
– Other residential and non-residential (notTCP)
(DSCP=0)
– Should these be xxxx11?
• Except for IP Telephony and control plane,
classification is done at entry to core (DSCP is
only trusted in one edge VLAN, viz. telephony)
• No use of 802.1p (outside of telephony LAN)
30 May 2001
Gaylord: Virginia Tech Campus
Overview
21
Traffic Shaping
• Outgoing only (ATM OC-3 … nominally
100Mbps)
–
–
–
–
–
aaanet 40Mbps
resnet 40Mbps
notTCP 10Mbps
CBWFQ
No priority queuing (at this time)
• No shaping for on-campus traffic
30 May 2001
Gaylord: Virginia Tech Campus
Overview
22
Traffic Policing
• On-campus
– No policing
– All best-effort
– Lots of excess capacity
• Incoming to campus
– Same as on-campus
• Outbound
– Police resnet to 60Mbps (nominal 60% of
Internet pipe) at entry to border area
– Formerly specifically policed “application of
special interest”
30 May 2001
Gaylord: Virginia Tech Campus
Overview
23
Path of a packet
Catalyst
1900/1924
Catalyst
1900/1924
100Base-FX downlinks
Catalyst
1900/1924
Catalyst
1900/1924
Catalyst
1900/1924
DSCP marking happens here
n x 100Base-FX
Cisco
Catalyst
5000
30 May 2001
Cisco
Catalyst
6500
Core
Gaylord: Virginia Tech Campus
Overview
24
Path of a packet
BUR
Catalyst
6500
HIL
Catalyst
6500
Modem
Pool
From building network
DSCP marking happens here
SHA
Catalyst
6500
ISB
Catalyst
6500
OWE
Catalyst
6500
ISB
Catalyst
6000
Policing happens here
Internet
N.W.V
Internet2
ISB
Cisco
7500
ISB
Cisco
7500
CAS
Catalyst
6500
MSAP
DSL
VT
ATM
Backbone
CAS
Cisco
7500
T1/FR
LMDS
Conditioning happens here
30 May 2001
Gaylord: Virginia Tech Campus
Overview
25
What we have learned
• Outbound congestion creates poor inbound
TCP performance
• Conditioning alone does not work well when
offered load is high
• Leave room for bursts (set sum of
committed traffic to 90% of link?)
• Policing alone works for isolated traffic
classes
• Doing anything is better than doing nothing
- even for voice
• Divide and conquer
30 May 2001
Gaylord: Virginia Tech Campus
Overview
26
Futures
• Priority queuing for voice
• Traffic classes for distance learning
(DLoIP ™), Research Computing, et al.
• Per-user classification and user-class
policing
• Netflow data gathering and traffic
classification
• Remote monitoring of end-to-end
performance
• How do we measure bursts?
30 May 2001
Gaylord: Virginia Tech Campus
Overview
27
For More Information
• http://rdweb.cns.vt.edu/
• http://www.networkvirginia.net/
30 May 2001
Gaylord: Virginia Tech Campus
Overview
28