20050719-Jumbo-Jorgenson
Download
Report
Transcript 20050719-Jumbo-Jorgenson
Jumbo Packets
for
Application Performance
Joint Techs Workshop
Vancouver, BC
2005
Apparent
Networks
9k APP Project
Rutherford
Research
CA*net4 – 9k App Project
HEPnet
WestGrid
Subatomic Physics
Rutherford
Research
Apparent
Networks
Internet2 9k MTU Project
Bill Rutherford (Rutherford Research/RRX)
Kevin Walsh (Cal-ITEC/SDSC)
John Moore (NC-ITEC/NCSU)
Loki Jorgenson (Apparent Networks/SFU)
Paul Schopis (ITEC-Ohio/OARnet)
Jorge Serran (CUDI2/UNAM)
Dave Hartzell (NASA Ames Research Center)
Bill Jones (University of Texas-Austin)
Woojin Seok (Supercomputing Korea/KISTI)
Rutherford
Research
Apparent
Networks
CA*net4 9k App Project
Bill Rutherford (Rutherford Research/RRX – Project Coordinator)
Loki Jorgenson (Apparent Networks/SFU – Project Coordinator)
Thomas Tam (CANARIE/CA*net4 – CANARIE/UCLP Coordinator)
Bryan Caron (TRIUMF/UofAlberta – TRIUMF/UCLP Coordinator)
Randy Sobie (HEPnet/UVic – HEPnet President/Grid Integration)
Brian Corrie (WestGrid/IRMACS/SFU - IRMACS Coordinator)
Rob Ballantyne (IRMACS/SFU - IRMACS Network Coordinator)
Martin Siegert (WestGrid/SFU – WestGrid/GridStore Coordinator)
Dave Bickle (HEPnet/UVic – HEPnet Coordinator/Grid Integration)
Ken Howard (Network Services/UVic – Network Coordinator)
Peter van Epp (Network Services/SFU – Network Coordinator)
Rutherford
Apparent
Research
Networks
9k MTU > App > DDS
3 yrs
1 yrs
2 yrs
9k MTU
Project
9k App
Project
9k DDS
Project
Core R&E
Lightpath R&E
Router Troubleshooting
Large Xfer Measurements
International Links
Internet 2 Sponsored
Physics Participants
Manual Lightpath
TRIUMF to CERN Test
Few 9k Taps onto Core
Main Tap SDSC
Rutherford
Research
Lightpath R&E
Non-Routed… UCLP
Clone-Tune 9k APP… Handoff
Build Dedicated 9k Links
Provide 9k Switch Biochem
ZX GBICs/quad gigE/campus MM
UVic - SFU - UofA
CANARIE NETERA BCNET
Prelim 9k SOA Framework
3 Universities Only 9k Link on Campus
Drug Discovery System Flow
Physics & Biochem Participants
Tune 9k SOA Framework
HEPnet - WestGrid - TRIUMF
Integrate 9k SOA Framework
9k Node - HPC - 9k Node
Demo 9k SOA-DDS Based Collab
Viz HPC 9k Tests
9k SOA-DDS Software Dev Examples
NFS HPC 9k Tests
Complete Handoff … ;-)
Apparent
Networks
What are Jumbo Packets ?
IFG
PRE
MAC/
LLC
IP Header
Header
TCP Header
Header
Payload
Payload
Data
FCS
FCS
MSS(1460bytes)
Packet (1500 bytes = MTU)
Frame (1518 bytes)
OSI
Layer
7
6
5
4
3
2
1
Rutherford
Research
Description
Application
Presentation
Session
Transport
Network
Data Link
Physical
Why 9K ?
Maximum Segment Size (MSS)
Maximum Transmission Unit (MTU) = Packet
Frame
Apparent
Networks
Internet2 9K MTU Project - 2003
GigE 2-way bandwidth vs. MTU
from Kansas City to various universities
2000
Standard 1500 MTU
1800
2-way Bandwidth (Mbps)
1600
1400
1200
1000
800
600
400
200
0
0
2000
4000
6000
8000
10000
M TU size (bytes)
Rutherford
Research
Apparent
Networks
CA*net4 9K MTU Project - 2004
CA*net4 - MTU Performance
Vancouver, Ottawa, Dalhousie, Edmonton
2000
1800
Standard 1500 MTU
2-way Bandwidth (Mbps)
1600
1400
1200
1000
800
600
400
200
0
0
2000
4000
6000
8000
10000
MTU Size (bytes)
Rutherford
Research
Apparent
Networks
Approaching Application Performance
Metrics for network applications
E.g. MOS as indication of VoIP performance
Ranges of network behaviors
Models of dependence
ITU Y-1541 exemplar
Rutherford
Research
Apparent
Networks
Application Test Bed
vizserver.westgrid.ca
Routed IPv4 = 206.12.24.8
Lightpath IPv4 = 172.31.24.8
Reference IPv4 = 10.12.24.8
IRMACS
9,216 bytes
note: possibly use unrouted black hole range
10.12.24.0/22 to shadow routed IPv4 from
Westgrid 206.12.24.0/22 for lightpaths ... ?
... use 172.31 due to conflicts ...
7
SFU
64,000 bytes
Cisco 6509
gigE
172.31
VLAN
Enterasys ER16
gigE
dedicated
0
gigE port
long range ZX GBICs
from CANARIE
SM 22 km
ZX GBIC
gigE
ONS15454
Port_3 ZX
ZX GBIC
SONET
MM
172.31 VLAN
CANARIE
NETERA
VANCOUVER
CALGARY
gigE
gigE
MM
IBM p650 AIX
MM
172.31 VLAN
172.31 VLAN
gigE
gridstore.westgrid.ca
Routed IPv4 = 206.12.24.65
Lightpath IPv4 = 172.31.24.65
Reference IPv4 = 10.12.24.65
route all 172.31.0.0/16
(172.31) to one VLAN of 6 on
4 port gigE link aggregator
Rutherford
Research
CANARIE
NETERA
EDMONTON
9,216 bytes
Cisco 6509
ONS15454
gigE
gigE
CANARIE
BCNET
VICTORIA
UofA
Physics
Linux
dual port
lightpath.phys.ualberta.ca
Routed IPv4 = 129.128.241.113
Lightpath IPv4 = 172.31.241.113
Reference IPv4 = 10.128.241.113
HEPnet
172.31 VLAN
10,239 bytes
Enterasys N7
gigE
gigE
TRIUMF
campus router
SONET
172.31 VLAN
172.31 VLAN
STS-24c
phase 2 reconfigure N7 in
series between ER16 and
Cisco 6509
gigE
Dell L3
ONS15454
STS-24c
CANARIE
BCNET
campus router
9,018 bytes
SONET
STS-24c
SGI ONYX 3000
IRMACS
phase 1 direct path from
ER16 to Cisco 6509 (ZX GBIC
slot not available on 6509)
UCLP STS-24c
Cisco ONS 10,000 bytes
ONS15454
UVic
Network Services
Linux
dual port
phys02.comp.uvic.ca
Routed IPv4 = 142.104.21.13
Lightpath IPv4 = 172.31.21.13
Reference IPv4 = 10.104.21.13
WestGrid
Apparent
Networks
9k APP Project – Lightpaths
Rutherford
Research
Apparent
Networks
9k APP Project - Short Circuit Routing
Machine 0 dual port
Routed
Machine 1 dual port
1.5k
always on
NIC-0
NIC-0
IPv4-1-0
IPv4-0-0
APP-0
APP-1
Lightpath Short Circuit
on/off
NIC-1
9k
NIC-1
IPv4-1-1
IPv4-0-1
Local map unrouted black hole
IPv4 to start ... try routed switch
over later... ?
APP-Socket-0-1
VM/OS-Socket-0-1
Handler TCP-Session-0-1
TCP-Socket/Port-0-1
LLC-Socket/Port/ARP-0-1
MAC-0-1
Rutherford
Research
Apparent
Networks
Approaching User Experience
User-driven application performance
Examples…
Interaction with 3D models
Collaboration with multiple
models/data/voice/video
Massive data set manipulation
Typical applications
3D Visualization on remote server
Distributed file systems
Rutherford
Research
Apparent
Networks
9k APP – Test Results
Raw performance
2000
2000
1800
1800
1600
1600
1400
1400
1200
1200
1000
1000
800
800
600
600
400
400
200
200
0
0
0
1000
2000
3000
4000
5000
6000
7000
SFU to Devstore
Rutherford
Research
8000
9000
10000
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
UoA to Devstore
Apparent
Networks
9k APP – File System Results
Raw performance
Bonnie performance
1600
25000
1400
20000
1200
1000
15000
800
10000
600
400
5000
200
0
0
0
1000
2000
3000
4000
5000
6000
7000
SFU to Devstore
Rutherford
Research
8000
9000
10000
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
SFU to Devstore
Apparent
Networks
9k APP – File System Results
Raw performance
Bonnie performance
2000
100000
1800
90000
1600
80000
70000
1400
60000
1200
50000
1000
40000
800
30000
600
20000
400
10000
200
0
0
0
1000
2000
3000
4000
5000
6000
7000
SFU to Vizserver
Rutherford
Research
8000
9000
10000
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
SFU to Vizserver
Apparent
Networks
9k APP – File System Results
Raw performance
Bonnie performance
1400
4000
1200
3500
3000
1000
2500
Data Missing
800
2000
600
1500
400
1000
200
500
0
0
0
1000
2000
3000
4000
5000
6000
7000
UoA to Vizserver
Rutherford
Research
8000
9000
0
1000
2000
3000
4000
5000
6000
UoA to Vizserver
Apparent
Networks
NEXT PHASE
9k DDS Project – Drug Discovery System
• Combined Physics, Viz, Grid, Bio
• Preliminary integration (SOA)
• Collaborative viz
• Distributed files (databases)
• Instruments (XRD, NMR)
• GTK4 integration
• UCLPv2 integration
Rutherford
Research
Apparent
Networks
9k DDS Project – Network Overview
campus router
Supplied and set up by
TRIUMF, reimbursed
by 9k DDS Project
gigE
9k Switch
PENCE
10/8 VLAN
9k APP phase 1 direct path
from ER16 to Cisco 6509
(ZX GBIC slot not available
on 6509)
route all 10.0.0.0/8 (10/8)to
dedicated port of 8 port
gigE link aggregator
vizserver.westgrid.ca
Routed IPv4 = 206.12.24.8
Lightpath IPv4 = 10.12.24.8
note: possibly use unrouted black hole range
10.12.24.0/22 to shadow routed IPv4 from
Westgrid 206.12.24.0/22 for lightpaths ... ?
7
note: available bandwidth on "lightpath" e2e is dependent
on configuration of ONS15454 and activity of ports
IRMACS
9,216 bytes
SFU
64,000 bytes
Cisco 6509
Enterasys ER16
10/8
VLAN
gigE
ZX GBIC
gigE
MM
dedicated
0
gigE port
long range ZX GBICs
from CANARIE
ZX GBIC
SONET
SGI ONYX 3000
IRMACS
9k DDS phase 1 tap
TRIUMF gigE port
from ONS for setup
CANARIE
NETERA
VANCOUVER
CALGARY
gigE
9,216 bytes
Cisco 6509
ONS15454
gigE
MM
gigE
MM
gigE
gigE
gridstore.westgrid.ca
route all 10.0.0.0/8 (10/8)to
one VLAN of 6 on 4 port
gigE link aggregator
Routed IPv4 = 206.12.24.65
Lightpath IPv4 = 10.12.24.65
Linux
dual port
UVic
phys02.comp.uvic.ca
Network Services Routed IPv4 = 142.104.21.13
CANARIE
BCNET
VICTORIA
10/8 VLAN
10/8 VLAN
Physics
HEPnet
10/8 VLAN
10,239 bytes
Enterasys N7
IBM p650 AIX
lightpath.phys.ualberta.ca
Routed IPv4 = 129.128.241.113
Lightpath IPv4 = 10.128.241.113
UofA
campus router
SONET
10/8 VLAN
TRIUMF
Linux
dual port
gigE
gigE
CANARIE
NETERA
EDMONTON
STS-24c
9k APP phase 2 reconfigure
N7 in series between ER16
and Cisco 6509
gigE
10/8 VLAN
STS-24c
CANARIE
BCNET
campus router
9,018 bytes
Dell L3
ONS15454
SONET
STS-24c
10/8 VLAN
lightpath.pence.ca
Routed IPv4 = 129.128.139.2XX
Lightpath IPv4 = 10.128.139.2XX
UofA
Biochemistry
gigE
UCLP STS-24c
Cisco ONS 10,000 bytes
ONS15454
ONS15454
Port_3 ZX
SM 22 km
gigE
Linux
dual port
gigE
9k DDS phase 2
reconfigure to
dedicated gigE port
from ONS
Supplied by Pence
9k DDS phase 1 tap HEPnet
gigE port from ONS for setup
Lightpath IPv4 = 10.104.21.13
9k DDS phase 2
reconfigure to
dedicated gigE port
from ONS
WestGrid
9k DDS phase 1 tap WestGridIRMACS gigE port from ONS
for setup
gigE
9k DDS phase 2
reconfigure to
dedicated gigE port
from ONS
campus router
Supplied and set up by
WestGrid, reimbursed
by 9k DDS Project
campus router
TVBR
9k Switch
Supplied by TVBR
9k Switch
MBB
10/8 VLAN
10/8 VLAN
Linux
dual port
gigE
Supplied by MBB
lightpath.mbb.sfu.ca
Routed IPv4 = 142.58.213.XXX
Lightpath IPv4 = 10.58.213.XXX
Rutherford
Research
gigE
Supplied and set up by
HEPnet, reimbursed
by 9k DDS Project
SFU
Molecular Biology and Biochemistry
UVic
Biochemistry & Microbiology
Linux
dual port
lightpath.bioc.uvic.ca
Routed IPv4 = 142.104.33.XXX
Lightpath IPv4 = 10.104.33.XXX
Apparent
Networks
End of Presentation
Rutherford
Research
Apparent
Networks