Transcript keynote2
CA*net 4 A Network for Grids using Grid Technology Bill St. Arnaud CANARIE Inc – www.canarie.ca [email protected] CA*net 4 Drivers-1 > Set up lightpaths to no cost peering exchanges – Most lambda sales in Canada and USA are for “Remote peering” to no cost peering points – Allows for considerable savings in Internet transit costs – Each lightpath is directly connected to a high volume peer and bypasses peering router – Good example is “STAR LIGHT” where high volume peers have direct connect and small volume peers use a router – CA*net 4 “customer controlled patch panel” allows peers to change peering relationship remotely without contacting technical staff at peering exchange • Very similar in concept to WorldCom “Peermaker” at MAE-E and MAEW CA*net 4 Drivers-2 > Eliminate expensive high end routers and replace them with lower cost optical switches – But circuits are NOT intended to replace packet networks – Use rich mesh of circuits between edge routers to eliminate high cost of 10GbE core routers – 10Gbe routers ~ $500K with interfaces at ~$200k each – 10Gbe switches ~$25K with interfaces at ~$20k each – Trade off between cost of multiple lightpaths versus cost of high end core routers – 10Gbe wavelengths ~$1000/km for 5 years (lifetime of router) – Assume 1 GbE lightpaths per edge institution then • One 5000km Gbe lightpath (or 8 x 600km GbE) lightpaths per institution is cheaper than routers • But hard to create a full mesh – so let institutions or end user create and control partial mesh – Disadvantage is no sharing of bandwidth with stat muxing CA*net 4 Drivers-3 > Allows customer to create “customer owned and managed” networks with resource heterogeneity – Integration of wavelengths and dark fiber from different carriers – Create customer controlled VPNs for downstream users and overlay networks across multiple suppliers – Customers can manage their own restoral and protection schemes – Allows for inter-domain end to end setup of VPNs – End users do not need to to signal carrier for VPN management • Create VPNs • Cross connect VPNs from independent users • Partition or spawn VPNs • Establish VPNs across multiple management domains CA*net 4 Drivers-4 > Lambda Grids - “Underlay” networks to support Grids and overlay projects like PlanetLab and Oceanstore – A lot of exciting research into overlay networks – At some point in time when traffic volume is sufficient in overlay network to setup its own direct path > Soon high end grid applications will have sufficient traffic volume to require their own underlay networks ”Complementing” routed networks – Not a replacement for routed networks – only increasing the direct peering mesh of the routed network – But peering may be more dynamic (and not globally advertised) than traditional IP BGP peering > Discipline or applications specific networks – VBLI grids like European EVN – High energy physics grid – Ultralight – NEES grid, Bio-informatics Grid, etc Example – EVN traffic flows over GEANT SE NORDUnet JANET UK JIVE PL SURFnet CZ PSNC BE NL DFN DE2 DE1 FR 2.5G 10G CH AT IT GARR 256M 512M 1G ? Provided courtesy of Dai Davies Issues > How do you charge for bandwidth and usage when single application traffic dwarfs all other IP traffic? > Who pays for the traffic volume when it sinks into POP? > Possible solutions: – GMPLS (with QoS) • Requires expensive routers and complex coordinated central management to setup and tear down tunnels • Does not address issue of traffic charging • Interdomain still unproven – Optical overlay/underlay –ASON – same problems as GMPLS – Application specific optical BGP networks CA*net 4 Drivers-5 > Spatial QoS – TCP throughput over long fat pipes very susceptible to packet loss, MTU, TCP kernel, Buffer memory, AQM optimized for commodity Internet, Auto negotiating Ethernet, etc – May also require consistent and similar TCP throughput for multiple sites to maintain coherency for grids and SANs – Some exciting new TCP protocols like FAST, XCP, etc • • Mice and Elephant problem Without careful design may look like a DOS attack on a router network – Many commercial SAN/Grid products will only work with QoS network – Some users want to have super jumbo MTU (64K) or protocols other than IP Spatial QoS x.x.x.1 Normal BGP path y.y.y.1 Optical “Peermaker” Only y.y.y.1 advertised to x.x.x.1 via OBGP path Application or end user controls peering of BGP optical paths for transfer of elephants!!! OBGP path Only x.x.x.1 advertised to y.y.y.1 via OBGP path OBGP applied to EVN SE NORDUnet JANET UK JIVE PL SURFnet CZ PSNC BE NL DFN DE2 DE1 FR CH AT IT GARR CA*net 4 Drivers 6 > Extend the Internet end to end principle to circuit based networks – The success of the Internet is largely attributable to the e2e principle – No state maintained in the network – Allowed development of exciting new applications or services > Can the same principles be applied to circuit based networks? – Will it engender the same creativity in new applications and services? > MPLS and ASON are classic network state based solutions for VPNs – CA*net 4 architecture is an alternate approach – All VPNs are BGP direct static routes using lightpaths CA*net 4 is NOT a network > It is an aggregation of point to point 10 Gbps wavelengths from a number of carriers > CA*net 4 is made up of may parallel networks > The wavelengths and switches are partitioned into smaller lightpaths user control of the switch partition which are used for a variety of applications particularly grids International Grid Testbed – 10 Gbe server to server to CERN WESTgrid – 1 Gbe lightpaths for distributed backplane CA*net 4 IP network – traditional IP hierarchical routed network Numerous lightpaths to support direct peering between regional networks and universities – Lightpaths to support Terabyte file transfer from CERN for high energy physics that bypasses all routers – Lightpaths to support TransLight projects between North America, Europe and Asia – Many, many more coming – Virtual Astronomy, HDTV video walls, etc – – – – Service Provisioning Layer Grid Application Web Server User Access Layer Grid Service Interface LPO Grid Service GT3 Hosting Environment LPO Factory Service LPO Delegate Service RMI RMI LPO Service EJB Home LPO Service Implementation LPO Service EJB Remote J2EE Application Server JDBC RMI DB Resource Management Layer Create service Access service OGSI-conformant services OGSI-conformant services LPO advertisement LPO query LPO termination LPO access LPO reconfiguration LPO spawning LPO concatenation End-to-end LPO establishment Resource Management Layer Service Provisioning Layer RMI Request Controller LPO Controller Programmable LPO Controller Controller Resource Agent LPO Controller LPO Controller LPO Controller Switch Interface TL1 CA*net 4 RMI LPO Space More Information > http://www.canarie.ca/canet4/library/canet4design.ht ml > http://www.canarie.ca/canet4/obgp/index.html > http://www.canarie.ca/canet4/library/customer.html > Thanks to the design teams at Carleton U, Ottawa U, CRC, UQAM, UoWaterloo, Montague, etc