Transcript my presentatio - lhcone

Dynamically
Provisioned Networks
as a Substrate for
Science
David Foster
CERN
Objectives
• To explain in a high-level way why dynamic circuits
are needed to serve demanding scientific users.
• To put the activities into an overall context of global
research networking and show the future directions.
David Foster, CERN
Environment
• Science projects are global enterprises
o Megascience: LHC, ITER, LoFAR, JIVE, SKA …….
o ESFRI projects funded under FP7:
• http://ec.europa.eu/research/infrastructures/index_en.cfm?pg=ri_pro
jects_fp7
o Increasing coordination on an increasingly global scale.
• Computing for science is increasingly distributed
o Clouds and Grids
• People are mobile but lifestyle choice is important
o Work anywhere, anytime.
o “Think Globally but act Locally”
• Open access to information is empowering
o “Bring Science to the Scientists” - Bring the best minds to the problem.
David Foster, CERN
Situation
• As perceived by the user community, networks are
a victim of their own success
o Expected to be transparent. “Networking is not a problem”
o Assumed to be infinite and free (or nearly so).
• Data volumes are increasing
o LHC creates 6-7 PB raw data per year , all 4 experiments together.
o CERN generates in total 15 PB of data per year, all 4 experiments together.
o The raw data, 6-7PB/year, is distributed and there are more than
120PB/year of data products that are created and stored world-wide.
o Other science project collaborations will generate equivalent or more
data.
David Foster, CERN
Characterization of the
User Marketplace
David Foster, CERN
Cees de Laat
http://ext.delaat.net/talks/cdl-2005-02-13.pdf
Issues With Demanding
Users
• There are more and more of them.
• The swamping of IP infrastructures with traffic from
“well connected sites”
o Occurs when the capability of a site are approaching that of the routed
IP network.
o Looks like a “denial of service” to the other users.
• Solution 1: Build a bigger routed IP network.
o A big investment to solve a problem for relatively few users.
o All domains in any end-end path must do the same.
o Only temporary, new users will come with bigger requirements.
• Solution 2: Give the sites “what they need when
they need it”.
o May be considered as “Just in time provisioning”
o Has led to the circuit approach.
David Foster, CERN
How Circuits are Used
• For efficiently using resources:
o Long term or static circuits if the number of sites is small
(~10)
o Dynamic circuit provisioning for a community that is
manageable and has continual needs (~100 sites)
o Dynamic circuit provisioning for a large community that
has occasional needs (periodic data transfer)
• For traffic management:
o Separates flows from the general IP infrastructure.
David Foster, CERN
LHC: Case Study
• LHC tier-1 sites are connected at 10G in a semi-mesh.
o LHCOPN
• Tier-2 sites (originally) needed 1Gbps to realistically be
“part of” the grid community.
o A dedicated circuit to India enabled and effectively empowered the TIFR.
• Now, tier-2 sites are increasingly connected at 10G to be
able to dynamically access all data products wherever
they are (remember the 120PB/year?)
o But connected to where? The association between Tier-1 and Tier-2 has
“disappeared”.
o All sites must be able to access all sites, so IP is the best fit!
• Probably so at 1Gbps/site, but not at 10Gbps/site.
• A new approach was needed
o LHCONE: http://lhcone.net
David Foster, CERN
What is LHCONE?
• A sociology
o Has helped to raise awareness of end site needs.
o T2’s in Europe are requesting increasing capacity from the NREN’s
• A process
o Transatlantic bandwidth review of all R&E circuits
o Discussions on really how to make a cross domain network.
o Process is perhaps more important than outcome in delivering collaboration
and focus
• as long as the outcome works!
• An architecture
o Use of open exchanges to empower networks and users
o Use of software for managing network capacity through dynamic provisioning.
• OpenFlow is the flavour of the month
• A model
o The internet-2 OS3E is an open exchange architecture to support all sciences.
Experience with LHCONE will be important.
David Foster, CERN
Open Exchanges
•
•
A growing consensus on the way forward
o
o
I2 members meeting discussion with Bill St Arnaud.
A paper in preparation on a “definition”
o
They promote customised bilateral relationships by the exchange point owner not interfering
• “lightweight” rules for connecting, link policies controlled by the link owners.
They have no specific commercial allegiance
• So-called “carrier-neutral”
Users of the exchange point (can be NREN’s or end users) like them because they remain in
control and directly manage the relationships.
• no third-party involvement
They provide the possibility to create diverse solutions by working with different partners.
• risk management
They do not impose technical decisions, so everyone can go at their own speed.
• Avoids “lowest common denominator” solutions
They allow for organic growth and new entrants are welcome both as exchange operators and
connected parties
• Avoids single point of failure in the system as a whole
Optical exchanges permit provisioning of circuits of different transport protocols to exchange
traffic.
• http://www.broadnets.org/2004/workshop-papers/Gridnets/DijkstraF.pdf
They allow for activities to follow means and ambition.
• They can be “pay as you go” and not “subscription based”
Why is there so much interest?
o
o
o
o
o
o
o
David Foster, CERN
To Be Resolved
• Costs incurred to connect to an open exchange.
o
o
Depends on the exchange operator and the “last mile” provider.
Will be born by the end-site connecting.
o
o
Currently born by the exchange owners, but is this scalable?
Might be also the responsibility of the science community but they are classically not
structured to fund central network resources.
• Needs to be more awareness that networks are not free inside the science
communities.
• Costs incurred to interconnect open exchanges
• Management of multi-domain circuit based infrastructures
o
o
o
o
Is a hot topic and has been for some time.
• Buzzword heaven: Oscars, ION, Dragon, DRAC, Federica, OpenFlow, DICE, NSI
…
Many software solutions are used and under development.
• But some are moving much faster than others. We need some consolidation.
Some collaboration activities underway to develop domain interworking.
• But we need a more open and inclusive process.
Operations and management processes are still to be agreed.
• No real process addressing this at present.
David Foster, CERN
Summary
• Science needs are increasing and diversifying rapidly
o Big growth in large international projects in all areas needing global high
bandwidth connectivity.
o The pressure is to always seek Open, Neutral and Diverse solutions.
• The best service at the best price.
• Circuit based approaches are inevitable
o They address the needs of the network providers to serve the high-end users
with resource efficiency and manageability.
o The downward pressure on bandwidth costs from commercial operators make
them increasingly cost effective.
• Open Exchanges are inevitable
o Because of the compelling combination of sociological, business and
technical rationale.
o We are not a strict hierarchy of users, nren’s, operators at world or european
level and perhaps becoming less so as time goes on.
David Foster, CERN