Transcript slides

content distribution and “protocol”:
from hierarchical trees to
distributed graphs
Jason Gaedtke
[email protected]
November 7, 2007
abstract
• codec efficiency advances, recording device and
storage commoditization, and broadband access
network maturity have enabled an explosion in the
consumption of professional and user-generated IP
video
• traditional media distributors and emerging Internet
providers are challenged to drive cost from their IP
video business models
• the P2P architectural paradigm and the philosophical
and cultural concept of “Protocol” offer a compelling
alternative to traditional, hierarchical CDNs
definition (Galloway, MIT Press 2004)
• “protocol”
–
–
–
–
–
–
a system of distributed management/control strategies
facilitates P2P relationships between autonomous entities
anti-hierarchical and anti-authority
engenders localized decision-making, versus centralized
accommodates massive contingency/change
represents the outcome (not the antecedent) of
distributed behavior
• analysis occurs at the intersection of philosophy,
culture and technology
• control moves from authorities to protocol standards
philosophical and political
considerations
• mesh vs. hierarchy
– favors horizontal, distributed organization and control
– e.g. TCP/IP vs. DNS (hindered by points of control/failure)
– end-to-end principal: state/intelligence at the edges
• historical movement
– centralized (server) -> decentralized (cluster) -> distributed (net)
– low barrier-to-entry empowers participation, breeds innovation
– a similar decentralized/distributed control model underlies
proven and successful open source collaboration
• this concept, Protocol, is native to the logical
design/architecture of the Internet (TCP/IP)
– infrastructure and associated corporate capital investment
notwithstanding (i.e., root of present Net Neutrality debate)
business and economic considerations
• primary appeal: storage, processing, and bandwidth
cost avoidance; >90% savings
– application infrastructure is self-organizing/healing;
minimal admin/ops overhead
• must consider/provide user value-proposition (beyond
“free content”) motivating participation and resource
contribution
• Internet public policy and intellectual property
law/enforcement still evolving
• growing need for security and data integrity assurances
technical considerations
• P2P substrates:
• form a decentralized, self-organizing and fault-tolerant overlay
network
• provide efficient request routing, deterministic object location,
and load-balancing in an application-independent manner
• facilitate application-specific object replication, caching, and
fault recovery
• enable robust and efficient data and service availability,
reliability and geographical/route diversity and redundancy
• offer compelling scaling features and performance typically
O(log(N))
• participatory design, development and control model engenders
innovation
applications
• substantial academic and corporate research over the past
five+ years
• horizontal, mesh networks and P2P applications evolving
beyond file-sharing
– music/movies/tv “sharing”: Napster, Gnutella, KaZaA, eDonkey,
FreeNet, BitTorrent
– voice: Skype, IETF’s P2P SIP
– video: Azureus, Joost
– storage: OceanStore, Ivy
– commercial content distribution: Move, Grid, Pando, Red
Swoosh (Akamai), Kontiki (VeriSign)
– gaming and virtual worlds: Quazal, FT’s Solipsis
– virtual economies: Scrivener (Rice), Tribler (Harvard)
appendix
research and references
•
“Protocol: How Control Exists after Decentralization”
– Alex Galloway (NYU)
– MIT Press, 2004
– Philosophical foundations in Marx, Foucault, Deleuze, Jameson, Hardt
•
academic research
–
–
–
–
–
–
•
Berkeley (OceanStore persistence)
Columbia (P2P SIP audio/video communications)
MIT (Chord DHT, Ivy file-system)
Purdue (Pastry, Dynamic P2P Source Routing)
Rice (FreePastry, Squirrel web cache)
Washington (Pastry, BitTyrant)
corporate R&D
– France Telecom (Solipsis virtual world, Maay search engine,
– Microsoft (Herald pub/sub, SimPastry, PAST archive, SplitStream CDN)
•
Distributed Computing Industry Association (DCIA) P4P Working Group
– Explicit Communications for Cooperative Control Between P2P and Network Providers
•
•
IETF P2P SIP
looking to add CableLabs to this list
enabling technologies
• reliable and efficient storage, search and
discovery algorithms
– DHTs: Chord, CAN, Pastry, Tapestry
•
•
•
•
•
•
identity
reputation
security
virtual economy
quality of service
multicast