Transcript slides
content distribution and “protocol”: from hierarchical trees to distributed graphs Jason Gaedtke [email protected] November 7, 2007 abstract • codec efficiency advances, recording device and storage commoditization, and broadband access network maturity have enabled an explosion in the consumption of professional and user-generated IP video • traditional media distributors and emerging Internet providers are challenged to drive cost from their IP video business models • the P2P architectural paradigm and the philosophical and cultural concept of “Protocol” offer a compelling alternative to traditional, hierarchical CDNs definition (Galloway, MIT Press 2004) • “protocol” – – – – – – a system of distributed management/control strategies facilitates P2P relationships between autonomous entities anti-hierarchical and anti-authority engenders localized decision-making, versus centralized accommodates massive contingency/change represents the outcome (not the antecedent) of distributed behavior • analysis occurs at the intersection of philosophy, culture and technology • control moves from authorities to protocol standards philosophical and political considerations • mesh vs. hierarchy – favors horizontal, distributed organization and control – e.g. TCP/IP vs. DNS (hindered by points of control/failure) – end-to-end principal: state/intelligence at the edges • historical movement – centralized (server) -> decentralized (cluster) -> distributed (net) – low barrier-to-entry empowers participation, breeds innovation – a similar decentralized/distributed control model underlies proven and successful open source collaboration • this concept, Protocol, is native to the logical design/architecture of the Internet (TCP/IP) – infrastructure and associated corporate capital investment notwithstanding (i.e., root of present Net Neutrality debate) business and economic considerations • primary appeal: storage, processing, and bandwidth cost avoidance; >90% savings – application infrastructure is self-organizing/healing; minimal admin/ops overhead • must consider/provide user value-proposition (beyond “free content”) motivating participation and resource contribution • Internet public policy and intellectual property law/enforcement still evolving • growing need for security and data integrity assurances technical considerations • P2P substrates: • form a decentralized, self-organizing and fault-tolerant overlay network • provide efficient request routing, deterministic object location, and load-balancing in an application-independent manner • facilitate application-specific object replication, caching, and fault recovery • enable robust and efficient data and service availability, reliability and geographical/route diversity and redundancy • offer compelling scaling features and performance typically O(log(N)) • participatory design, development and control model engenders innovation applications • substantial academic and corporate research over the past five+ years • horizontal, mesh networks and P2P applications evolving beyond file-sharing – music/movies/tv “sharing”: Napster, Gnutella, KaZaA, eDonkey, FreeNet, BitTorrent – voice: Skype, IETF’s P2P SIP – video: Azureus, Joost – storage: OceanStore, Ivy – commercial content distribution: Move, Grid, Pando, Red Swoosh (Akamai), Kontiki (VeriSign) – gaming and virtual worlds: Quazal, FT’s Solipsis – virtual economies: Scrivener (Rice), Tribler (Harvard) appendix research and references • “Protocol: How Control Exists after Decentralization” – Alex Galloway (NYU) – MIT Press, 2004 – Philosophical foundations in Marx, Foucault, Deleuze, Jameson, Hardt • academic research – – – – – – • Berkeley (OceanStore persistence) Columbia (P2P SIP audio/video communications) MIT (Chord DHT, Ivy file-system) Purdue (Pastry, Dynamic P2P Source Routing) Rice (FreePastry, Squirrel web cache) Washington (Pastry, BitTyrant) corporate R&D – France Telecom (Solipsis virtual world, Maay search engine, – Microsoft (Herald pub/sub, SimPastry, PAST archive, SplitStream CDN) • Distributed Computing Industry Association (DCIA) P4P Working Group – Explicit Communications for Cooperative Control Between P2P and Network Providers • • IETF P2P SIP looking to add CableLabs to this list enabling technologies • reliable and efficient storage, search and discovery algorithms – DHTs: Chord, CAN, Pastry, Tapestry • • • • • • identity reputation security virtual economy quality of service multicast