Transcript ********** ***** *** Internet

Πολυμεσικό Υλικό στο Internet:
Συγχρονισμός, Επεξεργασία και
Διακίνηση
Multimedia content delivery in the Internet: CDNs, Real
Time Streaming, Multimedia platforms & video service
providers
26/11/2015
Β. Μάγκλαρης <[email protected]>
Μ. Γραμματικού <[email protected]>
Δ. Καλογεράς <[email protected]>
www.netmode.ntua.gr
Outline
• Content Delivery Networks (CDNs)
• Video on Demand servers/clouds
• Real Time Streaming
Content Delivery Networks (CDNs)
http://en.wikipedia.org/wiki/Content_delivery_network
• The first generation CDNs were not encountered before the late 90′s
• CDN is a large distributed system of servers deployed in multiple data
centers across the Internet
• It serves content to end-users with high availability and high
performance
• It serves a large fraction of the Internet content :
– including web objects (text, graphics and scripts), downloadable objects
(media files, software, documents), applications (e-commerce, portals),
live steaming media, on-demand streaming media, and social networks
Single server
distribution
and CDN
distribution
CDN Providers uses different Business
Model
• Google maintains it’s own delivery network consisting
of large number of GGC (Google Global Cache) nodes
placed on ISPs network and help in serving Google’s
static content
• Akamai (whose core business is into Cache delivery)
put their servers on large number of edge networks
but they stay as disconnected small islands
• New comers in the industry like Limelight, Cloudflare
put node in major datacenter and direct connection to
major networks via peering from IXPs
Notable Content Delivery Service Providers
http://en.wikipedia.org/wiki/Content_delivery_network
Free CDNs[edit]
•BootstrapCDN
•CloudFlare
•Coral Content Distribution Network
•Incapsula (free version with Incapsula advertisement)
Traditional commercial CDNs[edit]
•Akamai Technologies
•Amazon CloudFront
•Aryaka
•Windows Azure CDN
•CacheFly
•CDNetworks
•ChinaCache
•Cotendo (acquired by Akamai)
•Distil Networks
•EdgeCast Networks
•Highwinds Network Group
•HP Cloud Services
•Incapsula
•Internap
•LeaseWeb
•Level 3 Communications
•Limelight Networks
•MaxCDN
•MegaFon
•MetaCDN
•Mirror Image Internet
•NACEVI
•OnApp
•OVH
•Rackspace Cloud Files
•Speedera Networks (acquired by Akamai)
•StreamZilla CDN Europe
Telco CDNs[edit]
•AT&T Inc.
•Bharti Airtel
•Bell Canada
•BT Group
•Deutsche Telekom
•Hibernia Networks
•KT (formerly Korea Telecom)
•KPN
•Level 3 Communications
•MegaFon
•NTT
•Pacnet
•PCCW
•Qualitynet
•SingTel
•SK Broadband
•Tata Communications
•TeliaSonera
•Telecom Argentina
•Telecom Italia
•Telecom New Zealand
•Telefonica
•Telenor
•Telstra
•Telus
•Turk Telekom
•Verizon
Commercial CDNs using P2P for delivery[edit]
•BitTorrent, Inc.
•Internap
•Pando Networks
•Rawflow
CDN 2014 – 2017: Operations and Analytics
http://www.reportsnreports.com/reports/287274-cdn-2014-2017-operations-andanalytics.html
• CDN (Content Delivery Network) revenue is projected at $3.36
billion in 2014, up 19%,directed by user appetite for self-selected
content
• Total 2013 commercial value of media and entertainment video
(views and advertising), movie/TV files, music listening and
downloads (including self-hosting entities such as Google and
Amazon) stood at $3.35 billion, of which $1.05 billion (31.3%) was
delivered through CDN contracts
• Video viewing and advertising (combining self-hosted networks),
TV/ movies and music accounted for 2.4+ billion gigabytes of data
transfer, worth $1.6 billion in commercial market value (bandwidth
and co-lo fees only), when priced at prevailing rates and against a
volume (gigabytes delivered) model
•
CDNs include Akamai Technologies, Limelight Networks, Cd Networks, China Cache,
Mirror Image, Level 3, High winds, Cloud flare, Tata Communications and MaxCDN,
among others
Published By: AccuStream Research
Content Delivery Networks
http://www.globaldots.com/the-history-of-content-delivery-networks-cdn
• In the past, the development of CDNs sought to deal
with extreme bandwidth pressures, first as video
streaming was growing in demand as well as the
number of content providers
• Now, CDNs are a continual trend, with the emergence
of utility - cloud computing, charging end-users on
demand and involving all layers of cloud computing:
– SaaS (Software as a Service), e.g. Google Docs, Amazon
CloudFront
– PaaS (Platform as a Service) e.g. Google App Engine
– IaaS (Infrastructure as a Service), e.g. GRNET Okeanos,
Amazon EC2
Layers of Cloud Computing
http://www.globaldots.com/the-history-of-content-delivery-networks-cdn
Content Νetworking Τechniques
• The Internet was designed according to the end-to-end neutrality
principle. Core network : simple and moves the intelligence to the
network end-points (hosts, clients)
• CDNs distribute a variety of applications employing techniques to
optimize content delivery. The resulting tightly integrated overlay
uses web caching, server-load balancing, request routing, and
content services
– Web caches store popular content on servers that have the greatest
demand for the content requested
– Server-load balancing uses one or more techniques including servicebased (global load balancing) or hardware-based
• A variety of algorithms are used to route appropriately the request.
These include:
–
–
–
–
–
Global Server Load Balancing
DNS-based request routing
Dynamic metafile generation
HTML rewriting and
Anycasting
Algorithms used in Request Routing in CDNs
http://www.codeproject.com/Articles/391966/Overview-of-CDN-Content-DeliveryNetwork
• Global Server Load Balancing (GLSB) enables the content to be
obtained from a server pool in a sequential manner using roundrobin method and redirect the request in case of inactive server
sessions
• DNS-based request routing: when a request is made (URL), the
local DNS server provides the IP address of the nearest matching
CDN node. If the Local DNS is not able to resolve the URL, it
forwards the request to the Root DNS server, which then provides
the nearest possible CDN server IP
• Dynamic metafile generation includes creation of a metafile, which
has an ordered hierarchy of CDN domains connected to a Main
server and helps in the load balancing on each of CDN nodes
connected to it
Protocols used in Request Routing in CDNs
http://www.codeproject.com/Articles/391966/Overview-of-CDN-Content-DeliveryNetwork
• ICAP (Internet Content Adaptation Protocol) is a high level protocol
that helps in generating http requests and delivers contents from
the CDN servers
• OPES (Open Pluggable Edge Services) uses a Processor in order to
share contents to the end users. This processor duplicates the
content at each CDN node and traces the route followed by each
request made by the user and notifies the user once the content is
found
• ESI (Edge Side Includes) avoids back end processing delays hence
providing dynamic contents with ease. It breaks web content into
fragments and delivers dynamic contents to end users
Content Delivery Networks (CDNs)
Content Delivery Networks State of the Art, Insights, and Imperatives_2008,
Mukaddim Pathan, Rajkumar Buyya and Athena Vakali
• Most CDNs are operated as an Application Service
Provider (ASP) on the Internet (also known as ondemand software or SaaS) targeting Content
Providers, web administrators etc., wishing to
optimize delivery to end-users
• CDN Service and Functionalities:
–
–
–
–
–
–
Storage and Management of content
Distribution of content among edge servers
Cache management
Delivery of encoded media
Backup and disaster recovery solutions
Monitoring and Performance
P2P and Private CDNs
• Peer-to-peer CDNs
– In P2P content-delivery networks, clients provide
resources as well as use them, so the content
serving capacity of P2P networks can
actually increase as more users begin to access
the content
• Private CDNs
– The owners create their own CDN
– A private CDN consists of Points of Present (PoPs)
that are only serving content for their owner
Top International CDNs make deals directly with ISPs
and/or Internet Exchanges
http://blog.streamingmedia.com/2014/07/apples-cdn-now-live.html
https://peering.google.com/about/
• Apple’s CDN Now Live: Has Paid Deals With
ISPs, Massive Capacity In Place
• Google Peering and Content Delivery
– Data centers
– Backbone
– Edge Points of Presence (POPs)
– Google's edge caching infrastructure
• Netflix Announces New Content Delivery
Network, Offering Free Caches To ISPs
Free CDN Services for WordPress Blog
• CloudFare : the data centers are located all across the
world to serve you the content at blazing fast speed
• Photon CDN: is a content delivery network that loads
images on your blog through the powerful
WordPress.com servers
• Swarm CDN: it offers a free plan with 250 GB of free
transfer
• jsDelivr: CDN service for hosting JavaScript files
• Coral CDN: is a P2P (peer-to-peer) based content
delivery network and a project from MIT. Billions of
computers are connected across the world, making it
absolutely free
A Pioneer Content Delivery Network (CDN): Akamai
http://www.akamai.com/html/about/company_history.html
Erik Nygren, Ramesh K. Sitaraman, Jennifer Sun, The Akamai Network: A Platform
for High-Performance Internet Applications
• Akamai's beginnings lie in a challenge posed by World Wide
Web inventor Tim Berners-Lee at MIT in early 1995
• The company launched commercial service in April 1999
• The platform is made up of a distributed network of servers
and intelligent software, delivering over two trillion
interactions daily
• The Akamai Intelligent Platform is constantly
monitoring Web conditions to:
– Identify, absorb, and block security threats
– Provide instant device-level detection and optimization
– Make decisions based on comprehensive knowledge of
network conditions
– Present unprecedented business and technical insights
Akamai
Erik Nygren, Ramesh K. Sitaraman, Jennifer Sun, “The Akamai Network: A
Platform for High-Performance Internet Applications”, ACM SIGOPS Operating
Systems Review, Vol. 44, No 3, July 2010
The figure above shows how Akamai maintained high
connectivity for customers during the 2008 cable cuts that
caused widespread Internet outages in the Middle East, Asia,
and Africa.
Video-on-Demand
http://en.wikipedia.org/wiki/Video_on_demand
http://en.wikipedia.org/wiki/IPTV
• Video-on-Demand (VoD) are systems which allow
users to select and watch/listen to video or audio
content when they choose to, rather than having to
watch at a specific broadcast time. IPTV technology is
often used to bring video on demand to televisions and
personal computers
• IPTV or IP/TV was developed as a multimedia service
for television/video/audio/text/graphics/data delivered
over IP based networks managed to provide the
required level of quality of service and experience,
security, interactivity and reliability (initially developed
by Precept Software in 1995, acquired by Cisco in
1998)
Video-on-Demand
http://en.wikipedia.org/wiki/Video_on_demand
• Pay-Per-View (PPV) is a type of pay television
service by which a subscriber of a service
provider (cable TV, Satellite and Broadband
Internet) can purchase events/shows/programs
• In 2006, HBO generated 3.7 million PPV buys
with $177 million in gross sales
• In 2007, HBO sold 4.8 million PPV buys with
$255 million in sales
Video-on-Demand Issues
Common problems/issues a VoD needs to address:
• Load Distribution on Server
• Media Content Management
• Adapt to Dynamic Network Bandwidth
• Decide on Buffer/Cache
• Rate Control
• Scalability & Cost Effectiveness
• To provide Reliability & Availability
Real Time Streaming Services
http://en.wikipedia.org/wiki/Streaming_media
http://en.wikipedia.org/wiki/HTTP_Live_Streaming
http://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP
• Live Streaming, which refers to content delivered live over the
Internet, requires a form of source media (e.g. a video camera), an
encoder to digitize the content, a media publisher, and a content
delivery network to distribute and deliver the content.
• Progressive Download: the video is downloaded but viewing can
begin once a small portion of the file is buffered
• Adaptive Bitrate Streaming is based on HTTP progressive
download with very small files, so that they can be compared to the
streaming of packets (like RTSP, RTP). While in the past video
streaming technologies utilized streaming protocols such as RTP
(UDP) with RTSP (TCP), today's adaptive streaming technologies are
almost exclusively based on HTTP (multi-threaded TCP)
Adaptive Bitrate Streaming
http://en.wikipedia.org/wiki/Adaptive_bitrate_streaming
• Adaptive Bitrate Streaming provides consumers with the bestpossible experience, since the media server automatically adapts to
any changes in each user's network and playback conditions. With
adaptive streaming the video source bit-rate is adapted on the fly so
that the user can experience the maximum quality which is allowed
by the time-varying available bandwidth
• Adaptive Bitrate Streaming implementations: Apple HTTP Live
Streaming, Microsoft's Smooth Streaming, Adobe's HDS, MPEGDASH (deployed by video content providers e.g. YouTube, Netflix and
video streaming services e.g. Wowza )
References
• http://en.wikipedia.org/wiki/Content_delivery_network
• R. Buyya et al. (eds.), Content Delivery Networks, Springer-Verlag Berlin
Heidelberg 2008, chapter 1
• http://www.akamai.com/html/about/company_history.html
• https://easyipv6.wordpress.com/2013/06/28/building-a-fast-contentnetwork-with-anycast/
• http://blog.streamingmedia.com/2014/07/apples-cdn-now-live.html
• Arlitt, M. and Jin, T. A workload characterization study of 1998 world cup
Web site. IEEE Network, pp. 30–37, 2000.
• http://en.wikipedia.org/wiki/Streaming_media
• http://www.globaldots.com/the-history-of-content-delivery-networkscdn
• http://en.wikipedia.org/wiki/HTTP_Live_Streaming
• http://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP
• http://en.wikipedia.org/wiki/Comparison_of_video_hosting_services
• http://en.wikipedia.org/wiki/Comparison_of_streaming_media_systems