slides - Euro Ecom Cmu
Download
Report
Transcript slides - Euro Ecom Cmu
eCommerce Technology
20-751
Web Content Delivery
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Outline
• The importance of content
• How is it delivered?
– Content delivery networks
• Peer-to-Peer Technologies
– Napster, Gnutella, Freenet, Morpheus
• Streaming
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
The Internet
Looks Simple on the Outside
Content Providers
End Users
Internet
SOURCE: CRAIG SNYDER
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Inside – The 4 Bottlenecks
1. First Mile
2. Peering
3. Backbone
NAP
4. Last Mile
UUNet
NTT
NAP
Deutsche
Telekom
SOURCE: CRAIG SNYDER
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
HTML Title Page for www.xyz.com
with Embedded Objects
<html>
<head>
<title>Welcome to xyz.com!</title>
</head>
<body>
<img src=“http://www.xyz.com/logos/logo.gif”>
<img src=“ http://www.xyz.com/jpgs/navbar1.jpg”>
<h1>Welcome to our Web site!</h1>
<a href=“page2.html”>Click here to enter</a>
</body>
</html>
SOURCE: BRUCE MAGGS
Downloading www.xyz.com
- before Akamai
DNS
1
WWW.XYZ.COM
2 10.10.123.8
5
Content Provider’s
Web Server
3
4
6
• User enters www.xyz.com
• Browser requests IP
address for www.xyz.com
• DNS returns IP address
• Browser requests HTML
• Content provider’s web
server returns HTML
7
10.10.123.8
• Browser obtains IP addresses for
hostnames listed in URLs of objects
embedded on page
• Browser requests
embedded objects
• Content provider’s web server
returns embedded objects
SOURCE: BRUCE MAGGS
DNS Resolution
.com .net
Root
(ICANN)
4
5
TTL:
1 Day
Local Name
Server
6
7
TTL:
30 Minutes
10
8
3
1
xyz.com
DNS Servers
2
Browser’s
Cache
9
OS
SOURCE: BRUCE MAGGS
Problems with the Centralized
Approach
• Slow
– content must traverse multiple
backbones and long distances
• Unreliable
– delivery may be prevented by
congestion or backbone
peering problems
• Not scalable
– usage limited by bandwidth
available at master site
• Inferior streaming quality
– packet loss, congestion, and
narrow pipes degrade stream
quality
SOURCE: BRUCE MAGGS
The Akamai Solution
• Monitors the Internet
and routes around
trouble spots
• Distributes all forms
of content and
supports applications
• Provides feedback
on hit counts to
content providers
SOURCE: BRUCE MAGGS
Downloading www.xyz.com
- The Akamai way
DNS
1
Content Provider’s
Web Server
5
WWW.XYZ.COM
2
3
4
6
• User enters www.xyz.com
• Browser requests IP
address for www.xyz.com
• DNS returns IP address
• Browser requests HTML
• Content provider’s web server
returns page with Akamaized
URLs
• Browser obtains IP address
of optimal Akamai server for
embedded objects
• Browser obtains objects
from optimal Akamai server
SOURCE: BRUCE MAGGS
Content Delivery Using Akamai
Embedded URLs are Converted to ARLs
<html>
<head>
<title>Welcome to xyz.com!</title>
</head>
ak
<body>
<img src=“ http://www.xyz.com/logos/logo.gif”>
<img src=“ http://www.xyz.com/jpgs/navbar1.jpg”>
<h1>Welcome to our Web site!</h1>
<a href=“page2.html”>Click here to enter</a>
</body>
</html>
SOURCE: BRUCE MAGGS
Typical Page Content
Total page
Total Akamai Served
87,550 bytes
68,756 bytes
Banner Ads
16,174 bytes
Logos
3,395 bytes
Gif links
22,395 bytes
Navigation Bar
9,674 bytes
Fresh Content
17,118 bytes
78%
Page Served by Akamai
SOURCE: BRUCE MAGGS
Akamai DNS Resolution
4
xyz.com
510.10.123.5
xyz.com’s
nameserver
akamai.net
8
a212.g.akamai.net
7
6
.com .net
Root
(ICANN)
9
15.15.125.6
ak.xyz.com
10 g.akamai.net
20.20.123.55
11
Akamai High-Level DNS Servers
12a212.g.akamai.net
30.30.123.5
Local Name
Server
End User
16
Browser’s
Cache
Akamai Low-Level DNS Servers
14
3
1
13
2
15
OS
SOURCE: BRUCE MAGGS
DNS Maps & Time-To-Live
• Map creation is
based on
measurements of:
–
–
–
–
Internet congestion
System loads
User demands
Server status
• Maps are
constantly
recalculated:
– Every few minutes
for HLDNS
– Every few seconds
for LLDNS
Time To Live
1 day
Root
30 min.
HLDNS
30 sec.
LLDNS
TTL of DNS responses gets shorter
further down the hierarchy
SOURCE: BRUCE MAGGS
Advantages of the Akamai
Solution
• Fast
– Content is served
from locations near
to end users
• Reliable
– No single point
of failure
– Automatic fail-over
• Scalable
– Master site no
longer requires
massive available
bandwidth
SOURCE: BRUCE MAGGS
Akamai
Distributing servers allows better handling of peak loads
SOURCE: AKAMAI
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Akamai Network Deployment
13000+
Servers
Current Installations
1000+
Networks
66+
Countries
Over 1300 Web
Sites are Now Akamaized
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Anatomy of a Network
Origin Server Scalability,
Speed of Light
Peering Point
Congestion
Available
Bandwidth
“Middle Mile”
“Last Mile”
Gigabit Optical
Network
Peering Capacity,
ISP Network Capacity
Internet Backbone
Cross-Internet
connections
T1, DSL,
Cable Modem 10 Mb to 1 Gb
Dial-up
Ethernet
Local Loop
Premises
Network
SOURCE: CISCO
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Content Network Building Blocks
Content
Delivery
Services
Content
Delivery
Networks
L2/L3
Networks
Web Hosting
E-Commerce
Content Distribution
& Management
Content
Routing
Streaming
Content
Switching
Applications
Content
Edge-Delivery
Intelligent Network Services
Highly available, scalable, performance network at Layer 2/3
Mobile
L2 = DATA LINK LAYER
L3 = NETWORK LAYER
Fixed
Wireless
Cable
DSL
Dedicated/
ATM/FR
ISDN/Dial
SOURCE: CISCO
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Content Delivery Map
Primary Data Center
CDM
Content
Distribution
Manager
Distributed Data Center
Origin Web
Server
Content Edge
Delivery
Content Distribution
Content Routing
Content Switching
Content Edge-Delivery
Content Delivery Node
Content
Router
Content Delivery Node
User Community
Local DNS
Server
SOURCE: CISCO
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
Content
Switching
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Peer-to-Peer Content Delivery
• Idea: store content at individual nodes
instead of in one central place
– Similar to Akamai
• Problem: discovery (searching)
• How do we find the content?
• Unexpected effect
– Clients and servers can be anonymous and
untraceable
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Non-Indexed Search (e.g. Gnutella)
D
B X?
A
X?
X?
E
X?
X?
X?
C
X?
F
X
SOURCE: ARTURO CRESPO
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Non-Indexed Search
• Advantages
– Simple
– Robust
– Anonymous
• Disadvantages
– Performance
– Network flooding
– Anonymous
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Centralized Index Search (Napster)
I
X
…
X?
X?
X
A
C
B
D
SOURCE: ARTURO CRESPO
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Centralized Index Search
• Advantages
– Performance
– Can be shut down
• Disadvantage
– Vulnerable to attack
– Index node may become a bottleneck
– Can be shut down
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Freenet: Chain Algorithm
Darling,
Tell me
the truth!
Believe me, I
don’t have it.
SOURCE: CHUN ZHANG
Freenet
C
B
D
A
But I know Joe
may have it
since I
borrowed
similar stuff
him last time.
I
SOURCE: CHUN ZHANG
Freenet
Sorry, No
C
B
A
D
A, Help me!
I
SOURCE: CHUN ZHANG
Morpheus Supernode Architecture
SN2
SN4
SN3
SN1
SOURCE: PAVEL ZASLAVSKY
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Morpheus Query with Fast Stream
56.78.79.5
SN2
SN3
SN1
234.67.8.9
123.5.78.9
234.67.8.9
56.78.79.5
123.5.78.9
SOURCE: PAVEL ZASLAVSKY
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Content-Addressable Networks
• Build a distributed directory similar to DNS by
observing queries and their answers
fish(dolphin, shark)
fish(dolphin, shark)
animal
Insect(ant, fly)
Insect(ant, fly)
SOURCE: HAIYONG WANG
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Delivering Video
• Uncompressed video
– 640 x 480 resolution
– 24-bit color = 7.37 megabits/frame
– 30 frames per second = 210 megabits/sec.
– 1920 x 1080 = 1.5 Gbps
• Compressed video
– MPEG-2. Broadcast quality. 15 Mbps
– AVI, QuickTime. 500 Kbps
– RealNetworks. Designed for network speeds.
20-200 Kbps. (At 20 Kbps, compression factor is
greater than 10,000.)
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Streaming
• Begin delivering content without full download
– Lower latency
– Low client storage requirement
– Can start streaming at any point (not necessarily
beginning
– Live streaming possible (real-time compression)
• Disadvantages
– Lower quality, also affected by network connection
– More time needed to compress
– Need special server
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
1.
True Streaming
USER REQUESTS
STREAMING MEDIA
2.
BROWSER
REQUESTS
pnm
METAFILE
4.
pnm
METAFILE
BROWSER
with video
plug-in
5.
CLIENT SIDE
SERVER SIDE
3.
SERVER
UPLOADS
METAFILE
pnm
METAFILE
pnm
METAFILE
20-751 ECOMMERCE TECHNOLOGY
REALPLAYER
PLUG-IN
PLUG-IN
REQUESTS
STREAM
STREAM
6.
WEB SERVER
BROWSER
INVOKES
PLUG-IN
REALSERVER
UPLOADS
STREAM
REALSERVER
STREAM
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
How Streaming Works
Web page is on
WWWanserver
HTTP server
User visits a
web page. Page
Metafile causes
contains a link
Player connects
thetoplayer to
to a metafile on
End users
the
media
server
launch
and
Media server
web server.
Mediaand
filerequests
is on the
the
passes to it the
streams file to the
a media server
path of the media
users player forfile.
server and media
viewing.
file.
Media server
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
SMIL: Layout and Synchronization
Real
Server
REQUEST .SMI
FILE
SMIL =SYNCHRONIZED MULTIMEDIA INTEGRATION LANGUAGE
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
SOURCE: REALNETWORKS
COPYRIGHT © 2002 MICHAEL I. SHAMOS
SMIL: Layout and Synchronization
Real
Server
REQUEST .SMI
FILE
(0,0)
376
video region
news region
184
(0,144)
ticker region
<head><layout>
<root-layout height="184" width=”376" background-color="black"/>
<region id="video" left="0" top="0" height="144" width="176" z-index="1"/>
<region id="news" left="176" top="0" height="144" width=”200" z-index="1"/>
<region id="ticker" left="0" top="144" height="40" width=”376" z-index="1"/>
</layout></head>
SOURCE: REALNETWORKS
SMIL: Layout and synchronization (3/3)
Real
Server
REQUEST .SMI
FILE
(0,0)
376
video region
video.rm
news region
newstext.rt
184
(0,144)
ticker region
tickertext.rt
<body><par>
<video id="the_video" src="rtsp://media.real.com/video.rm" region="video"/>
<text id="the_news" src="rtsp://media.real.com/newstext.rt" region="news"/>
<text id="the_ticker" src="rtsp://media.real.com/tickertext.rt" region="ticker"/>
</par></body>
SOURCE: REALNETWORKS
Content Creation/Delivery Needs
• Localized content
– geography & language-specific pages
•
•
•
•
•
•
•
Personalization
Streamlined content
Wireless content
Match content richness to line speed
Animation
Software, file downloads
Compression
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Catalog Management
SOURCE: CARDONET
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Catalog Management
cXBL = Common XML Business Library
cXML = Commerce XML
SOURCE: CARDONET
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Content Creation Tools
• Authoring tools
– FrontPage, Word
• Graphic Tools
– Paint, Illustrator
• Animation
– Flash
– Virtual Reality Modeling Language (VRML)
• Audio/Video
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS
Q&A
20-751 ECOMMERCE TECHNOLOGY
SUMMER 2002
COPYRIGHT © 2002 MICHAEL I. SHAMOS