Transcript Principles of reliable data transfer

No Class on Friday
There will be NO class on:
FRIDAY 1/27/17
Homework 2 is out: Due 2/8/17
Project 2 is out: Due 2/22/17
1
Web caches (proxy server)
Goal: satisfy client request without involving origin server
o user sets browser: Web
accesses via cache
o browser sends all HTTP
requests to cache
o
o
object in cache: cache
returns object
else cache requests
object from origin
server, then returns
object to client
origin
server
client
client
Proxy
server
origin
server
2
More about Web caching
o Cache acts as both client
and server
o Typically cache is installed
by ISP (university,
company, residential ISP)
Why Web caching?
o Reduce response time for
client request.
o Reduce traffic on an
institution’s access link.
o Internet dense with caches
enables “poor” content
providers to effectively
deliver content (but so
does P2P file sharing)
3
Caching example
Assumptions
o average object size = 100,000
bits
o avg. request rate from
institution’s browsers to origin
servers = 15 req/sec
o delay from institutional router
to any origin server and back
to router = 2 sec
Consequences
utilization on LAN = 15%
o utilization on access link = 100%
o total delay = Internet delay +
access delay + LAN delay
o
= 2 sec + minutes + milliseconds
origin
servers
public
Internet
1.5 Mbps
access link
institutional
network
10 Mbps LAN
o
institutional
cache
4
Caching example (cont)
Possible solution
o increase bandwidth of access
link to, say, 10 Mbps
Consequences
o
o
o
o
o
utilization on LAN = 15%
utilization on access link = 15%
Total delay = Internet delay +
access delay + LAN delay
= 2 sec + msecs + msecs
often a costly upgrade
origin
servers
public
Internet
10 Mbps
access link
institutional
network
10 Mbps LAN
institutional
cache
5
Caching example (cont)
origin
servers
Install cache
o suppose hit rate is .4
Consequence
public
Internet
o 40% requests will be
satisfied almost immediately
o 60% requests satisfied by
origin server
o utilization of access link
reduced to 60%, resulting in
negligible delays (say 10
msec)
o total avg delay = Internet
delay + access delay + LAN
delay = .6*(2.01) secs +
milliseconds < 1.4 secs
1.5 Mbps
access link
institutional
network
10 Mbps LAN
institutional
cache
6
Conditional GET
o Goal: don’t send object if
cache has up-to-date cached
version
o cache: specify date of
cached copy in HTTP request
o
If-modified-since:
<date>
o server: response contains no
object if cached copy is upto-date:
o
HTTP/1.0 304 Not
Modified
server
cache
HTTP request msg
If-modified-since:
<date>
HTTP response
object
not
modified
HTTP/1.0
304 Not Modified
HTTP request msg
If-modified-since:
<date>
HTTP response
object
modified
HTTP/1.0 200 OK
<data>
7
Outline
r Principles of network applications
m
m
App architectures
App requirements
r Web and HTTP
r FTP
8
FTP: the file transfer protocol
user
at host
r
r
r
r
FTP
FTP
user
client
interface
local file
system
file transfer
FTP
server
remote file
system
transfer file to/from remote host
client/server model
m client: side that initiates transfer (either to/from
remote)
m server: remote host
ftp: RFC 959
ftp server: port 21
9
FTP: separate control, data connections
r
r
r
r
r
FTP client contacts FTP
server at port 21, specifying
TCP as transport protocol
Client obtains authorization
over control connection
Client browses remote
directory by sending
commands over control
connection.
When server receives a
command for a file transfer,
the server opens a TCP data
connection to client
After transferring one file,
server closes connection.
TCP control connection
port 21
FTP
client
TCP data connection
port 20
FTP
server
Server opens a second TCP data
connection to transfer another
file.
Control connection: “out of
band”
FTP server maintains “state”:
current directory, earlier
authentication
10
Application layer
Electronic Mail
SMTP, POP3, IMAP
DNS
P2P file sharing
11
Electronic Mail
outgoing
message queue
user mailbox
Three major components:
user agents
mail servers
simple mail transfer
protocol: SMTP
user
agent
mail
server
User Agent
SMTP
a.k.a. “mail reader”
composing, editing, reading
mail
mail messages
server
e.g., Outlook, elm, Mozilla
Thunderbird, iPhone mail
client
user
outgoing, incoming messages
agent
stored on server
SMTP
SMTP
user
agent
mail
server
user
agent
user
agent
user
agent
12
Electronic Mail: mail servers
user
agent
Mail Servers
mailbox contains incoming
messages for user
message queue of outgoing
(to be sent) mail messages
SMTP protocol between mail
servers to send email
messages
client: sending mail
server
“server”: receiving mail
server
mail
server
SMTP
SMTP
mail
server
user
agent
SMTP
user
agent
mail
server
user
agent
user
agent
user
agent
13
Scenario: Alice sends message to Bob
1) Alice uses UA to compose
message and “to”
[email protected]
2) Alice’s UA sends message
to her mail server; message
placed in message queue
3) Client side of SMTP opens
TCP connection with Bob’s
mail server
1
user
agent
2
mail
server
3
4) SMTP client sends Alice’s
message over the TCP
connection
5) Bob’s mail server places the
message in Bob’s mailbox
6) Bob invokes his user agent
to read message
mail
server
4
5
6
user
agent
14
SMTP: final words
SMTP uses persistent
connections
SMTP requires message
(header & body) to be in 7bit ASCII
Comparison with HTTP:
HTTP: pull
SMTP: push
both have ASCII
command/response
interaction, status codes
HTTP: each object
encapsulated in its own
response msg
SMTP: multiple objects
sent in multipart msg
15
Mail message format
SMTP: protocol for
exchanging email msgs
RFC 822: standard for text
message format:
header lines, e.g.,
To:
From:
Subject:
different from SMTP
commands!
header
blank
line
body
body
the “message”, ASCII
characters only
16
Message format: multimedia extensions
MIME: multimedia mail extension, RFC 2045, 2056
additional lines in msg header declare MIME content
type
MIME version
method used
to encode data
multimedia data
type, subtype,
parameter declaration
encoded data
From: [email protected]
To: [email protected]
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
.........................
......base64 encoded data
17
Mail access protocols
user
agent
SMTP
SMTP
sender’s mail
server
access
protocol
user
agent
receiver’s mail
server
SMTP: delivery/storage to receiver’s server
Mail access protocol: retrieval from server
POP: Post Office Protocol [RFC 1939]
• authorization (agent <-->server) and download
IMAP: Internet Mail Access Protocol [RFC 1730]
• more features (more complex)
• manipulation of stored msgs on server
HTTP: Hotmail , Yahoo! Mail, etc.
18
POP3 protocol
authorization phase
client commands:
user: declare username
pass: password
server responses
+OK
-ERR
transaction phase, client:
list: list message numbers
retr: retrieve message by
number
dele: delete
quit
S: +OK POP3 server ready
C: user bob
S: +OK
C: pass hungry
S: +OK user successfully logged
on
C: list
S: 1 498
S: 2 912
S: .
C: retr 1
S: <message 1 contents>
S: .
C: dele 1
C: retr 2
S: <message 1 contents>
S: .
C: dele 2
C: quit
S: +OK POP3 server signing off
19
Outline
Electronic Mail
SMTP, POP3, IMAP
DNS
P2P file sharing
20
DNS: Domain Name System
People: many identifiers:
SSN, name, passport #
Internet hosts, routers:
IP address (32 bit) used for addressing
datagrams
“name”, e.g.,
www.yahoo.com - used
by humans
Q: map between IP
addresses and name ?
Domain Name System:
distributed database
implemented in hierarchy of
many name servers
application-layer protocol
host, routers, name servers to
communicate to resolve names
(address/name translation)
note: core Internet
function, implemented as
application-layer protocol
complexity at network’s
“edge”
21
DNS
DNS services
Hostname to IP
address translation
Host aliasing
Canonical and alias
names
Mail server aliasing
Load distribution
Replicated Web
servers: set of IP
addresses for one
canonical name
Why not centralized DNS?
single point of failure
traffic volume
distant centralized
database
maintenance
doesn’t scale!
22
Distributed, Hierarchical Database
Root DNS Servers
com DNS servers
yahoo.com amazon.com
DNS servers DNS servers
org DNS servers
pbs.org
DNS servers
edu DNS servers
poly.edu umass.edu
DNS serversDNS servers
Client wants IP for www.amazon.com; 1st approx:
Client queries a root server to find com DNS
server
Client queries com DNS server to get amazon.com
DNS server
Client queries amazon.com DNS server to get IP
address for www.amazon.com
23
DNS: Root name servers
contacted by local name server that can not resolve name
root name server:
contacts authoritative name server if name mapping not known
gets mapping
returns mapping to local name server
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also Los Angeles)
d U Maryland College Park, MD
k RIPE London (also Amsterdam,
g US DoD Vienna, VA
Frankfurt)
i Autonomica, Stockholm
(plus 3
h ARL Aberdeen, MD
other
locations)
j Verisign, ( 11 locations)
m WIDE Tokyo
e NASA Mt View, CA
f Internet Software C. Palo Alto,
CA (and 17 other locations)
13 root name
servers worldwide
b USC-ISI Marina del Rey, CA
l ICANN Los Angeles, CA
24
TLD and Authoritative Servers
Top-level domain (TLD) servers: responsible
for com, org, net, edu, etc, and all top-level
country domains uk, fr, ca, jp.
Network solutions maintains servers for com TLD
Educause for edu TLD
Authoritative DNS servers: organization’s
DNS servers, providing authoritative
hostname to IP mappings for organization’s
servers (e.g., Web and mail).
Can be maintained by organization or service
provider
25
Local Name Server
Does not strictly belong to hierarchy
Each ISP (residential ISP, company,
university) has one.
Also called “default name server”
When a host makes a DNS query, query is
sent to its local DNS server
Acts as a proxy, forwards query into hierarchy.
26
Example
root DNS server
2
Host at cis.poly.edu
wants IP address for
gaia.cs.umass.edu
3
TLD DNS server
4
5
local DNS server
dns.poly.edu
1
8
requesting host
7
6
authoritative DNS server
dns.cs.umass.edu
cis.poly.edu
gaia.cs.umass.edu
27
Recursive queries
root DNS server
recursive query:
puts burden of name
resolution on
contacted name
server
iterated query:
contacted server
replies with name of
server to contact
“I don’t know this
name, but ask this
server”
2
3
7
6
TLD DNS serve
local DNS server
dns.poly.edu
1
5
4
8
requesting host
authoritative DNS server
dns.cs.umass.edu
cis.poly.edu
gaia.cs.umass.edu
28
DNS: caching and updating records
once (any) name server learns mapping, it caches
mapping
cache entries timeout (disappear) after some
time
TLD servers typically cached in local name
servers
• Thus root name servers not often visited
update/notify mechanisms under design by IETF
RFC 2136
http://www.ietf.org/html.charters/dnsind-charter.html
29
DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)
Type=A
name is hostname
value is IP address
Type=NS
name is domain (e.g.
foo.com)
value is IP address of
authoritative name
server for this domain
Type=CNAME
name is alias name for some
“cannonical” (the real) name
www.ibm.com is really
servereast.backup2.ibm.com
value is cannonical name
Type=MX
value is Canon. name of
mailserver associated with
alias name
30
DNS protocol, messages
DNS protocol : query and reply messages, both with
same message format
msg header
identification: 16 bit #
for query, reply to query
uses same #
flags:
query or reply
recursion desired
recursion available
reply is authoritative
31
DNS protocol, messages
Name, type fields
for a query
RRs in reponse
to query
records for
authoritative servers
additional “helpful”
info that may be used
32
Pure P2P architecture
no always-on server
arbitrary end systems
directly communicate
peers are intermittently peer-peer
connected and change IP
addresses
Three topics:
file distribution
searching for information
case Study: Skype
33
File Distribution: Server-Client vs P2P
Question : How much time to distribute file
from one server to N peers?
us: server upload
bandwidth
Server
us
File, size F
dN
uN
u1
d1
u2
ui: peer i upload
bandwidth
d2
di: peer i download
bandwidth
Network (with
abundant bandwidth)
34
File distribution time: server-client
server sequentially
sends N copies:
NF/us time
client i takes F/di time
to download
Time to distribute F
to N clients using
client/server approach
Server
F
us
dN
u1 d1 u2
d2
Network (with
abundant bandwidth)
uN
= dcs = max
{ NF/u , F/min(d ) }
s
i
i
increases linearly in N
(for large N)
35
File distribution time: P2P
Server
server must send one
copy: F/us time
client i takes F/di time
to download
NF bits must be
downloaded (aggregate)

F
us
d2
Network (with
abundant bandwidth)
dN
uN
fastest possible upload rate: us +
dP2P = max
u1 d1 u2
Su
i
{ F/u , F/min(d ) , NF/(u + Su ) }
s
i
i
s
i
36
Server-client vs. P2P: example
Client upload rate = u, F/u = 1 hour, us = 10u, dmin ≥ us
Minimum Distribution Time
3.5
P2P
Client-Server
3
2.5
2
1.5
1
0.5
0
0
5
10
15
20
25
30
35
N
37