Transcript Chapter2

Chapter 2: Application layer
 2.1 Principles of
 2.6 P2P file sharing

 2.8 Socket programming



network applications
2.2 Web and HTTP
2.3 FTP
Online gaming
2.4 Electronic Mail


SMTP,
POP3, IMAP
 2.5 DNS
 2.7 VOIP
with TCP


Introduce c sock program
Programming assignment
 2.9 Socket programming
with UDP
 2.10 Building a Web
server
2: Application Layer
1
Online Gaming Design Requirement
 Consistency
 Every gamer has the exactly same view of the game
 Security
 No cheating, fair play Robustness
 A failure of one client has no impact on others
 Real time
 Processes are delivered no later than the time needed
for effective control
 Robustness
 The failure of one client does not affect others
 Scalable
 Less bandwidth requirement
 Dial-up connection is good to work
2: Application Layer
2
Basic Game Model
 Single machine game basic model
 Basic animation: small time-delay refreshing
 Interaction: command  action in next turn
time
 Multiplayer online game basic model
 Each machine runs the exact simulation
• How? Random number generator
A client’s commands known to all clients
 A command is executed on all client at the same
time

2: Application Layer
3
Consistency :
Bucket Synchronization
A
time
B
 Bucket interval > transmission delay
 All calculations are delayed until the end of
each cycle
 Bucket frequency = rate a human vision
perceives smooth motion
 What if the network delay is longer?
2: Application Layer
4
Consistency: Pipelining
1
2
3
4
5
time
6
 Commands issued between turn 2 and turn 3 will be
executed at turn 6

Suppose transmission delay is < 3 turns
 Speed control and “lag”
 Adaptive turn frequency and delay time for exact view on
all clients
• Communication delay, computer simulation speed
• One client too slow  all clients run slow
2: Application Layer
5
Solving Inconsistency
•
Happens if a command packet is lost or received
too late.
 Methods
 All clients pause and wait for lost command
 Server update states to all clients
•
Example: your role player returns back to original
position
 Others?
•
•
•
Sacrifice one client by forcing this client
synchronizes with the server
Kick out the slowest client
??
 Goal: minimize the inconsistency period
2: Application Layer
6
Transmission Protocol
 Usually use UDP for transport protocol
 Achieve reliable transfer by application
layer

Benefit:
• Game program can decide which data should be
reliably delivered
– Do: game session update message
– Don’t: too late command message
• Game will not stuck by one not transmitted packet

Disadvantage:
• Make programming much more complex
2: Application Layer
7
Security: Game Cheating
 Suppress-correct cheat :
 Host gains advantage by purposefully dropping update
messages
 Look ahead cheat :
 Cheat program makes decision right after receiving
updates from participating players
• Example: automatic dodge bullet, automatic aim target

Popular in war gaming
• Gamer (human) response is much slower than look ahead
response!
2: Application Layer
8
Security: Game Cheating
 Solutions:
 Remote client program verification
• Install verification patch
– This has been widely used in grid computing
• Client uploading gaming data continuously to verifying server
– Verifying server detects anomaly behaviors
– Can use statistical pattern for detection, e.g.,
» Steven Gianvecchio, Zhenyu Wu, Mengjun Xie, and
Haining Wang, Battle of Botcraft: Fighting Bots in Online
Games with Human Observational Proofs,
In ACM CCS 2009, Chicago, IL, November 2009.

Cryptographic method for pure distributed systems
2: Application Layer
9
Materials on Online Gaming
 Chris GauthierDickey “COMP 4705: Networks and Games”
 "1500 archers on a 28.8: Network Programming in the Age of
Empires and Beyond"

http://www.3impact.com/3Impact_Engine/hulk7123/multiplayer.htm
 Learn Internet Game Programming with Java

Talks about how to program java-based web online game
• Not multiplayer, but play through the Internet
2: Application Layer
10
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 VOIP
 2.8 Socket programming
with TCP
 2.9 Socket programming
with UDP
 2.10 Building a Web
server
2: Application Layer
11
FTP: the file transfer protocol
user
at host
FTP
FTP
user
client
interface
file transfer
local file
system
FTP
server
remote file
system
 transfer file to/from remote host
 client/server model
client: side that initiates transfer (either to/from
remote)
 server: remote host
 ftp: RFC 959
 ftp server: port 21

2: Application Layer
12
FTP: separate control, data connections
TCP control connection
port 21
 FTP client contacts FTP




server at port 21, specifying
TCP as transport protocol
Client obtains authorization
over control connection
Client browses remote
directory by sending
commands over control
connection.
When server receives a
command for a file transfer,
the server initiates a TCP
data connection to client
After transferring one file,
server closes connection.
FTP
client
TCP data connection
port 20
FTP
server
 Server initiates a second TCP
data connection to transfer
another file.
 Control connection: “out of
band”
 FTP server maintains “state”:
current directory, earlier
authentication
2: Application Layer
13
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 VOIP
 2.8 Socket programming
with TCP
 2.9 Socket programming
with UDP
 2.10 Building a Web
server
2: Application Layer
14
Electronic Mail
outgoing
message queue
user mailbox
user
agent
Three major components:
 user agents
 mail servers
mail
server
SMTP
 simple mail transfer
protocol: SMTP
User Agent
 a.k.a. “mail reader”
 composing, editing, reading
mail messages
 e.g., Eudora, Outlook, elm,
Netscape Messenger
 outgoing, incoming messages
stored on server
SMTP
mail
server
user
agent
SMTP
user
agent
mail
server
user
agent
user
agent
user
agent
2: Application Layer
15
Electronic Mail: mail servers
user
agent
Mail Servers
 mailbox contains incoming
messages for user
 message queue of outgoing
(to be sent) mail messages
 SMTP protocol between mail
servers to send email
messages
 client: sending mail
server
 “server”: receiving mail
server
mail
server
SMTP
SMTP
mail
server
user
agent
SMTP
user
agent
mail
server
user
agent
user
agent
user
agent
2: Application Layer
16
Electronic Mail: SMTP [RFC 2821]
 uses TCP to reliably transfer email message from client
to server, port 25
 direct transfer: sending server to receiving server
 three phases of transfer
 handshaking (greeting)
 transfer of messages
 closure
 command/response interaction
 commands: ASCII text
 response: status code and phrase
 messages must be in 7-bit ASCII
 The reason why an email with an attached file sent
out is bigger than original attached file
2: Application Layer
17
Scenario: Alice sends message to Bob
1) Alice uses UA to compose
message and “to”
[email protected]
2) Alice’s UA sends message
to her mail server; message
placed in message queue
(SMTP protocol)
3) Client side of SMTP opens
TCP connection with Bob’s
mail server
1
user
agent
2
mail
server
3
4) SMTP client sends Alice’s
message over the TCP
connection
5) Bob’s mail server places the
message in Bob’s mailbox
6) Bob invokes his user agent
to read message
mail
server
4
5
6
user
agent
2: Application Layer
18
Sample fake email sending
S: 220 longwood.cs.ucf.edu
C: HELO fake.domain
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: <[email protected]>
S: 250 [email protected]... Sender ok
C: RCPT TO: <[email protected]>
S: 250 [email protected] ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: from: “fake man” <[email protected]>
C: to: “dr. who” <who@who>
C: subject: who am I?
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 longwood.cs.ucf.edu closing connection
2: Application Layer
19
Try SMTP interaction for yourself:
 telnet servername 25
 see 220 reply from server
 enter HELO, MAIL FROM, RCPT TO, DATA, QUIT
commands
 “mail from” the domain may need to be existed
 “rcpt to” the user needs to be existed
 A mail server may or may not support “relay”
• CS email server supports relay in campus
above lets you send email without using email client
(reader)
2: Application Layer
20
SMTP: final words
 SMTP uses persistent
connections
 SMTP requires message
(header & body) to be in 7bit ASCII
 SMTP server uses
CRLF.CRLF to determine
end of message
Comparison with HTTP:
 HTTP: pull
 SMTP: push
 both have ASCII
command/response
interaction, status codes
 HTTP: each object
encapsulated in its own
response msg
 SMTP: multiple objects
sent in one msg
Problem: Only support ASCII content
2: Application Layer
21
Message format: multimedia extensions
 MIME (Multi-purpose Internet Mail Extensions)

multimedia mail extension, RFC 2045, 2056
 additional lines in msg header declare MIME content
type
MIME version
method used
to encode data
multimedia data
type, subtype,
parameter declaration
encoded data
From: [email protected]
To: [email protected]
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
.........................
......base64 encoded data
2: Application Layer
22
Mail access protocols
user
agent
SMTP
SMTP
sender’s mail
server
access
protocol
user
agent
receiver’s mail
server
 SMTP: deliver to receiver’s server
 Mail access protocol: retrieval from server



POP: Post Office Protocol [RFC 1939]
• authorization (agent <-->server) and download
• Port: 110
IMAP: Internet Mail Access Protocol [RFC 1730]
• more features (more complex)
• manipulation of stored msgs on server
HTTP: Hotmail , Yahoo! Mail, etc.
2: Application Layer
23
POP3 protocol (tcp: 110)
authorization phase
 client commands:
user: declare username
 pass: password
 server responses
 +OK
 -ERR

transaction phase, client:
 list: list message numbers
 retr: retrieve message by
number
 dele: delete
 quit
Telnet example (also check MIME)
S:
C:
S:
C:
S:
+OK POP3 server ready
user bob
+OK
pass hungry
+OK user successfully logged
C:
S:
S:
S:
C:
S:
S:
C:
C:
S:
S:
C:
C:
S:
list
Length(bytes)
1 498
2 912
.
retr 1
<message 1 contents>
.
dele 1
retr 2
<message 1 contents>
.
dele 2
quit
+OK POP3 server signing off
2: Application Layer
on
24
POP3 (more) and IMAP
More about POP3
 Read a message will
get a copy of it into
client machine

User can delete the
message on server by
choice
 One Message as a unit
 Cannot read a message
without downloading all
of it attachments
 Trouble for lowbandwidth users
IMAP
 Keep all messages in one
place: the server
 Allows user to organize
messages in folders

Create/delete folders on
email server
 Can retrieve any parts
of a message


Header only
Header + text content
 More popular than POP3
Both have encrypted protocols
2: Application Layer
25
Web-based Email
 Hotmail, Yahoo mail, AOL, …
 Convenient:
 only need Internet Browser
 User can access any where
 Unified interface:
 Same interface, email folders, any where
 Similar to IMAP on message organizing
 Combination with other Web services
 Hotmail+MSN messenger, AOL, new mail reminder
 Free!
2: Application Layer
26
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 VOIP
 2.8 Socket programming
with TCP
 2.9 Socket programming
with UDP
 2.10 Building a Web
server
2: Application Layer
27
DNS: Domain Name System
People: many identifiers:

SSN, name, passport #
Internet hosts, routers:


IP address (32 bit) –
128.119.40.12
unique ID
“name”, e.g.,
www.yahoo.com - used
by humans
Q: map between IP
addresses and name ?
Domain Name System:
 distributed database
implemented in hierarchy of
many name servers
 application-layer protocol
host, routers, name servers to
communicate to resolve names
(address/name translation)
 note: core Internet
function, implemented as
application-layer protocol
 complexity at network’s
“edge” philosophy
2: Application Layer
28
DNS
DNS services
 Hostname to IP
address translation
 Host aliasing


Canonical and alias
names
Many names for a single
host
 Mail server aliasing
Why not centralize DNS?
 single point of failure
 traffic volume
 distant centralized
database
 maintenance
doesn’t scale!
 Load distribution
 Replicated Web
servers: set of IP
addresses for one
canonical name
2: Application Layer
29
Distributed, Hierarchical Database
Root DNS Servers
com DNS servers
yahoo.com
amazon.com
DNS servers DNS servers
org DNS servers
pbs.org
DNS servers
edu DNS servers
ucf.edu
umass.edu
DNS serversDNS servers
Client wants IP for www.amazon.com; 1st approx:
 Client queries a root server to find com DNS
server
 Client queries “com” DNS server to get
amazon.com DNS server
 Client queries amazon.com DNS server to get IP
address for www.amazon.com
2: Application Layer
30
DNS: Root name servers
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also Los Angeles)
d U Maryland College Park, MD
g US DoD Vienna, VA
h ARL Aberdeen, MD
j Verisign, ( 11 locations)
k RIPE London (also Amsterdam, Frankfurt)
i Autonomica, Stockholm (plus 3 other
locations)
m WIDE Tokyo
e NASA Mt View, CA
f Internet Software C. Palo Alto, CA (and
17 other locations)
b USC-ISI Marina del Rey, CA
l ICANN Los Angeles, CA
13 root name
servers worldwide
2: Application Layer
31
TLD and Authoritative Servers
 Top-level domain (TLD) servers: responsible
for com, org, net, edu, etc, and all top-level
country domains uk, fr, ca, jp.
Network solutions maintains servers for com TLD
 Educause for edu TLD

 Authoritative DNS servers: organization’s
DNS servers, providing authoritative
hostname to IP mappings for organization’s
servers (e.g., Web and mail).

Can be maintained by organization or service
provider (paid by the organization)
2: Application Layer
32
Local Name Server
 Does not strictly belong to hierarchy
 Each ISP (residential ISP, company,
university) has one

Also called “default name server”
 When a host makes a DNS query, query is
sent to its local DNS server

Acts as a proxy (cache), forwards query into
hierarchy
2: Application Layer
33
Example
root DNS server
2
 Host at cis.poly.edu
3
wants IP address for
gaia.cs.umass.edu
TLD DNS server
4
5
local DNS server
dns.poly.edu
1
8
requesting host
7
6
authoritative DNS server
dns.cs.umass.edu
cis.poly.edu
gaia.cs.umass.edu
2: Application Layer
34
Recursive queries
recursive query:
2
 puts burden of name
resolution on
contacted name
server
 heavy load?
iterated query:
 contacted server
replies with name of
server to contact
 “I don’t know this
name, but ask this
server”
root DNS server
3
7
6
TLD DNS server
local DNS server
dns.poly.edu
1
5
4
8
requesting host
authoritative DNS server
dns.cs.umass.edu
cis.poly.edu
gaia.cs.umass.edu
2: Application Layer
35
DNS: caching and updating records
 once (any) name server learns mapping, it caches
mapping
 cache entries timeout (disappear) after some
time (keep fresh copy)
 TLD servers typically cached in local name
servers
• Thus root name servers not often visited
2: Application Layer
36
DNS records
DNS: distributed db storing Resource Records (RR)
RR format: (name,
 Type=A
 name is hostname
 value is IP address
value, type, ttl)
 Type=CNAME
 name is alias name for some
“canonical” (the real) name
www.ibm.com is really
 Type=NS
servereast.backup2.ibm.com
 name is domain (e.g.
 value is canonical name
foo.com)
 value is IP address of
 Type=MX
authoritative DNS
 value is name of mailserver
server for this domain
associated with name
2: Application Layer
37
DNS protocol, messages
DNS protocol : query and reply messages, both with
same message format
msg header
 identification: 16 bit #
for query, reply to query
uses same #
 flags:
 query or reply
 recursion desired
 recursion available
 reply is authoritative
2: Application Layer
38
DNS protocol, messages (UDP 53)
Name, type fields
for a query
RRs in response
to query
records for
authoritative servers
additional “helpful”
info that may be used
Let’s check a web example using Wireshark!
(MX record: nslookup –type=MX cs.ucf.edu or
dig mx cs.ucf.edu)
2: Application Layer
39
Inserting records into DNS
 Example: just created startup “Network Utopia”
 Register name networkuptopia.com at a registrar (e.g., Network
Solutions)


Need to provide registrar with names and IP addresses of your
authoritative name server (primary and secondary)
Registrar inserts two RRs into the com TLD server:
(networkutopia.com, dns1.networkutopia.com, NS)
(dns1.networkutopia.com, 212.212.212.1, A)
 Put in authoritative server dns1.networkutopia.com

Type A record for www.networkuptopia.com

Type MX record for networkutopia.com (email)
• Type CName for networkuptopia.com (alias)
• Type A record for the email server
 How do people get the IP address of your Web site?
2: Application Layer
40
Chapter 2: Application layer
 2.1 Principles of
network applications


app architectures
app requirements
 2.2 Web and HTTP
 Internet gaming
 2.4 Electronic Mail
 SMTP, POP3, IMAP
 2.6 P2P file sharing
 VOIP
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.5 DNS
2: Application Layer
41
P2P: Example
original “Napster” design
1) when peer connects, it
informs central server:


Bob
centralized
directory server
1
IP address
content
2) Alice queries Central for
“Hey Jude”
3) Central tells Alice that Bob
has it and Bob’s IP
4) Alice downloads file from
Bob (via HTTP, why?)
5) While Alice downloads,
others download files from
Alice’s computer, too
peers
1
3
1
2
1
4
Alice
2: Application Layer
42
P2P Example: BitTorrent
 Online Introduction

http://en.wikipedia.org/wiki/Bittorrent
 Designed by Bram Cohen in 2001
 BitTorrent Features

Remove centralized management server
• Use individual file-centered server

Better design on
• Load balancing and robustness
• File sharing enforcement
2: Application Layer
43
BitTorrent: Individual File-Centered P2P Design
 User A provides a BitTorrent “torrent”

Metadata: Tracker IP, port, file name
• Tracker provided by user A or a hosting

Publish torrent in well-know website
Tracker
 User A also provides a peer with full file
(“seed”)

once peer has entire file, it may (selfishly)
leave or (altruistically) remain (“seed”)
 Other users download seed first


Connect to tracker to get management
information
Download file from peers
 Robustness feature


peer
File broken into many piece (256KB/chunk)
Rarest-first download
• Make sure every piece has equal number of
copies in P2P network
2: Application Layer
44
BitTorrent (2)
Pulling Chunks
 at any given time,
different peers have
different subsets of
file chunks
 periodically, a peer
(Alice) asks each
neighbor for list of
chunks that they have.
 Alice sends requests
for her missing chunks
 rarest first
Sending Chunks: tit-for-tat
 Alice sends chunks to four
neighbors currently
sending her chunks at the
highest rate
 re-evaluate top 4 every
10 secs
 every 30 secs: randomly
select another peer,
starts sending chunks
 newly chosen peer may
join top 4
 “optimistically unchoke”
2: Application Layer
45
BitTorrent: Tit-for-tat
(1) Alice “optimistically unchokes” Bob
(2) Alice becomes one of Bob’s top-four providers; Bob reciprocates
(3) Bob becomes one of Alice’s top-four providers
With higher upload rate,
can find better trading
partners & get file faster!
2: Application Layer
46
BitTorrent Security Issues
 Common security problems in P2P network

Malicious sharing files
 Control-based vulnerability
 Tracker provided by ordinary users
• No authentication or authority

Tracker-controlled P2P network
• Tracker provides:
– Peer’s IP, connection port, peer’s obtained file pieces
2: Application Layer
47
BitTorrent Security Issues

Malicious tracker
can
• Divert P2P peers
to connect to a
victim on an
arbitrary port
• See our paper on
bitTorrent attack
(on my webpage)
2: Application Layer
48
Distributed Hash Table (DHT)
 DHT = distributed P2P database
 Database has (key, value) pairs;
 key: ss number; value: human name
 key: content type; value: IP address
 Peers query DB with key

DB returns values that match the key
 Peers can also insert (key, value) peers
2: Application Layer
49
DHT Identifiers
 Assign integer identifier to each peer in range
[0,2n-1].

Each identifier can be represented by n bits.
 Require each key to be an integer in same range.
 To get integer keys, hash original key.
 eg, key = h(“transformer trailer”)
 This is why they call it a distributed “hash” table
2: Application Layer
50
How to assign keys to peers?
 Central issue:

Assigning (key, value) pairs to peers.
 Rule: assign key to the peer that has the
closest ID.
 Convention in lecture: closest is the
immediate successor of the key.
 Ex: n=4; peers: 1,3,4,5,8,10,12,14;
key = 13, then successor peer = 14
 key = 15, then successor peer = 1

2: Application Layer
51
Circular DHT (1)
1
3
15
4
12
5
10
8
 Each peer only aware of immediate successor
and predecessor.
 “Overlay network” (n=4 for this example)
2: Application Layer
52
Circle DHT (2)
O(N) messages
on avg to resolve
query, when there
are N peers
0001
I am
Who’s resp
0011
for key 1110 ?
1111
1110
0100
1110
1110
1100
1110
1110
Define closest
as closest
successor
1010
0101
1110
1000
2: Application Layer
53
Circular DHT with Shortcuts
1
3
15
Who’s resp
for key 1110?
4
12
5
10
8
 Each peer keeps track of IP addresses of predecessor,
successor, short cuts.
 Reduced from 6 to 2 messages.
 Possible to design shortcuts so O(log N) neighbors, O(log
N) messages in query
2: Application Layer
54
Peer Churn
1
•To handle peer churn, require
3
15
4
12
5
10
each peer to know the IP address
of its two successors.
• Each peer periodically pings its
two successors to see if they
are still alive.
8
 Peer 5 abruptly leaves
 Peer 4 detects; makes 8 its immediate successor;
asks 8 who its immediate successor is; makes 8’s
immediate successor its second successor.
 What if peer 13 wants to join?
2: Application Layer
55
P2P Case study: Skype
Skype clients (SC)
 inherently P2P: pairs
of users communicate.
 proprietary
Skype
login server
application-layer
protocol (inferred via
reverse engineering)
 hierarchical overlay
with SNs
 Index maps usernames
to IP addresses;
distributed over SNs
Supernode
(SN)
2: Application Layer
56
Peers as relays
 Problem when both
Alice and Bob are
behind “NATs”.

NAT prevents an outside
peer from initiating a call
to insider peer
 Solution:
 Using Alice’s and Bob’s
SNs, Relay is chosen
 Each peer initiates
session with relay.
 Peers can now
communicate through
NATs via relay
2: Application Layer
57
Why using P2P?
 Resource bottleneck in server
 Take advantage of CPU, disk storage, bandwidth of
millions of computers



Grid computing: distributed computation (CPU)
File sharing: (storage, bandwidth)
Video streaming: (storage, bandwidth)
 Direct communication (less delay)
 Internet gaming, video/audio meeting
 Copyright avoidance
 Movie/music/software Piracy
2: Application Layer
58