Transcript Notes - Systems@NYU

Networks and distributed systems
Lec 1:
Evolution of the Internet
Jinyang Li
[email protected]
Networks and distributed systems
1
Know your staff
• Instructor: prof. Jinyang Li
– [email protected]
– Office Hour: Wed 5-6pm (715 Broadway Rm 705)
• Class webpage
http://www.news.cs.nyu.edu/classes/fa07
• Register for class mailing list
Networks and distributed systems
2
The course will teach you …
• to appreciate design principles of the Internet
– How it works and why it works
• to address new networking challenges
– How to do independent research
Networks and distributed systems
3
Who should take this class?
• Core grad-level class
– Satisfy M.S. requirement of a “project” class
– Satisfy Ph.D. breadth requirement
• Pre-requisite:
– Basic knowledge on networks
– Programming experience
• Useful books:
– Computer networks (Peterson & Davie)
– TCP/IP illustrated (Stevens)
Networks and distributed systems
4
Class material
• Lectures/readings
– Read assigned research papers before class
– Participate in class discussion
• Assignments
– Solve concrete problems, get your hands dirty!
• Projects
– Can you identify and tackle a challenge with guidance?
Networks and distributed systems
5
Grading
• Participation 20%
– two in-class mini-quiz on “readings du jour”
• Two take home assignments 20%
• P roject 60%
– Teams of 2-3 people
– Starting new week
– Bi-weekly meetings with me
Networks and distributed systems
6
Questions?
• Sign up sheet
Networks and distributed systems
7
A brief history of communication
• Telephone networks
– Dial to set up a path
– Paths carry analog voice signals from one
phone to another
– Networking means building paths
Networks and distributed systems
8
Building paths  connecting wires
Switchboard
Operators 1960
Networks and distributed systems
9
The quest for a survivable
network
• Sputnik --> ARPA --> survivable networks
• Telephone network is not survivable
– Destroy of a switching center is highly disruptive
– Not possible to build reliable paths under attacks
Networks and distributed systems
10
Packet switching
• Baran &
Davies (60s)
• Packets are digital, self-contained, of limited size
• Decentralized store and forward
– Networking means delivering packets to endpoints
pkt
len
header
Src
Dst
Addr addr
payload
Networks and distributed systems
11
An example of packet switching
H2
H1:P4
H2:P1
H3:P2
H2 H1
1
4
3
2
H1
H3
2
1
H1:P1
H2:P2
H3:P3
3
H2 H1
Networks and distributed systems
12
ARPANET
Networks and distributed systems
13
Internet:
Connecting many networks
•
Many packet switching networks
– ARPANET, Packet radio, SATnet
•
•
Goal: make networks work together!
Solution: TCP/IP
Kahn
&Cerf
Networks and distributed systems
14
Alternative #1:
single technology, single network
•
•
•
•
Render existing networks/apps useless
Does not accommodate new technology
Hard for decentralized control
(early phone network is like this)
Networks and distributed systems
15
Alternative #2: Translation Gateway
H1: ABCD
H2: 计算机
Translation
gateway
• Translation is hard
– different features/headers, N^2 combinations!
– How to translate addresses?
Networks and distributed systems
16
3. Internet wins
H2: 128.122.108.71
H1: 18.26.4.9
IP router
H1, GW’s
low-level addr
H2, GW’s
low-level addr
H1,H2’s IP addr
• IP over everything
– A uniform header / addressing format
Networks and distributed systems
17
Internet design challenges
• How to address networks and hosts?
– Address size? Resolve IP addr to subnet addr?
• How to compute route and forward packets?
• How to reliably deliver packets?
– Error recovery
– Flow control
• How to cope with different max packet size?
Networks and distributed systems
18
Addressing scheme
• Early 80s:
– 32-bit globally unique IP address
– 8 bit net number, 24 bit host number
– Embed subnet address to low 24 bit
•Now: 32-bit
– Variable length net number (CIDR)
– Address resolution protocol (ARP) to obtain
subnet addr (MAC addr) from IP
Networks and distributed systems
19
Routing
• Early 80s:
– 256-entry routing table, indexed by top 8 bits of addr
– Static default g/w
•Now:
– Intra-domain routing: OSPF, RIP
– Inter-domain routing: BGP
– approx. 250,000 BPG entries now
Networks and distributed systems
20
Reliable delivery
• Early 80s
– IP is best-effort only
– TCP ran at end hosts for error/flow control
•Now:
– IP is best effort only
– TCP is separated from IP
– TCP performs both error and congestion control
Networks and distributed systems
21
Packet size policy
• Early 80s:
– Senders only know local net’s MTU
– G/Ws fragment large packets into smaller MTUS
– End hosts reassembles fragments
•Now: same. :-)
Networks and distributed systems
22
“Internet” demo 1977
ARPANET
PR
net
SateNET
Networks and distributed systems
23
Internet map 1987
Networks and distributed systems
24
Why TCP/IP wins?
• Universal
– IP-over-everything
– Best effort only
– End-to-end design
• Robust
– Soft-state only inside network
– Fate sharing
– Be liberal in what you accept; be conservative in
what you send
Networks and distributed systems
25
Internet’s growing stage
•
•
•
•
•
1978 TCP/IP split
1984 Domain name system
1986 Incorporating congestion control in TCP
1990 ARPANET disappears, first ISP is born
Nodes double every year….
Networks and distributed systems
26
The revolution, good and bad
•
•
•
•
Email 1971
Apple II 1977, IBM PC 1981
Web 1990
VoIP, File sharing, Video streaming, Web 2.0
• Worms 1988, viruses
• DoS attacks
• Spam
Networks and distributed systems
27
Internet design goals
1. Interconnect different networks
–
–
Packet switching
Uniform addressing and IP header
2. Robust
–
–
Route packets instead of building path
Network is state-less, forwards packets based on addr
3. Flexible
–
–
IP is best effort only
Separate TCP from IP
Networks and distributed systems
28
The more problematic goals
4. Decentralization
– Routing across multiple admin domains is
still error-prone
5. Cheap and easy to attach new nodes
– Cumbersome to attach new devices, move
existing ones around
6. Accountability
Networks and distributed systems
29
Internet weaknesses
•
•
•
•
Assumes trusted participants
Assumes non-greedy sources
Security
Hard to incrementally deploy new protocols
Networks and distributed systems
30
New challenges
Networks and distributed systems
31
New types of networks: wireless
 2007
MIT Cartel
Networks and distributed systems
32
New networks: wireless mesh
Networks and distributed systems
33
New networks: sensor
Networks and distributed systems
34
New services
• What’s the next killer app?
Networks and distributed systems
35
Battling existing woes
Networks and distributed systems
36
Battling existing woes
Networks and distributed systems
37
Course Syllabus
1. Core networking concepts
– Naming and addressing
– Routing
– Managing shared resources
2. Wireless
3. Network services
4. Security
Networks and distributed systems
38
Part I: Core networking concepts
Reliable transport
Networks and distributed systems
39
Coping with best-effort
• Why don’t applications use IP directly?
– IP is a host-to-host protocol
– Many applications want reliable, in-order delivery
Networks and distributed systems
40
TCP software architecture
browser
ssh
apache
User-space
write
sshd
User-space
kernel
read
Networks and distributed systems
kernel
41
Coping with best-effort
• Challenges for a reliable transport protocol
–
–
–
–
Loss
Variable delays
Packet reordering
Duplicate packets
Networks and distributed systems
42
TCP overview
• Provides in-order, reliable, duplex byte-streams
Src
port
Dst
port
Seq #
Ack #
flags window cksum
• Uses cumulative ACKs
Data: 1:1460
Ack:
1461
1461:1700
1701
1701:1999
2000:2500
1701
Networks and distributed systems
2501:2800
1701
43
Reliability via retransmission
• How does TCP know when to re-transmit?
• Timer driven
– No ACKs for a while…
• Data driven
– Many duplicate ACKs
Networks and distributed systems
44
Timer-driven retransmission
• What is the ideal time to retransmit?
• What if we literally use RTT as timeout?
Networks and distributed systems
45
Timer-driven retransmission
• Calculate running average of RTT
– EWMA: srtt =  * r + (1 - ) * srtt
• Set timeout (RTO)
– Used to use RTO = 2 * srtt
– Now: RTO = srtt + 4 * rttdev
rttdev =  * |r-srtt| + (1- ) * rttdev
Networks and distributed systems
46
An example RTT distribution
Avg: 99.2ms
Std: 1.4ms
Networks and distributed systems
47
TCP timers
• What if a retransmission times out?
– Exponential back off
• TCP timeouts are extremely conservative
– Granularity of 500ms or 200ms
Networks and distributed systems
48
Fast retransmit
Data: 1:1460
Ack:
1461
1461:1700
1701:1999
2000:2500
1701
1701
2501:2800
1701
• If a segment is lost, duplicate ACKs result
• TCP retransmit upon seeing 3 duplicate
ACKs
Networks and distributed systems
49
Fast retransmit
• What would trick fast retransmit into
spurious retransmission?
• When would fast retransmit fail to avoid
timeout?
– Loss of a re-sent packet
– Multiple losses in a window
Networks and distributed systems
50
Fast Recovery
• How should sender change its congestion
window due to loss?
– Unchanged?
– Set to 1?
– Half ?
Networks and distributed systems
51
Is TCP good for all applications?
• TCP imposes high delay for retransmitted
packets
• TCP enforces in-order delivery
Networks and distributed systems
52