Transcript pptx - Dongsu Han

NETWORK PROGRAMMING
INTRO TO DISTRIBUTED SYSTEMS
FALL 2013
Some material taken from publicly available lecture slides
including Srini Seshan’s and David Anderson’s
Dongsu Han
L3-socket
Today’s Lecture

Lecture
 Transport
protocols: UDP and TCP
 Version control using SVN

TA-led programming session
 Editor,
socket programming
 Debugging with gdb

Announcement
 Programming
Assignment 1 is out. (Due in two weeks--추석)
Review of Previous Lectures

What is a distributed system? (L1)
 Definition
 Example

What are some of the important decisions made in the Internet
architecture? (L2)
 Narrow
 Why?:
waist: Network provides minimal functionality (best-effort)
to connect existing networks that are heterogeneous
 Stateless
architecture: no per-flow state inside network
 Why?
 For
survivability
Transport Protocols

Lowest level end-to-end protocol.
Header generated by sender is interpreted
only by the destination
 Routers view transport header as part of the
payload; they are not supposed to see the
transport header.

7
7
6
6
5
5
Transport
Transport
IP
IP
IP
Datalink
2
2
Datalink
Physical
1
1
Physical
router
4
What does the transport layer do?

No per-flow (connection) state in the network
 Transport
layer maintains the connection state. Transport layer identifies an
end-to-end connection.

How does the transport layer identify an end-point (application)?
De-multiplexing: Port numbers




Network layer gets you to the host.
A port identifies the sending or receiving socket (application).
A socket is one endpoint of a two-way communication between two
programs running on the network. A socket is bound to a port number
so that the transport layer can identify the application that data is
destined to be sent.
Identifying two end-points: An endpoint = <IP address, Port #>
Source endpoint = <SrcIPAddr, SrcPort>
 destination endponit = <DestIPAddr, DestPort>

Protocol and Port Demultiplexing
7


Multiple choices at each layer
Applications open a socket (identified by the port number)
Port 80
Port 22 Port 80 Port 22
TCP
UDP
Network
IP
TCP/UDP
IP
NET1
NET2
…
NETn
Type Field
=IP
Protocol Field Port Number
= TCP
= 80
Server and Client
Server and Client exchange messages over the
network through a common Socket API
Clients
Server
TCP/UDP
user
space
ports
Socket API
TCP/UDP
IP
IP
Ethernet Adapter
Ethernet Adapter
8
kernel
space
hardware
Transport protocols

UDP
 Provides

only integrity and de-multiplexing
TCP adds…
UDP: User Datagram Protocol [RFC 768]
10


“No frills,” “bare bones” Internet transport
protocol, “Best effort” service
UDP segments may be:



Lost, duplicated (bug, spurious retransmission at L2)
Delivered out of order to app
Connectionless:


No handshaking between UDP sender, receiver
Each UDP segment handled independently
Why is there a UDP?
• No connection establishment
(which can add delay)
• Simple: no connection state
at sender, receiver
• Small header
• No congestion control: UDP
can blast away as fast as de
sired
UDP, cont.
11

Often used for streaming mu
ltimedia apps (e.g., Skype)
 Loss
tolerant
 Rate sensitive

Other UDP uses (why?):
Length, in
bytes of UDP
segment,
including
header
32 bits
Source port #
Dest port #
Length
Checksum
 DNS

Reliable transfer over UDP
 Must
be at application layer
 Application-specific error rec
overy
Application
data
(message)
UDP segment format
Transport protocols

UDP
 Provides

only integrity and de-multiplexing
TCP adds…
 Connection-oriented
 Reliable
(error control)
 Ordered byte stream (in-order delivery)
 Bi-directional byte-stream
 Flow and congestion controlled
TCP

Protocol implemented entirely at the ends
 Fate

sharing
Abstraction provided
 Connection
oriented protocol (hand-shake or connection establishment)
 In-order byte-stream

Implemented functionality (next lecture)
 Flow
control
 Reliability (error control)
 Reassembly (in-order delivery)
 Congestion control
Difference in abstraction

UDP
 Recv(len)
call gives you a packet (the payload part)
len
len
UDP segment (packet)

UDP segment (packet)
TCP
establishment (hand-shake)  Will see in the TA led session.
Application
 Recv(len) call give you a sequence of bytes
send()
 Connection
len
Application
Recv()
len
Byte stream
Byte stream
Incoming stream
TCP– What goes on underneath
Byte stream
Packet (1~7)
Packet (8~10)
Packet (9~11)
Receive buffer
Packet (8~10)
I need seq# 8.
User Datagram Protocol(UDP):
An Analogy
UDP





Postal Mail
Single socket to receive messages
No guarantee of delivery
Not necessarily in-order delivery
Datagram – independent packets
Must address each packet





Single mailbox to receive letters
Unreliable 
Not necessarily in-order delivery
Letters sent independently
Must address each reply
Example UDP applications
Multimedia, voice over IP
16
Transmission Control Protocol
(TCP): An Analogy
Telephone Call
TCP




Reliable – guarantee delivery
Byte stream – in-order delivery
Connection-oriented – single socket per
connection
Setup connection followed by data
transfer




Guaranteed delivery
In-order delivery
Connection-oriented
Setup connection followed by
conversation
Example TCP applications
Web, Email, Telnet
REVISION CONTROL & MAKE
Overview

Revision control systems
 Motivation
 Features
 Subversion

primer
Make
 Simple
gcc
 Make basics and variables
 Testing

Useful Unix commands
Dealing with large codebases

Complications
 Code
synchronization
 Backups
 Concurrent modifications

Solution: Revision Control System (RCS)
 Store
all of your code in a repository on a server
 Metadata to track changes, revisions, etc…
 Also useful for writing books, papers, etc…
Basic RCS features (1/2)

Infinite undo
 go

Automatic backups
 all

back to previous revisions
your code ever, forever saved
Changes tracking
 see
differences between revisions
Basic RCS features (2/2)

Concurrency
 Multiple

Snapshots
 Save

people working at the same time
stable versions on the side (i.e. handin)
Branching
 Create
diverging code paths for experimentation
Typical RCS workflow
1.
2.
3.
4.
5.
6.
Create repository on RCS server
Checkout the repository to your local machine
Work locally: create, delete and modify files
Update the repository to check for changes other people might ha
ve made
Resolve any existing conflicts
Commit your changes to the repository
RCS implementations

Revision Control System (RCS)


Concurrent Versions System (CVS)


Early system, no concurrency or conflict resolution
Concurrency, versioning of single files
Subversion (SVN)

File and directory moving and renaming, atomic commits
Concurrent edits (1/2)
Carnegie
File v1
Mellon
Both check out file
Edits (lines 30-40)
Edits (lines 1-20)
commit
File v2
Update
Merged automatically!
Concurrent edits (2/2)
Carnegie
File v1
Mellon
Both check out file
Edits (lines 10-20)
Edits (lines 1-20)
commit
File v2
Update
Conflict needing manual inte
rvention!
Resolving conflicts

When changes can’t be merged automatically

SVN gives you 3 files:




file.mine : your file
file.rx : the remote file
file : original file with marked conflicts
You can



Discard your changes
Discard others’ changes
Go over each conflict and arbitrate as appropriate
Interacting with SVN

Command line


Readily available in andrew machines
Graphical tools



Easier to use
Subclipse for Eclipse gives IDE/SVN integration
tortoiseSVN
실습 server



IP: 143.248.141.52 ~ 143.248.141.59 (8대)
ID: your student id (e.g., 20110111)
Password: ee324EE#@$
Svn password














20060584 28482
20071122 6534
20081275 51085
20090058 50346
20090276 54586
20100120 23860
20100137 29297
20100441 17889
20100827 30690
20100967 7532
20110343 4967
20110456 9139
20110773 8574
20111047 36671
Command line SVN example (1/2)
$ svn co –-username=“<<student id>>” https://ina.kaist.ac.kr/svn/
test
$ cd test/trunk
$ vim <<student id>>.txt
$ vim <<student id>>.
$ svn add <<student id>>.txt
$ svn commit -m 'adding a text file with my student id‘
$ svn update
$ cd ../
$ svn cp trunk tags/<<student id>>_submission
$ svn commit -m 'making a tag of the trunk for submission!‘
$ echo -e ”<<student id>>" >> student.txt
$ svn commit –m “added my student id”
Command line SVN example (2/2)
Revision control lets you note (and then see) what you changed:
> svn log student.txt
r986 | ntolia | 2006-08-01 17:13:38 -0400 (Tue, 01 Aug 2006) | 6 lines
This allows the sp to get rid of chunks early before a transfer is complete.
Useful when a file is requested in-order and the file size > mem cache size
> svn diff -r 1:2 student.txt
Index: file
===================================================================
--- file (revision 1)
+++ file (revision 2)
@@-1,2+1,3@@
This isatestfile
-It startedwithtwolines
+It nolongerhastwolines
+it hasthree
General SVN tips





Update, make, test, only then commit
Merge often (svn up)
Comment commits
Avoid commit races
Modular design avoids conflicts
Know more


subversion.tigris.org for SVN software & info
svnbook.red-bean.com for SVN book
Makefile

You are on your own.
Make



Utility for executable building automation
Saves you time and frustration
Helps you test more and better
Simple gcc
If we have files:
• prog.c: The main program file
• lib.c: Library .c file
• lib.h: Library header file
% gcc -c prog.c -o prog.o
% gcc -c lib.c -o lib.o
% gcc lib.o prog.o -o binary
gcc flags
• Useful flags
1. -g: debugging hook
2. -Wall: show all warnings
3. -Werror: treat warning as errors
4. -O0, -O1, -O2, -O3: optimization level
5. -DDEBUG: macro for DEBUG (#define DEBUG)
• Avoid using dangerous optimizations that could affe
ct correctness
More gcc
% gcc -g -Wall -Werror -c prog.c -o prog.o
% gcc -g -Wall -Werror -c lib.c -o lib.o
% gcc -g -Wall -Werror lib.o prog.o -o binary
This gets boring, fast!
Makefile basics
• Build targets
target: dependency1 dependency2 ...
unix command (start line with TAB)
unix command
Makefile example
binary: lib.o prog.o
gcc -g -Wall lib.o prog.o -o binary
lib.o: lib.c
gcc -g -Wall -c lib.c -o lib.o
prog.o: prog.c
gcc -g -Wall -c prog.c -o prog.o
clean:
rm *.o binary
Makefile variables (1/7)
• Variables
CC = gcc
CFLAGS = -g -Wall -Werror
OUTPUT = binary
Makefile variables (2/7)
binary: lib.o prog.o
gcc -g -Wall lib.o prog.o -o binary
lib.o: lib.c
gcc -g -Wall -c lib.c -o lib.o
prog.o: prog.c
gcc -g -Wall -c prog.c -o prog.o
clean:
rm *.o binary
Makefile variables (3/7)
CC = gcc
CFLAGS = -g -Wall
OUTPUT = binary
$(OUTPUT): lib.o prog.o
$(CC) $(CFLAGS) lib.o prog.o -o binary
lib.o: lib.c
$(CC) $(CFLAGS) -c lib.c -o lib.o
prog.o: prog.c
$(CC) $(CFLAGS) -c prog.c -o prog.o
clean:
rm *.o $(OUTPUT)
Makefile variables (4/7)
CC = gcc
CFLAGS = -g -Wall
OUTPUT = binary
$(OUTPUT): lib.o prog.o
$(CC) $(CFLAGS) lib.o prog.o -o binary
lib.o: lib.c
$(CC) $(CFLAGS) -c lib.c -o lib.o
prog.o: prog.c
$(CC) $(CFLAGS) -c prog.c -o prog.o
clean:
rm *.o $(OUTPUT)
Makefile variables (5/7)
CC = gcc
CFLAGS = -g -Wall
OUTPUT = binary
OBJFILES = lib.o prog.o
$(OUTPUT): $(OBJFILES)
$(CC) $(CFLAGS) $(OBJFILES) -o binary
lib.o: lib.c
$(CC) $(CFLAGS) -c lib.c -o lib.o
prog.o: prog.c
$(CC) $(CFLAGS) -c prog.c -o prog.o
clean:
rm *.o $(OUTPUT)
Makefile variables (6/7)
CC = gcc
CFLAGS = -g -Wall
OUTPUT = binary
OBJFILES = lib.o prog.o
$(OUTPUT): $(OBJFILES)
$(CC) $(CFLAGS) $(OBJFILES) -o binary
lib.o: lib.c
$(CC) $(CFLAGS) -c lib.c -o lib.o
prog.o: prog.c
$(CC) $(CFLAGS) -c prog.c -o prog.o
clean:
rm *.o $(OUTPUT)
Makefile variables (7/7)
CC = gcc
CFLAGS = -g -Wall
OUTPUT = binary
OBJFILES = lib.o prog.o
$(OUTPUT): $(OBJFILES)
$(CC) $(CFLAGS) $(OBJFILES) -o binary
%.o: %.c
# $<: dependency (%.c)
# $@: target (%.o)
$(CC) $(CFLAGS) -c $< -o $@
clean:
rm *.o $(OUTPUT)
Simple Test Script (1/2)
% ./server 6667 &
% cat testfile.01 | ./testscript.py
% cat testfile.02 | ./testscript.py
% killall -9 server
Simple Test Script (2/2)
#/bin/sh
echo “Starting server on port 6667.”
./server 6667 &
SERVERPID = $!
echo “Running test files.”
cat testfile.01 | ./testscript.py
cat testfile.02 | ./testscript.py
echo “Killing server process.”
kill $(SERVERPID)
Augmenting the Makefile for testing (1/2)
CC = gcc
CFLAGS = -g -Wall
OUTPUT = binary
OBJFILES = lib.o prog.o
all: $(OUTPUT)
$(OUTPUT): $(OBJFILES)
$(CC) $(CFLAGS) $(OBJFILES) -o binary
%.o: %.c
# $<: dependencies (%.c)
# $@: target (%.o)
$(CC) $(CFLAGS) -c $< -o $@
clean:
rm *.o $(OUTPUT)
Augmenting the Makefile for testing (2/2)
CC = gcc
CFLAGS = -g -Wall
OUTPUT = binary
OBJFILES = lib.o prog.o
all: $(OUTPUT)
test
$(OUTPUT): $(OBJFILES)
$(CC) $(CFLAGS) $(OBJFILES) -o binary
%.o: %.c
# $<: dependencies (%.c)
# $@: target (%.o)
$(CC) $(CFLAGS) -c $< -o $@
test: $(OUTPUT)
sh ./testscript.sh
clean:
rm *.o $(OUTPUT)
Using the Makefile
% make
% make test
% make clean
• Know more
Google
– “makefile example”
– “makefile template”
– “make tutorial”
– Chapter 3 of Dave Andersen’s notes
Extra: useful Unix commands (1/2)
• find “func_name” in files
% grep -r func_name .
• replace “bad_func_name” to “good_func_n
ame”
% sed -e “s/bad_func_name/good_func_name/g”\
prog.c > prog.c.new
Extra: useful Unix commands (2/2)
• find a file named “prog.c”
% find -name prog.c
• download files from the Internet
% wget http://address/to/file.tar.gz
• untar and unzip a file
% tar xzvf file.tar.gz
Assignment




You have access to svn repository at
https://ina.kaist.ac.kr/svn/<student id>
However, we will change your password (to some unknown integer).
Luckily, we made a server that gives you a hint.
When your client asks whether the username and the password match,
the server can tell you whether the password provided is less than,
equal to, or greater than your real password.
Assignment I

Protocol is defined in the document. (See course web page).
Query
Server (provided by the instructor)
Client
Response


Part I: Assume the network is reliable. (due 9/23)
Part II: The server may drop a packet. (due 9/27)