Transcript Network - Introduction to Computer System

Networking Programming (I)
1
A Client-Server Transaction
1. Client sends request
Client
process
4. Client
handles
response
Server
process
3. Server sends response
Resource
2. Server
handles
request
Note: clients and servers are processes running on hosts
(can be the same or different hosts).
2
Hardware Organization of a Network Host
register file
CPU chip
ALU
system bus
memory bus
main
memory
I/O
bridge
MI
Expansion slots
I/O bus
USB
controller
mouse keyboard
graphics
adapter
disk
controller
monitor
disk
network
adapter
3
network
Logical Structure of an internet
host
router
host
router
router
router
router
router
• Ad hoc interconnection of networks
– No particular topology
– Vastly different router & link capacities
• Send packets from source to destination by hopping
through networks
– Router forms bridge from one network to another
– Different packets may take different routes
4
Transferring data over an internet
LAN1
(1)
client
server
protocol
software
data
PH
data
PH
LAN2
(8)
data
(7)
data
PH
FH2
(6)
data
PH
FH2
protocol
software
FH1
LAN1 frame
(3)
Host B
data
internet packet
(2)
Host A
LAN1
adapter
LAN2
adapter
Router
FH1
LAN1
adapter
LAN2
adapter
LAN2 frame
(4)
PH: Internet packet header
FH: LAN frame header
data
PH
data
FH1
protocol
software
PH
FH2
(5)
5
Global IP Internet
• Based on the TCP/IP protocol family.
– UDP (Unreliable Datagram Protocol)
• uses IP to provide unreliable datagram delivery from
process-to-process.
– TCP (Transmission Control Protocol)
• uses IP to provide reliable byte streams (like files) from
process-to-process.
• Accessed via a mix of Unix file I/O and
functions from the Berkeley sockets
interface.
6
Hardware and software organization
of an Internet application
Internet client host
Internet server host
Client
User code
Server
TCP/IP
Kernel code
TCP/IP
Sockets interface
(system calls)
Hardware interface
(interrupts)
Network
adapter
Hardware
and firmware
Network
adapter
Global IP Internet
7
Programmer’s view of the Internet
• Hosts are mapped to a set of 32-bit IP
addresses.
– 202.120.225.9
• The set of IP addresses is mapped to a set of
identifiers called Internet domain names.
– 202.120.225.9 is mapped to bbs.fudan.edu.cn
• A process on one host communicates with a
process on another host over a connection.
8
Dotted decimal notation
• By convention, each by in a 32-bit IP address
is represented by its decimal value and
separated by a period
• IP address 0x8002C2F2 = 128.2.194.242
9
Dotted decimal notation
• Functions for converting between binary IP
addresses and dotted decimal strings:
– inet_aton: converts a dotted decimal string to an
IP address in network byte order.
– inet_ntoa: converts an IP address in network by
order to its corresponding dotted decimal string.
– “n” denotes network representation. “a” denotes
application representation.
10
IP Addresses
• 32-bit IP addresses are stored in an IP
address struct
– IP addresses are always stored in memory in
network byte order (big-endian byte order)
/* Internet address structure */
struct in_addr {
unsigned int s_addr; /* network byte order (big-endian) */
};
11
IP Addresses
• Handy network byte-order functions:
– htonl: convert long int from host to network byte
order.
– htons: convert short int from host to network byte
order.
– ntohl: convert long int from network to host byte
order.
– ntohs: convert short int from network to host byte
order.
12
IP Address Structure
• IP (V4) Address space divided into classes:
Class A
0123
8
0 Net ID
Class B
10
Class C
110
16
24
Host ID
Net ID
Host ID
Net ID
Class D 1 1 1 0
Multicast address
Class E
Reserved for experiments
1111
31
Host ID
• Network ID Written in form w.x.y.z/n
– n = number of bits in host address
– E.g., CMU written as 128.2.0.0/16
• Class B address
• Unrouted (private) IP addresses:
10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
13
Internet Domain Names
unnamed root
mil
mit
edu
cmu
gov
berkeley
cs
ece
first-level domain names
com
amazon
www
second-level domain names
third-level domain names
208.216.181.15
cmcl
pdl
kittyhawk
imperial
128.2.194.242
128.2.189.40
14
Domain Naming System (DNS)
• The Internet maintains a mapping between IP
addresses and domain names in a distributed
database called DNS.
– Conceptually, we can think of the DNS database as
being millions of host entry structures:
15
Domain Naming System (DNS)
/* DNS host entry structure */
struct hostent {
/* official domain name of host */
char *h_name;
/* null-terminated array of domain names */
char **h_aliases;
/* host address type (AF_INET) */
int
h_addrtype;
/* length of an address, in bytes */
int
h_length;
/* null-terminated array of in_addr structs*/
char **h_addr_list;
};
16
Domain Naming System (DNS)
• Functions for retrieving host entries from
DNS:
– gethostbyname: query key is a DNS
domain name
– gethostbyaddr: query key is a an IP
address
17
hostname: a program that queries DNS
int main(int argc, char **argv) { /* argv[1] is a domain name
char **pp;
* or dotted decimal IP addr */
struct in_addr addr;
struct hostent *hostp;
if (inet_aton(argv[1], &addr) != 0)
hostp = Gethostbyaddr((const char *)&addr, sizeof(addr),
AF_INET);
else
hostp = Gethostbyname(argv[1]);
printf("official hostname: %s\n", hostp->h_name);
for (pp = hostp->h_aliases; *pp != NULL; pp++)
printf("alias: %s\n", *pp);
for (pp = hostp->h_addr_list; *pp != NULL; pp++) {
addr.s_addr = *((unsigned int *)*pp);
printf("address: %s\n", inet_ntoa(addr));
}
}
18
Networking Programming
19
Internet connections
• Clients and servers communicate by sending
streams of bytes of connections .
– point-to-point, full-duplex, and reliable
• A connection is uniquely identified by the
socket addresses of its endpoints (socket
pair)
– (cliaddr:cliport, servaddr:servport)
20
Internet connections
• A port is a 16-bit integer that identifies a
process:
– Ephemeral port : assigned automatically on client
when client makes a connection request
– Well-known port : associated with some service
provided by a server (e.g., port 80 is associated
with Web servers)
21
Putting it all Together:
Anatomy of an Internet Connection
Client socket address
128.2.194.242:51213
Client
Server socket address
208.216.181.15:80
Connection socket pair
(128.2.194.242:51213, 208.216.181.15:80)
Client host address
128.2.194.242
51213 is an ephemeral port
allocated by the kernel
Server
(port 80)
Server host address
208.216.181.15
80 is a well-known port
associated with Web servers
22
What is a socket?
• To the kernel, a socket is an endpoint of
communication
• To an application, a socket is a descriptor
that lets an application read/write from/to
the network.
Remember: All Unix I/O devices, including networks, are
modeled as files
23
What is a socket?
• Clients and servers communicate with each
other by reading from and writing to socket
descriptors
– Using regular Unix read and write I/O functions
Client
clientfd
Server
serverfd
24
What is a socket?
• The main difference between file I/O and
socket I/O is how the application “opens” the
socket descriptors
25
Overview of the Sockets Interface
Client
Server
socket
socket
bind
open_listenfd
open_clientfd
listen
connect
Client /
Server
Session
Connection
request
rio_writen
rio_readlineb
rio_readlineb
close
accept
rio_writen
EOF
Await connection
request from
next client
rio_readlineb
26
close
Key data structures
•Internet-style sockets are characterized by
– a 32-bit IP address and a port
•Defined in /usr/include/netinet/in.h
/* Internet address */
struct in_addr {
unsigned int s_addr;
/* 32-bit IP address */
};
27
Key data structures
• Defined in /usr/include/netdb.h
/* Domain Name Service (DNS) host entry */
struct hostent {
char
*h_name; /* official name of host */
char
**h_aliases; /* alias list */
int
h_addrtype;
/* host address type */
int
h_length;
/* length of address */
char
**h_addr_list;/*list of addresses */
}
28
Socket Address Structures
• Internet-specific socket address:
– Must cast (sockaddr_in *) to (sockaddr *) for
connect, bind, and accept
struct sockaddr_in {
unsigned short sin_family;
unsigned short sin_port;
struct in_addr sin_addr;
unsigned char
sin_zero[8];
};
sin_port
AF_INET
/*
/*
/*
/*
address family (always AF_INET) */
port num in network byte order */
IP addr in network byte order */
pad to sizeof(struct sockaddr) */
sin_addr
0
0
0
0
0
0
0
sa_family
sin_family
Family Specific
29
0
Overview of the Sockets Interface
Client
Server
socket
socket
bind
open_listenfd
open_clientfd
listen
connect
Connection
request
accept
30
Echo Client: open_clientfd
int open_clientfd(char *hostname, int port) { This function opens a connection
from the client to the server at
int clientfd;
hostname:port
struct hostent *hp;
struct sockaddr_in serveraddr;
if ((clientfd = socket(AF_INET, SOCK_STREAM, 0)) < 0)
Create
return -1; /* check errno for cause of error */
socket
/* Fill in the server's IP address and port */
if ((hp = gethostbyname(hostname)) == NULL)
return -2; /* check h_errno for cause of error */
bzero((char *) &serveraddr, sizeof(serveraddr));
serveraddr.sin_family = AF_INET;
bcopy((char *)hp->h_addr_list[0],
(char *)&serveraddr.sin_addr.s_addr, hp->h_length);
serveraddr.sin_port = htons(port);
/* Establish a connection with the server */
if (connect(clientfd, (SA *) &serveraddr,
sizeof(serveraddr)) < 0)
return -1;
return clientfd;
}
Create
address
Establish
connection
31
Echo Client: open_clientfd (socket)
• creates a socket descriptor on the client
– Just allocates & initializes some internal data structures
– AF_INET: indicates that the socket is associated with Internet
protocols
– SOCK_STREAM: selects a reliable byte stream connection
• provided by TCP
int clientfd;
/* socket descriptor */
if ((clientfd = socket(AF_INET, SOCK_STREAM, 0)) < 0)
return -1; /* check errno for cause of error */
... <more>
32
Echo client: open_clientfd()(connect)
int clientfd;
/* socket descriptor */
struct sockaddr_in serveraddr;
/* server address */
typedef struct sockaddr SA;
/* generic sockaddr */
...
/* Establish a connection with the server */
if (connect(clientfd, (SA *)&serveraddr,
sizeof(serveraddr))< 0)
return -1;
return clientfd;
}
33
Echo client: open_clientfd()(connect)
• Then the client creates a connection with the
server
– The client process suspends (blocks) until the
connection is created with the server.
– At this point the client is ready to begin
exchanging messages with the server via Unix I/O
calls on the descriptor clientfd.
34
Echo server: open_listenfd()
int open_listenfd(int port)
{
int listenfd, optval=1;
struct sockaddr_in serveraddr;
/* Create a socket descriptor */
if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) < 0)
return -1;
/* Eliminates "Address already in use" error from bind. */
if (setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR,
(const void *)&optval , sizeof(int)) < 0)
return -1;
... <more>
35
Echo server: open_listenfd() (cont)
...
/* Listenfd will be an endpoint for all requests to port
on any IP address for this host */
bzero((char *) &serveraddr, sizeof(serveraddr));
serveraddr.sin_family = AF_INET;
serveraddr.sin_addr.s_addr = htonl(INADDR_ANY);
serveraddr.sin_port = htons((unsigned short)port);
if (bind(listenfd, (SA *)&serveraddr, sizeof(serveraddr)) < 0)
return -1;
/* Make it a listening socket ready to accept
connection requests */
if (listen(listenfd, LISTENQ) < 0)
return -1;
return listenfd;
}
36
Echo server: open_listenfd()
• Initialize socket with server port number
• Accept connection from any IP address
struct sockaddr_in serveraddr; /* server's socket addr */
...
/* listenfd will be an endpoint for all requests to port
on any IP address for this host */
bzero((char *) &serveraddr, sizeof(serveraddr));
serveraddr.sin_family = AF_INET;
serveraddr.sin_port = htons((unsigned short)port);
serveraddr.sin_addr.s_addr = htonl(INADDR_ANY);
37
Echo server: open_listenfd() (bind)
• bind() associates the socket with the socket
address we just created.
int listenfd;
/* listening socket */
struct sockaddr_in serveraddr; /* server’s socket addr */
...
/* listenfd will be an endpoint for all requests to port
on any IP address for this host */
if (bind(listenfd, (SA *)&serveraddr, sizeof(serveraddr)) < 0)
return -1;
38
Echo server: open_listenfd (listen)
• listen() indicates that this socket will accept
connection (connect) requests from clients.
int listenfd; /* listening socket */
...
/* Make it a listening socket ready to accept connection requests */
if (listen(listenfd, LISTENQ) < 0)
return -1;
return listenfd;
}
• LISTENQ is constant indicating how many pending
requests allowed
• We’re finally ready to enter the main server loop that
accepts and processes client connection requests.
39
Echo server: main loop
• The server loops endlessly, waiting for
connection requests, then reading input from
the client, and echoing the input back to the
client.
main() {
/* create and configure the listening socket */
while(1) {
/* Accept(): wait for a connection request */
/* echo(): read and echo input line from client */
/* Close(): close the connection */
}
}
40
Echo server: accept()
• accept() returns a connected socket descriptor
(connfd) with the same properties as the
listening descriptor (listenfd)
– Returns when connection between client and server
is created and ready for I/O transfers.
– All I/O with the client will be done via the
connected socket.
• accept()also fills in client’s address.
41
Echo Server: accept Illustrated
listenfd(3)
Client
Server
clientfd
Connection
request
Client
listenfd(3)
1. Server blocks in accept,
waiting for connection request
on listening descriptor
listenfd
2. Client makes connection request by
calling and blocking in connect
Server
clientfd
listenfd(3)
Client
clientfd
Server
connfd(4)
3. Server returns connfd from
accept. Client returns from connect.
Connection is now established between
clientfd and connfd
42
Connected vs. Listening Descriptors
• Listening descriptor
– End point for client connection requests
– Created once and exists for lifetime of the server
• Connected descriptor
– End point of the connection between client and server
– A new descriptor is created each time the server
accepts a connection request from a client
– Exists only as long as it takes to service client
• Why the distinction?
– Allows for concurrent servers that can communicate
over many client connections simultaneously
• E.g., Each time we receive a new request, we fork a child to
43
handle the request
Networking Programming
--Web Server(I)
44
Web servers
• Clients and servers communicate using the
HyperText Transfer Protocol (HTTP)
– client and server establish TCP connection
– Client requests content
– Server responds with requested content
– client and server close connection (eventually)
• Current version is HTTP/1.1
– RFC 2616, June, 1999.
45
Web servers
Web
client
(browser)
HTTP request
Web
server
HTTP response
(content)
HTTP
TCP
IP
Web content
Streams
Datagrams
http://www.w3.org/Protocols/rfc2616/rfc2616.html
46
Web content
• Web servers return content to clients
– content: a sequence of bytes with an associated
MIME (Multipurpose Internet Mail Extensions)
type
47
Web content
• Example MIME types
– text/html
HTML page
– text/plain
Unformatted text
– application/postscript
Postcript document
– image/gif
Binary image encoded in GIF format
– image/jpg
Binary image encoded in JPG format
48
Static and dynamic content
• The content returned in HTTP responses can be
either static or dynamic
– Static content: content stored in files and retrieved in
response to an HTTP request
• Examples: HTML files, images, audio clips.
• Request identifies content file
– Dynamic content: content produced on-the-fly in
response to an HTTP request
• Example: content produced by a program executed by the
server on behalf of the client.
• Request identifies file containing executable code
Bottom line: All Web content is associated with a file that is
managed by the server.
49
URLs
• Each file managed by a server has a unique
name called a URL (Universal Resource
Locator)
• URLs for static content:
– http://www.cs.cmu.edu:80/index.html
– http://www.cs.cmu.edu/index.html
– http://www.cs.cmu.edu
• identifies a file called index.html, managed by a Web
server at www.cs.cmu.edu that is listening on port 80.
50
URLs
• URLs for dynamic content:
– http://www.cs.cmu.edu:8000/cgibin/adder?15000&213
• identifies an executable file called adder, managed by a
Web server at www.cs.cmu.edu that is listening on port
8000, that should be called with two argument strings:
15000 and 213.
51
Anatomy of an HTTP Transaction
unix> telnet www.cmu.edu 80
Trying 128.2.10.162...
Connected to www.cmu.edu.
Escape character is '^]'.
GET /index.shtml HTTP/1.1
HOST: www.cmu.edu
Client: open connection to server
Telnet prints 3 lines to the terminal
Client: request line
Client: required HTTP/1.1 HOST header
Client: empty line terminates headers.
HTTP/1.1 200 OK
Server: responds with web page
Date: Fri, 29 Oct 2010 19:41:08 GMT
Server: Apache/1.3.39 (Unix) mod_pubcookie/3.3.3 ...
Transfer-Encoding: chunked
Content-Type: text/html
...
Lots of stuff
Connection closed by foreign host. Server: closes connection
unix>
Client: closes connection and terminates
52
HTTP Requests
• HTTP request is a request line, followed by
zero or more request headers
• Request line: <method> <uri> <version>
– <version> is HTTP version of request (HTTP/1.0
or HTTP/1.1)
– <uri> is typically a URL for proxies, a suffix for
servers.
• A URL is a type of URI (Uniform Resource Identifier)
• See http://www.ietf.org/rfc/rfc2396.txt
– <method> is either GET, POST, OPTIONS,
HEAD, PUT, DELETE, or TRACE.
53
HTTP Requests (cont)
• HTTP methods:
– GET: Retrieve static or dynamic content
• Arguments for dynamic content are in URI
• Workhorse method (99% of requests)
– POST: Retrieve dynamic content
• Arguments for dynamic content are in the request body
–
–
–
–
–
OPTIONS: Get server or file attributes
HEAD: Like GET but no data in response body
PUT: Write a file to the server!
DELETE: Delete a file on the server!
TRACE: Echo request in response body
• Useful for debugging.
• Request headers: <header name>: <header data>
– Provide additional information to the server.
54
HTTP Responses
• HTTP response is a response line followed by zero or
more response headers.
• Response line:
•
<version> <status code> <status msg>
– <version> is HTTP version of the response.
– <status code> is numeric status.
– <status msg> is corresponding English text.
•
•
•
•
200
301
403
404
OK
Moved
Forbidden
Not found
Request was handled without error
Provide alternate URL
Server lacks permission to access file
Server couldn’t find the file.
• Response headers: <header name>: <header data>
– Provide additional information about response
– Content-Type: MIME type of content in response body.
55
– Content-Length: Length of content in response body.