Transcript chapter1

Client/Server Networking
Protocol Stack Summary


“turtles all the way down”
each layer uses the services of the layer below and provides a
service to the layer above

Python lets you work at the layer of your choice

programs are “cleaner” the higher the layer you use

layers work by “hiding” the layers below behind function calls
What lies under Socket?
TCP/UDP
Internet Protocol Stack
IP
“link layer”
Networking




About “sharing” resources.
Compare to sharing of disk, IO devices, etc done by programs
running on a computer
Computer Example: OS is the master controller
Network Example: Each participant “plays by the rules” but no
“controller”
Networking

All network cards share the same ethernet cable

all wireless transmitters share the same frequency channels

fundamental unit of sharing is “packet”

individual packets carry addressing info sufficient to arrive at
final destination.
Addressing

two layers: one hop at a time and end-to-end

single hop addressing performed by link layer

end-to-end addressing is IP

process addressing is TCP or UDP
IP Addresses

a.b.c.d: 0 <= a,b,c,d <= 255

New Paltz: 137.140.*.*

localhost: 127.0.0.1

private networks: 10.*.*.*, 172.16-31.*.*, 192.168.*.*
Domain Name Service

DNS converts host names into host IP addresses.

corresponds to directory assistance
address = socket.gethostbyname(name);
How DNS Works




gethostbyname() first looks in /etc/hosts
if this fails then it looks in /etc/resolv.conf for the address of
“directory assistance”, also called a DNS Server.
sends the request to this address
Observation: If your DNS server is down, you won't get
anywhere on the Internet.
Routing



Each time a packet arrives at a new node a decision must be
made at that node as to where to send the packet next.
Guiding principle of routing on the Internet is that each time a
packet “hops” from one node to another it is always one hop
closer to its final destination.
Exercise: Difference between host and node.
Lots of Reading



The classic text is TCP/IP Illustrated: Vol I by Richard Stevens.
PDF file available on the web at books.google.com among
other places
We will concentrate on Chapters 1-4, 9, 11, 14, 17-19.
EXAMPLE:

Given:
207 N. Defiance St, Archbold, OH

Find longitude and latitude
Protocol Stack:
GoogleMaps
?????
TCP
IP
Ethernet
GoogleMaps

googlemaps library (3rd party) uses

urllib, uses

httplib, uses

Socket, uses

TCP, IP,
Ethernet
GoogleMaps
URL
protocol stack
inside the actual
program itself
HTTP
Socket
TCP, IP and
Ethernet make up
the OS part of the
protocol stack
APIs vs Sockets:

well-tested

written by experts

common practice to use them

we still need to understand Sockets to
appreciate things that depend upon them
Wireshark:

lets you look at packets crossing the wire

needs root permissions

easy to filter out unneeded traffic

I saved some traffic and you can view it with Wireshark (see
course web page).
Highest Level API Example:

Fetch a JSON document without realizing it:
#!/usr/bin/env python
# Foundations of Python Network Programming - Chapter 1 - search1.py
# Not even clear you are using a web service
from googlemaps import GoogleMaps
address = '207 N. Defiance St, Archbold, OH'
print GoogleMaps().address_to_latlng(address)
GET Syntax:
#!/usr/bin/env python
# Foundations of Python Network Programming - Chapter 1 - search2.py
# HTML-level abstraction
import urllib, urllib2
try:
import json
except ImportError: # for Python 2.5
import simplejson as json
params = {'q': '207 N. Defiance St, Archbold, OH',
'output': 'json', 'oe': 'utf8'}
url = 'http://maps.google.com/maps/geo?' + urllib.urlencode(params)
rawreply = urllib2.urlopen(url).read()
reply = json.loads(rawreply)
print reply['Placemark'][0]['Point']['coordinates'][:-1]
GET Syntax:
#!/usr/bin/env python
# Foundations of Python Network Programming - Chapter 1 - search3.py
# HTTP level abstraction
import httplib
try:
import json
# json built in with Python 2.6
except ImportError: # for Python 2.5
import simplejson as json
path = ('/maps/geo?q=207+N.+Defiance+St%2C+Archbold%2C+OH'
'&output=json&oe=utf8')
connection = httplib.HTTPConnection('maps.google.com')
connection.request('GET', path)
rawreply = connection.getresponse().read()
reply = json.loads(rawreply)
print reply['Placemark'][0]['Point']['coordinates'][:-1]
GET Syntax:
#!/usr/bin/env python
# Foundations of Python Network Programming - Chapter 1 - search4.py
import socket
sock = socket.socket()
# OS functionality
sock.connect(('maps.google.com', 80))
sock.sendall(
'GET /maps/geo?q=207+N.+Defiance+St%2C+Archbold%2C+OH'
'&output=json&oe=utf8&sensor=false HTTP/1.1\r\n'
'Host: maps.google.com:80\r\n'
'User-Agent: search4.py\r\n'
'Connection: close\r\n'
'\r\n')
rawreply = sock.recv(4096)
print rawreply
GET Syntax:
# search4.py output
HTTP/1.1 200 OK
Content-Type: text/javascript; charset=UTF-8
Vary: Accept-Language
Date: Wed, 21 Jul 2010 16:10:38 GMT
Server: mafe
Cache-Control: private, x-gzip-ok=""
X-XSS-Protection: 1; mode=block
Connection: close
data transmitted by
web server
{
"name": "207 N. Defiance St, Archbold, OH",
"Status": {
"code": 200,
"request": "geocode"
},
"Placemark": [ {
...
"Point": {
"coordinates": [ -84.3063479, 41.5228242, 0 ]
}
}]
}
data read into
program variable
Things We've Seen:

protocols stacked on top of one another

higher level protocols using services of lower levels



programs get more specific and harder to maintain the lower
down you go
the idea behind high-level protocols is precisely to hide lower
levels
there's a whole lot going on below Socket.
The Stack:

Fundamental unit of shared information is the packet.

Typical packet structure:
ethernet
header
IP
header
TCP/UDP
header

Transmitted as a single unit (but serially)

Routing is generally at the packet level


program
data
Things packets contain: data, addresses, layering,
sequencing, protocol bytes, checksums
ethernet packets are called frames.
Ethernet:

14-byte header

addresses: two 6-byte addresses – source and destination

type: 2 bytes – 0800 == IP datagram

the two network cards involved can process the header
without using the CPU, RAM, etc.

cable length (100m) and MTU

CSMA/CD

Some of the details:
http://serverfault.com/questions/422158/
what-is-the-in-the-wire-size-of-a-ethernet-frame-1518-or-1542
IP Addresses:

32 bits: a.b.c.d

network address – n bits; host id – (32-n) bits

some times the network part has a subnet component; some
times the subnet component is carved out of the hostID bits

a.b == 137.140 == New Paltz network address

a.b.c == 137.140.8 == CS subnet at New Paltz

the part of the network address that is not subnet identifies an
organization like New Paltz.
IP Address Classes:
IP Special Addresses:

127.*.*.*: local to the current machine

10.*.*.*, 172.16-31.*.*, 192.168.*.*: private subnets.

none of these address found on the larger Internet.
IP Routing:


Guiding principle: after each hop you are one step closer to
your destination
typical local routing table contains a default entry pointing to
the Internet together with one entry for each local subnet the
host belongs to.
[pletcha@archimedes PPT]$ netstat -nr
Kernel IP routing table
Destination
Gateway
Genmask
Flags
0.0.0.0
192.168.1.1 0.0.0.0
UG
192.168.1.0
0.0.0.0
255.255.255.0 U
192.168.122.0 0.0.0.0
255.255.255.0 U
Iface
wlan0
wlan0
virbr0
IP Routing Next Hop Algorithm:




Search Destination column of table entries with H-flag set
which is an exact match to Destination IP in packet
If found and Flag is G or H then Gateway is next hop;
otherwise Destination IP is next hop.
If not found then calculate Dest IP && Genmask for each entry
that is not the default. If Dest IP && Genmask == Destination
column entry then if Flag is G or H then Gateway is next hop;
otherwise Destination IP is next hop.
Otherwise use the default entry. Flag is almost always G so
Gateway is next hop IP.
IP Routing Next Hop Algorithm:



Once you have the next hop IP you need to determine the
next hop ethernet.
The Address Resolution Protocol (ARP) converts the next hop
IP into a next hop ethernet. More recently replaced by the ip
neigh command
Exercise: Read up on ARP in TCP/IP Illustrated.
[pletcha@archimedes PPT]$ ip neigh
137.140.39.139 dev enp0s25 lladdr 00:c0:17:c2:14:f3 STALE
137.140.193.250 dev wlp3s0 lladdr 00:1f:29:07:e4:6a STALE
137.140.39.250 dev enp0s25 lladdr 00:21:a0:39:65:00 DELAY
ARP Example
●
From my laptop (137.140.8.104) I try to locate joyous
(137.140.8.101)
[pletcha@archimedes PPT]$ ping 137.140.8.101
PING 137.140.8.101 (137.140.8.101) 56(84) bytes of data.
64 bytes from 137.140.8.101: icmp_seq=1 ttl=64 time=0.266 ms
^C
●
Because of my routing table I know it is locally connected so
137.140.8.101 is “next hop”.
[pletcha@archimedes PPT]$ netstat -nr
Kernel IP routing table
Destination Gateway
Genmask
Flags MSS Window irtt Iface
0.0.0.0
137.140.8.250 0.0.0.0
UG
00
0 enp0s25
137.140.8.0 0.0.0.0
255.255.255.0 U
00
0 enp0s25
137.140.192.0 0.0.0.0
255.255.254.0 U
00
0 wlp3s0
ARP Request
ARP Reply
Packet Fragmentation:

The Internet Protocol Suite supports 64k packets but specific
IP networks support much smaller packets.

Ethernet networks support 1500 byte packets.

IP headers contain a Don't Fragment (DF) flag, set by sender.
–
DF not set, then a router can fragment a packet too
large to be forwarded on a particular interface.
–
DF set, router sends an ICMP message to original
sender so sender can fragment the original message
and try again.

UDP: DF unset by OS

TCP: DF set by OS
Packet Fragmentation (continued):

Each subnet has an MTU – Maximum Transmission Unit.

Path MTU = min hop MTU over all hops in a path

DSL providers make MTU = 1492.
–

Initially many service providers used MTU = 1500 and
disabled ICMP so never knew their “large” traffic was
being dropped.
TCP/IP Illustrated discusses how fragmentation actually
happens (Read Section 11.5).
TCP/IP Illustrated:

Pages to Look at
25, 38, 43, 44, 48, 58, 61, 63