Transcript www.cs.newpaltz.edu

Chapter II
net_py
Reliable or Unreliable


Do you need service that keeps packets in order and
guarantees delivery?
Are your packets “individual” and so can arrive in any order or
possibly even not at all?
Example:
Suppose you plan to send a single packet and get a single reply?

In the above example, you would only need to worry about
non-delivery (and hence non-reply).
net_py
UDP

Short, self-contained requests and responses.

Real-time traffic like voice.

Author says it is not used often but that is not to say it is not
useful.

A single server port can receive packets from millions of
distinct clients over time with no additional memory
allocation beyond original setup.

congested network routers tend to be more sympathetic to
UDP traffic compared to TCP traffic since they know that
the latter will be retransmitted if dropped by the router.

You can add your own “99% reliability” more cheaply
(performance) than using TCP's 100% reliability.

easier to use
net_py
Addresses and Ports
running program
file descriptor
socket memory allocation
port number
UDP
IP
Ethernet
net_py
Addresses and Ports



Ports allow multiple programs to use the same TCP/IP stack
Packets come in and go up the same stack. They are
“demultiplexed” at the top of the stack by port number to
different programs.
The pair (IP address:port number) is called a socket and
identifies a program connected to the internet, running on
some machine.
Example:
www.mysite.com:8080

Every packet sent on the internet contains a quadruple
(sourceIP: source port, destIP: dest port)
that identifies the connection.
net_py
Port Ranges

Well-known: 0-1023

Registered: 1024-4915: used by large companies

The Rest: above, used by us

How to find a well-known or registered port number
>>> import socket
>>> socket.getservbyname('domain')

Check out the IANA (www.iana.org)
net_py
Python Virtual Environment
●
●
●
●
It is a good idea to create your own virtual environment for
the various python programming projects found in this
course.
This gives you an easy way to install special python
packages just for your own use and leave the main python
install on your computer alone.
This is a useful skill for future python development you might
do.
Here is an on-line tutorial; read it carefully and follow the
instructions.
http://dabapps.com/blog/introduction-to-pip-and-virtualenv-python/
http://virtualenvwrapper.readthedocs.org/en/latest/
net_py
Using virtualenvwrapper
●
●
●
The virtualenv tutorial suggests that inside each project
directory (say Project1) there should be a project-specific
virtual environment (Project1/env).
There is an alternative approach suggested by
virtualenvwrapper – an extension of virtualenv.
When using using virtualenvwrapper your file hierarchy looks
like:
Projects:
ArcGIS BenchMark GeoIP Memcached QGIS Rabbit
.virtualenvs:
ArcGIS
hook.log.1
postmkvirtualenv premkvirtualenv
Benchmark
initialize
postrmproject prermproject
BenchMark
Memcached
postrmvirtualenv prermvirtualenv
GeoIP
postactivate preactivate
QGIS
get_env_details postdeactivate predeactivate Rabbit
hook.log
postmkproject premkproject
README
●
All the executable code in .virtualenvs is from the wrapper
package.
net_py
Sockets




Python makes calls to the lower level operating system-level
calls that implement the networking functionality
The python interface is slightly OO
Python's contact to these OS calls is through a program
construct called a socket
Sockets are “files” just like everything else and are accessed
via a file descriptor, which in python you never see but in C it
is all you see.

Sockets are the file descriptor

We read and write to sockets the same way we do files
net_py
import socket, sys
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# AF_INET is a “family” type
# SOCK_DGRAM means UDP and not TCP
MAX = 65535
PORT = 1060
# packet size
# port used by our server
if sys.argv[1:] == ['server']: # are these two lists equal
s.bind(('127.0.0.1', PORT)) # specify your local socket address
# since you are a server
print 'Listening at', s.getsockname() # returns a list
while True:
data, address = s.recvfrom(MAX) # wait for traffic
print 'The client at', address, 'says', repr(data) # data.toString()
s.sendto('Your data was %d bytes' % len(data), address)
# send reply to original sender
net_py
elif sys.argv[1:] == ['client']:
print 'Address before sending:', s.getsockname()
# 0.0.0.0:0 means no port on any interface
s.sendto('This is my message', ('127.0.0.1', PORT))
print 'Address after sending', s.getsockname()
# bind is automatic, but only to a port number
# 0.0.0.0:34567 means port number 34567 on any interface
data, address = s.recvfrom(MAX) # overly promiscuous - see Chapter 2
# will accept a datagramfrom anyone and not just the server
print 'The server', address, 'says', repr(data)
else:
print >>sys.stderr, 'usage: udp_local.py server|client'
net_py
elif sys.argv[1:] == ['client']:
print 'Address before bind:', s.getsockname()
s.bind('',55000)
print 'Address after bind:', s.getsockname()
s.sendto('This is my message', ('127.0.0.1', PORT))
print 'Address after sending', s.getsockname()
# bind is automatic, but only to a port number
# 0.0.0.0:55000 means port number 55000 on any interface
data, address = s.recvfrom(MAX) # overly promiscuous - see Chapter 2
# will accept a datagramfrom anyone and not just the server
print 'The server', address, 'says', repr(data)
else:
print >>sys.stderr, 'usage: udp_local.py server|client'
net_py
Ports


Remote access to a socket is through its port number.
Access from the program that opens the socket is through the
file descriptor.
net_py
Socket




Author says “both IP address and port number start as all
zeros – a new socket is a blank slate”. This is not entirely true.
recv() vs recvfrom():
recv(): does not return sender address and typical of client
code because clients typically go to a single server and so
“know” where any data they receive comes from.
recvfrom() returns sender address and is typical of server
code because servers receive data from many clients and
typically need to reply to a client request with a packet sent
right back to the same client. The socket.sendto() function
takes the address returned by recvfrom().
net_py
Congestion:


If you timeout and resend and the problem is that the server
is down, you will add useless traffic to the network and cause
congestion that will slow everyone down.
Best answer is each time you timeout extend the timeout
interval so that eventually you are resending a packet only
once per hour or more.

exponential backoff: delay *= 2

What about giving up?
best to give up only if you need a timely answer or not at all

What about trying forever?
weather icon example
net_py
More Complications:

Suppose a successful request/reply exchange takes 200 ms
on average.
Client can use this information to set a minimum timeout
delay to be at least 200 ms?

What does the server do with duplicate requests?
1) Reply again
2) Don't bother to repeat reply (How does the server know a previous reply
got to the original sender?).
In both cases the server must keep track of request identifiers
so it knows what requests have already been received.

If client also keeps track of request IDs and whether or not
they have been replied to then the client can quietly drop
duplicate replies.
net_py
reply lost
no need to pass off
to higher level but
must keep data ID
to know it has already
been forwarded
optional
net_py
Reliable Code
#!/usr/bin/env python
# Foundations of Python Network Programming - Chapter 2 - udp_remote.py
# UDP client and server for talking over the network
import random, socket, sys
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
MAX = 65535
PORT = 1060
# usage: udp_remote.py server [ <interface> ]
if 2 <= len(sys.argv) <= 3 and sys.argv[1] == 'server':
# server side
interface = sys.argv[2] if len(sys.argv) > 2 else '' # two single quotes
s.bind((interface, PORT)) # '' is the same as '0.0.0.0' == all interfaces
print 'Listening at', s.getsockname()
while True:
data, address = s.recvfrom(MAX) # server always needs client address
if random.randint(0, 1): # flip a coin and reply only if heads
print 'The client at', address, 'says:', repr(data)
s.sendto('Your data was %d bytes' % len(data), address)
else: # tails
print 'Pretending to drop packet from', address
net_py
Reliable Code
# Usage: udp_remote.py client <host>
elif len(sys.argv) == 3 and sys.argv[1] == 'client':
hostname = sys.argv[2]
s.connect((hostname, PORT)) # no packets exchanged;
print 'Client socket name is', s.getsockname()
delay = 0.1 # initial retry delay
while True: # keep resending if no reply
s.send('This is another message')
print 'Waiting up to', delay, 'seconds for a reply'
s.settimeout(delay) # so recv() will block only so long
try:
data = s.recv(MAX) # blocking <delay> seconds
except socket.timeout: # timeout expired
delay *= 2 # double timeout delay time
if delay > 2.0: # time to stop all this nonsense
raise RuntimeError('I think the server is down')
else:
break # we are done after one success,
# and can stop looping
print 'The server says', repr(data)
net_py
Code Alternatives:
•
See homework
net_py
Connecting vs Implicit Connecting
• In Listing 2-1 we used sendto(msg,address) on the client and
an implicit binding happened when the first datagram was sent.
• In Listing 2-2 we used send(msg) so the binding had to be
done explicitly, and at the same time we indicate where the
message is to be sent
s.connect(remote_host,remote_PORT)
...
s.send(data)
check netstat -an | grep udp
at this point
• In the first situation we could send to various different servers
by modifying the address argument. In the latter we can only
send to the address we originally connected to.
net_py
How things look after s.sendto(msg,address):
[pletcha@archimedes ~]$ python
>>> import socket
>>> s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
>>> s.sendto('my message',('wyvern.cs.newpaltz.edu',50000))
10
>>> s.getsockname()
('0.0.0.0', 52011)
[pletcha@archimedes ~]$ netstat -an | grep udp
udp
0
0 0.0.0.0:55188
0.0.0.0:*
udp
0
0 0.0.0.0:47503
0.0.0.0:*
udp
0
0 0.0.0.0:52011
0.0.0.0:*
udp
0
0 0.0.0.0:60386
0.0.0.0:*
udp
0
0 192.168.122.1:53
0.0.0.0:*
net_py
anyone can send data to
my port 52011.
Sneaking into the conversation
[pletcha@archimedes ~]$ python
>>> import socket
>>> s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
>>> s.sendto('my message',('wyvern.cs.newpaltz.edu',50000))
10
>>> s.getsockname()
('0.0.0.0', 33897)
>>> data,address = s.recvfrom(4000)
>>> print data
use the bound port number
Fake reply
as your destination
[pletcha@archimedes ~]$ python
>>> import socket
>>> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
>>> s.sendto('Fake reply',('127.0.0.1',33697))
10
net_py
How things look after s.connect(address):
> python
>>> import socket
>>> s =socket.socket(socket.AF_NET,socket.SOCK_DGRAM)
>>> s.connect((wyvern.cs.newpaltz.edu,50000))
[pletcha@archimedes ~]$ netstat -an | grep udp
udp
0
0 0.0.0.0:55188
0.0.0.0:*
udp
0
0 137.140.8.104:51400 137.140.4.187:50000 ESTABLISHED
udp
0
0 0.0.0.0:47503
0.0.0.0:*
udp
0
0 0.0.0.0:60386
0.0.0.0:*
udp
0
0 192.168.122.1:53
0.0.0.0:*
no one can send data
to my port 51400 except
what I connected to
(I really didn't connect
since no data was sent)
net_py
Exercise
•
Repeat the “sneak into the conversation” example running the client software
of Listing 2-2 and see that your message never gets read by the application.
It is rejected by UDP
What happens is that Ethernet and IP recognize the sneaky packet as intended
for this machine. IP forwards the packet to UDP. UDP looks up the destination
port and sees that it will only accept data from (wyvern, 50000)
• Lesson to learn: If you want to use sendto() instead of
connect() followed by send(), then you can use recvfrom() and
look at each sender address to be sure it is an address you
recognize.
• Exercise: Think of a UDP application that would have you
sending data to several destinations so you could expect
answers back from them all.
net_py
Request IDs
• Sending a packet ID with every packet makes it easier to
identify replies and know what request they are replying to.
• (See homework)
• Packet IDs are some protection against spoofing. Client that
doesn't call connect() can be exposed to unexpected
(spoofing) traffic. If the unwelcome traffic doesn't know the
packet ID it is not possible to fake a response.
net_py
Binding to Interfaces
• We have used bind() to listen on 127.0.0.1 or on ' ', which
means all network interfaces.
• We can specify an interface if we know its IP address
(remember /sbin/ifconfig -a)
• Connecting to the College using VPN I have 4 network
interfaces on my laptop
net_py
4 Network Interfaces at once:
[pletcha@archimedes 02]$ ifconfig -a
cscotun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1399
inet 137.140.108.133 netmask 255.255.255.224 destination 137.140.108.1 ...
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0 ...
virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255 ...
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.124 netmask 255.255.255.0 broadcast 192.168.1.255 ...
net_py
Ports are bound to Interfaces
[pletcha@archimedes 02]$ cat all_interfaces.sh
#!/usr/bin/bash
./udp_remote.py server 127.0.0.1&
sleep 1; ./udp_remote.py server 137.140.108.133&
sleep 1; ./udp_remote.py server 192.168.122.1&
sleep 1; ./udp_remote.py server &
sleep 1; ./udp_remote.py server 192.168.1.124&
[pletcha@archimedes 02]$ ./all_interfaces.sh
Listening at ('127.0.0.1', 1060)
Listening at ('137.140.108.133', 1060)
Listening at ('192.168.122.1', 1060)
Traceback (most recent call last):
File "./udp_remote.py", line 13, in <module>
s.bind((interface, PORT))
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
[pletcha@archimedes 02]$ Listening at ('192.168.1.124', 1060)
net_py
local host vs remote host:
•
Local packets can arrive destined for 127.0.0.1 but they can
also arrive locally by using the machine IP address, say
192.168.122.1.
•
By binding to 127.0.0.1, external clients can not talk to you.
•
By binding to 192.168.122.1 both internal and external clients
can talk to you.
•
Lesson to Learn: Binding means specifying both a port
number and a network interface (or all interfaces), so the basic
data structure is not just a port but an (IP address, port) pair, in
other words, a socket.
net_py
Two Interface problem:
•
Suppose a machine has two external interfaces –
192.168.122.1 and 192.168.1.124.
•
If we open a socket at (192.168.1.124,1060) but try to send
data to (192.168.122.1, 1060) what will happen? Apparently it
depends on the OS.
•
On my laptop I get:
[pletcha@archimedes 02]$ ps -ef | grep remote
pletcha 28577 3450 0 18:07 pts/1 python ./udp_remote.py server 192.168.122.1
[pletcha@archimedes 02]$ ./udp_remote.py client 192.168.1.124
Client socket name is ('192.168.1.124', 48271)
Waiting up to 0.1 seconds for a reply
Traceback (most recent call last):
File "./udp_remote.py", line 33, in <module>
data = s.recv(MAX)
socket.error: [Errno 111] Connection refused
net_py
Fragmentation, ETC:
Read the last 4 pages of Chapter 2 yourselves.
net_py