Transcript DB, Event, Network programming

Python for database access and
Event driven programming in
Python
Database access in Python
• A set of tutorials, including access to
MySQL and PostgreSQL
http://python.about.com/od/pythonanddatabases/Connecting_to_Databases_With_Python.htm
• A brief introduction to SQLite3
– Lightweight disk-based database
– See http://docs.python.org/library/sqlite3.html
– NOTE: There are many details of database access.
Here we will do a brief example and establish the
ability to do the basic operations. See this website for
a complete list of features
SQL3lite
http://docs.python.org/2/library/sqlite3.html
import sqlite3
conn = sqlite3.connect('sample.db')
c=conn.cursor()
#Create table
c.execute('''CREATE TABLE stocks
(date text, trans text, symbol text, \
qty real, price real)''')
#Insert a row of data
c.execute("INSERT INTO stocks VALUES \
('2006-01-05','BUY','RHAT',100,35.14)")
#Save (commit) the changes
conn.commit()
#Close the connection.
#Be sure the changes are committed first.
conn.close()
New database (sample.db) created
with one row of data
import sqlite3
cnx = sqlite3.connect('sample.db')
c = cnx.cursor()
Access the data
t = ('RHAT',)
c.execute('SELECT * FROM stocks WHERE symbol=?', t)
print c.fetchone()
print '======================\n'
# Larger example that inserts many records at a time
purchases = [('2006-03-28','BUY', 'IBM', 1000, 45.00),
('2006-04-05','BUY', 'MSFT', 1000, 72.00),
('2006-04-06','SELL', 'IBM', 500, 53.00),
]
c.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases)
for row in c.execute('SELECT * FROM stocks ORDER BY \ price'):
print row
Output
(u'2006-01-05', u'BUY', u'RHAT', 100.0, 35.14)
======================
(u'2006-01-05',
(u'2006-03-28',
(u'2006-04-06',
(u'2006-04-05',
u'BUY', u'RHAT', 100.0, 35.14)
u'BUY', u'IBM', 1000.0, 45.0)
u'SELL', u'IBM', 500.0, 53.0)
u'BUY', u'MSFT', 1000.0, 72.0)
The first entry in the last set came from the
original values in the db, from the program that
created the db.
Another sqllite3 example
•
•
•
•
First, import sqlite3
Create an object that represents the database
Use standard sql
This example from Nakul Rathod
import sqlite3
def main():
db = sqlite3.connect('test.db')
# connect to the database or create the database if it doesn't exist
db.row_factory = sqlite3.Row
db.execute('drop table if exists test')
db.execute('create table test(t1 text, i1 int)')
db.execute('insert into test (t1, i1) values(?,?)', ('one', 1))
“””you can directly substitute the values in ?,? but variable names
allow for code re-usability”””
db.execute('insert into test (t1, i1) values(?,?)', ('two', 2))
db.execute('insert into test (t1, i1) values(?,?)', ('three', 3))
db.commit()
# without a commit no changes to the database will be recorded
cursor = db.execute('select * from test order by i1')
for row in cursor:
print row['i1'], row['t1']
# Each record is fully indexed by the field heading due to Row Factory
main()
Spot check
• Create a database using sqlite3
• Each row in the database contains the
following information (Make about 5
entries)
– ID (alphanumeric)
– Last name (alpha)
– First name (alpha)
– Hire year (int)
• Enter data in random order, then display all
in order of hire year
Event driven programming
• This is another way to think about flow
control in a program.
• An important consideration is parallelism
(or pseudo-parallelism)
– If there is only one processor, and we write the
program to support more than one event at
once, then the one processor gives slices of
time to each event.
– All modern computers have multi core
architecture – more than one processor
available. It is possible to have more than one
event process happening at the same time.
So far
• Most of our programs are basically
sequential, with excursions into
functions and classes for special
purposes
• There is a clearly defined sequence of
actions that the program will perform.
Event Driven Programs
• In event-driven programs, the sequence of
actions is not known in advance
• The program has the facility to respond to
external events and take appropriate
action.
• Now, programming means
– determining what events might happen
– recognizing each event
– taking appropriate action for the event
– waiting for the next
• sometimes accommodating simultaneous events
Error events
• An event may be a special case of normal
programs – such as an error event
• We have seen how to handle errors
– This is a form of interrupt handling
number = 0
while not 1 <= number <= 10:
try:
number= int(raw_input('Enter number from 1\
to 10: '))
if not 1 <= number <= 10:
print 'Your number must be from 1 to 10:'
except ValueError:
print 'That is not a valid integer.'
Servers, GUIs
• Some programs do nothing except
respond to events
– Servers listen for a request and respond
• GUI interfaces present a pallet to the
user
– The operating system responds to the
choices made by the user
Wait loops and threads
• One way to implement event handlers is
with an endless loop that checks for an
indication of the event, then responds
• If there is only one process running, the
event takes control and no other event can
be serviced until that one is complete
• Threads allow multiple flows of execution
simultaneously
– Same processor
– Multiple processors
Network programming
• Book introduces event driven
programming in the context of
drawing
– Fine, but let’s look at networks
• A brief introduction to networks to
start
– Peer – to – Peer
– Client-Server
Client-Server Computing
• Much of our computing environment is
client-server based
• Server:
– provides a service
– waits for a request and does something to
satisfy the request
• Client:
– The controlling side of the interaction
– Connects to a server
– Issues a request
Server waits for
requests
• Server must be connected to a network
• Server software waits for requests,
does some processing, sends
responses
Locating a server
• Web as our primary communication method
to servers
• URL (or URI) names the server where a
resource is provided
• Class web page is located at
http://www.csc.villanova.edu/~cassel8000/2012/8000Syllabus.html
• www.csc.villanova.edu: the domain
• rest: the path to the resource
• Domain Name Server (DNS) translates from
the domain name to the IP address of the
machine.
– Multi-level – edu, villanova, csc
Network connections
• Each machine on the network has a
location, called an IP address
– think of it like a phone number
• Many possible connection requests
– Within the machine, a port identifies the
specific communication line
• think of it like an extension in a phone system
– A client requests a connection on a particular
port to reach the service of interest
• Now, messages flow between the client process and
the desired server process
Protocols
• How does the client request the desired
service?
• A protocol is a standard set of messages
to exchange between cooperating
entities to achieve a desired result
– What protocols do you know?
Spot check
• You know protocols
• How do you introduce two people?
– Suppose person A is senior to person B
• How do you answer a phone call
– In a business context?
– If you know the person who is calling?
• How do you issue a formal invitation?
– How do you respond to a formal invitation?
• Can you think of any other standard message
exchanges?
– Anyone remember Lawrence Welk –
• And a 1 and a 2 and a 3 …
Network protocols
• There are many that we will not consider
• For Client-server computing
– Server listens on the port (end point of the
socket)
– Client directs a message to the known port
– Message content tells the server what service
is wanted
– Server responds with a standard message, in
an agreed format
• Each side understands only the pre-defined
messages!
Threads
• Multiple threads are necessary when a
program must appear to be doing more
than one sequence of steps at the same
time.
• Example: Waiting for more than one
kind of event to occur. Have the code
that waits for each in a separate thread.
Thread example
from threading import Thread
from time import sleep
class MyThread(Thread):
def run(self):
print 'Second thread begins.'
sleep(5)
print 'Second thread is done.’
secondThread=MyThread()
secondThread.start()
print 'The program is done.'
Second thread begins.
The program is done.
Second thread is done.
Python Socket class
Syntax
Semantics
s.connect(host, port) connects socket s to
remote host at specified
port.
s.send(data)
sends data to connected
remote host.
s.recv(maxNumChars)
receives a
maxNumChars from the
remote host.
Disconnects socket s
from remote host.
s.close( )
Simple client
• Get time from the time server at NIST
• No need to send anything because the
server only does one thing.
– As soon as the connection is requested, the
desired service is identified
from socket import socket
connection = socket()
connection.connect(('time.nist.gov',13))
print connection.recv(1024)
Output:
56259 12-11-28 19:30:19 00 0 0 118.4 UTC(NIST) *
Process the returned data
• We know the exact format of the data,
because that is part of the protocol
• We can now process that, format it, do
anything we like with it.
from socket import socket
connection = socket()
connection.connect(('time.nist.gov',13))
fields=connection.recv(1024).split()
date = fields[1]
time = fields[2]
print 'Date (YY-MM-DD) is %s, time is %s (UTC)’\
%(date,time)
Spot check
• Connect to the NIST time server
• Print out the current date and time in this
format:
It is now <time> on <date>
Note the time reported. Is it right? How
would you adjust if needed?
• How would you make the date be in the
form
– November <day>, 2012
– (or, if you prefer) <day> November 2012
recv Method
•
There are four possibilities for the
outcome of a recv command
1. Receive string of length equal to maximum
length specified. May have further data that
can be retrieved by another call to recv.
2. Receive less data than maximum. That is the
end of that batch of data.
3. No data was sent so recv waits indefinitely.
4. Empty string is received. Connection was
closed by sender or network was disrupted.
Terry Scott University of Northern Colorado
2007 Prentice Hall
28
Client side
• Nothing particularly event driven on the
client side
• This is where the event originates
• The event driven software is on the
server side
Writing a server –
event driven programming
• A server is a basic event-driven program
• Server
– listen on a particular port for incoming
connections
– create a socket to manage each connection
– follow the chosen protocol for communication
with the client
• From the server side
– send means from server to client
– recv means from client to server
A bit about network protocols
• References to TCP
– Transmission control protocol
– Connection-oriented protocol for
establishing and maintaining a connection
between communicating entities on the
Internet
• handles things like corrupted transmissions,
dropped calls, etc.
TCPServer class
• Hides all the details of establishing a
reliable connection and making sure all
the data is delivered correctly, and in
order.
• Customize by defining a class to handle
each connection
A simple server
• Echo server
– Receives a message from a client and sends
the same message back.
• Actually useful for testing that a connection is
working, and the round trip time for a message
# Example from Goldwasser book
from SocketServer import TCPServer, BaseRequestHandler
class EchoHandler(BaseRequestHandler):
def handle(self):
message = self.request.recv(1024)
self.request.send(message)
# may need to customize localhost and port for your machine
echoServer = TCPServer( ('localhost', 9128), EchoHandler)
echoServer.serve_forever()
The client side of echo
• Connect to the server
• Send the message
• Get the message back
from socket import socket
echo=socket()
echo.connect(('localhost',9128))
msg = raw_input("Enter a message to echo:")
echo.send(msg)
print echo.recv(1024)
Spot check
• Get the echo server running on your
machine.
• Make the echo client connect to the
server and echo a message.
Basic web server
• Similar to the echo server, except return
a requested file instead of echoing the
message
• Overall outline
– Accept the connection
– Receive the file request
– Search for the file in the path provided
– If not there, return error message (404!)
– If there, return the file
Web server
from SocketServer import TCPServer, BaseRequestHandler
Note: runs on local
host, not accessible
over the Internet
class WebHandler(BaseRequestHandler):
def handle(self):
command = self.request.recv(1024)
if command[:3] == 'GET':
pagename = command.split()[1][1:]
# remove leading '/' for filename
try:
requestedFile = file(pagename, 'r')
Very simple.
content = requestedFile.read()
Assume file in local
requestedFile.close()
directory
header = 'HTTP/1.0 200 OK\r\n'
header += 'Content-Length: %d\r\n\r\n' % len(content)
self.request.send(header)
Note creation of file
self.request.send(content)
header information
except IOError:
# could not open the file sent back to
self.request.send('HTTP/1.0 404 Not Found\r\n\r\n')
requestor with the
webServer = TCPServer( ('localhost', 8080), WebHandler)
webServer.serve_forever()
file
Spot check
• With a partner, step through the web
server code and be sure you understand
each line. Take turns explaining the lines
to each other, one line at a time.
Persistence
• So far, connection made, something
happens, connection ends.
• Server is then free to handle another
connection.
• Client is disconnected entirely
• Some applications require persistence –
continuing connection between client
and server
Case Study:
Network Chat-Room
• Develop our own protocol for chat room.
• Connection has permanence unlike previous
examples – persistent connection.
• Client must monitor keyboard and also listen to
the socket.
• Use multithreading to accomplish this. Without
multithreading can get hung-up on listening to
socket or the keyboard.
• Following two slides show different messages
sent from client to server and server to client.
Terry Scott University of Northern Colorado
2007 Prentice Hall
40
Client to Server Messages
Message Type
Join the room
using given
identity.
Broadcast the
message to
everyone.
Send a private
message.
Format
'ADD %s\n' % screenName
'MESSAGE %s\n'%content
'PRIVATE %s\n%s\n'
%(recipient, content)
Quit the chat
'QUIT\n'
Terry Scott University of Northern Colorado
room.
2007 Prentice Hall
41
Server to Client Messages
Message Type
Format
New user has joined. 'NEW %s\n' % screenname
Message was
broadcast to
everyone.
Private message
was sent to user.
'MESSAGE %s\n%s\n' %
(sender, content)
'PRIVATE %s\n%s\n' %
(sender, content)
Someone has left the 'LEFT %s\n' % screenName
room.
Acknowleges
'GOODBYE\n'
request to quit.
Terry Scott University of Northern Colorado
2007 Prentice Hall
42
Chat Server
• Uses ThreadingTCPServer rather than
TCPServer. Needed since can be multiple
people in the chat room.
• _broadcast function used to send to
clients in the chatroom.
• Each person who joins the chat-room is
given a new socket, which is accessed via
a dictionary with the person’s screen
name as the key.
• Code for the server is on the next three
slides.
Terry Scott University of Northern Colorado
2007 Prentice Hall
43
Chat Server Code
from SocketServer import ThreadingTCPServer,
BaseRequestHandler
_socketLookup = dict()
def _broadcast(announcement):
for connection in _socketLookup.values():
connection.send(announcement)
class ChatHandler(BaseRequestHandler):
def handle(self):
username = 'Unknown'
active = True
while active:
transmission = self.request.recv(1024)
if transmission:
command = transmission.split()[0]
data = transmission[1+len(command):]
Terry Scott University of Northern Colorado
2007 Prentice Hall
44
if command == 'ADD':
username = data.strip()
_socketLookup[username] = self.request
_broadcast('NEW %s\n' %username)
elif command == 'MESSAGE':
_broadcast('MESSAGE %s\n%s\n'%(username,data))
elif command == 'PRIVATE':
rcpt = data.split('\n')[0]
if rcpt in _socketLookup:
content = data.split('\n')[1]
_socketLookup[rcpt].send('PRIVATE %s\n%s\n' \
%(username,content))
elif command == 'quit':
active = False
self.request.send('GOODBYE\n')
else:
active = False
self.request.close()
_socketLookup.pop(username)
_broadcast('LEFT %s\n' %username)
myServer = ThreadingTCPServer(('localhost',9000), ChatHandler)
myServer.serve_forever()
Chat client
# Program: chatclient.py
# Authors: Michael H. Goldwasser
#
David Letscher
#
# This example is discussed in Chapter 16 of the book
# Object-Oriented Programming in Python
#
from socket import socket
from threading import Thread
class IncomingThread(Thread):
def run(self):
stillChatting = True
while stillChatting:
# wait for more incoming data
transmission = server.recv(1024)
# 'server' will be
# defined globally at line 27
lines = transmission.split('\n')[:-1]
i = 0
while i < len(lines):
command = lines[i].split()[0]
# first keyword
param = lines[i][len(command)+1: ] # remaining info
if command == 'GOODBYE':
stillChatting = False
elif command == 'NEW':
print '==>', param, 'has joined the chat room'
elif command == 'LEFT':
print '==>', param, 'has left the chat room'
elif command == 'MESSAGE':
i += 1
# need next line for content
print '==>', param + ': ' + lines[i]
elif command == 'PRIVATE':
i += 1
# need next line for content
print '==>', param + ' [private]: ' + lines[i]
i += 1
instructions = """
-------------------------------------------Welcome to the chat room.
To quit, use syntax,
quit
To send private message to 'Joe' use syntax,
private Joe:how are you?
To send message to everyone, use syntax,
hi everyone!
-------------------------------------------"""
server = socket()
# shared by both threads
server.connect( ('localhost', 9000) ) # could be a remote host
username = raw_input('What is your name: ').strip()
server.send('ADD %s\n' % username )
incoming = IncomingThread()
incoming.start()
print instructions
active = True
# main thread for user input
while active:
message = raw_input()
# wait for more user input
if message.strip():
if message.rstrip().lower() == 'quit':
server.send('QUIT\n')
active = False
elif message.split()[0].lower() == 'private':
colon = message.index(':')
friend = message[7:colon].strip()
server.send('PRIVATE %s\n%s\n' \
% (friend,message[1+colon: ]) )
else:
server.send('MESSAGE ' + message)