Transcript NUST_MSIT8_DC_Lecture_5_1

Tuesday, December 23, 2008
If one ox could not do the job
they did not try to grow a
bigger ox, but used two oxen.
-
Grace Murray Hopper
(1906-1992)
1
Distributed Computing
Class: BSIT-8
Instructor: Raihan Ur Rasool
Chapter 04:
Inter-process Communication
Where are we ?







3
Introduction
The API for the Internet protocols
External data representation and marshalling
Client-Server communication
Group communication
Case study: interprocess communication in UNIX
Summary
External data representation and marshalling
introduction
[data structures and a sequence of bytes]
Why does the communication data need external data representation and
marshalling?
[data structures must be flattened]

Different data format on different computers, e.g., big-endian/little-endian
integer order, ASCII (Unix) / Unicode character coding
How to enable any two computers to exchange data values?
1.
The values be converted to an agreed external format before
transmission and converted to the local form on receipt
2.
The values are transmitted in the sender’s format, together with an
indication of the format used, and the receipt converts the value if
necessary


4
External data representation and marshalling
introduction





Why does the communication data need external data representation and marshalling?

Different data format on different computers, e.g., big-endian/little-endian integer
order, ASCII (Unix) / Unicode character coding
How to enable any two computers to exchange data values?
1.
The values be converted to an agreed external format before transmission and
converted to the local form on receipt
2.
The values are transmitted in the sender’s format, together with an indication of
the format used, and the receipt converts the value if necessary
External data representation

An agreed standard for the representation of data structures and
primitive values
Marshalling

The process of taking a collection of data items and assembling them into
a form suitable for transmission in a message

Usage: for data transmission or storing in files
Three alternative approaches to EDR

5
CORBA’s common data representation / Java’s object serialization & XML
Reading Assignment ???

CORBA’s Common Data Representation (CDR)
Java Object Serialization
Extensible Markup language (XML)

Next Time:




Recap
Client-server communication

6
CORBA’s Common Data Representation
(CDR)
Represent all of the data types that can be used as arguments and
return values in remote invocations in CORBA
15 primitive types



Short (16bit), long(32bit), unsigned short, unsigned long, float, char, …
Constructed types


Types that composed by several primitive types
A message example
The type of a data item is not given with the data representation
in message




7
It is assumed that the sender and recipient have common knowledge
of the order and types of the data items in a message.
RMI and RPC are on the contrary
CORBA CDR for constructed types
Type
Representation
sequence
length (unsigned long) followed by elements in order
string
length (unsigned long) followed by characters in order (can also
can have wide characters)
array
array elements in order (no length specified because it is fixed)
struct
in the order of declaration of the components
enumerated
unsigned long (the values are specified by the order declared)
union
type tag followed by the selected member
8
CORBA CDR message
Struct Person {
string name;
string place;
long year;
};
index in
sequence of bytes
0–3
4–7
8–11
12–15
16–19
20-23
24–27
The flattened form represents a
9
4 bytes
5
"Smit"
"h___"
6
"Lond"
"on__"
1934
notes
on representation
length of string
‘Smith’
length of string
‘London’
unsigned long
Person struct with value: {‘Smith’, ‘London’, 1934}
External data representation-The SUN XDR
standard


The entire XDR message consists of a sequence of 4-byte
objects:
example: ‘Hi’, ‘There’, 2001
2
“Hi--”
5
“ Ther”
“e---”
2001
sequence length
‘Hi’
‘There’
Must also define which end
of each object is the most significant
Integer 2001
The marshalled message of the example: 2 Hi 5 There 2001
10
Java object serialization

Serialization (deserialization)






The activity of flattening an object or a connected set of objects into a
serial form that is suitable for storing on the disk or transmitting in a
message
Include information about the class of each object
Handles: references to other objects are serialized as handles
 Each object is written once only
Example
Make use of Java serialization
 ObjectOutputStream.writeObject, ObjectInputStream.readObject
The use of reflection


Reflection : The ability to enquire about the properties of a class, such
as the names and types of its instance variables and methods
Reflection makes it possible to do serialization (deserialization) in a
completely generic manner
11
Indication of Java serialized form
Public class Person implements Serializable {
private String name;
private String place;
private int year;
public Person (String aName, String aPlace, int aYear){
name = aName;
place = aPlace;
year = aYear;
}
// followed by methods for accessing the instance variables
}
Person p = new Person(“Smith”, “London”, 1934);
Person
3
1934
Serialized values
8-byte version number
java.lang.String
int year
name:
5 Smith
6 London
Explanation
h0
class name, version number
java.lang.String number, type and name of
place:
instance variables
h1
values of instance variables
The true serialized form contains additional type markers; h0 and h1 are handles
12
Representation of a remote object reference
•An identifier for a remote object that is valid throughout a
distributed system
•Must ensure uniqueness over space and time
32 bits
32 bits
Internet address
port number
32 bits
time
32 bits
object number
interface of
remote object
•Existence of remote object references
•Life of the remote object references
•Remote object references may not be used for relocated objects
13
Objectives of this lecture



14
To study the general characteristics of interprocess
communication and the particular characteristics of both
datagram and stream communication in the Internet.
To be able to write Java applications that use the Internet
protocols and Java serialization.
The design issues for Request-Reply protocols and how
collections of data objects may be represented in
messages (RMI and language integration are left until
Chapter 5).
Where are we ?







15
Introduction
The API for the Internet protocols
External data representation and marshalling
Client-Server communication
Group communication
Case study: interprocess communication in UNIX
Summary
The request-reply protocol

The request-reply protocol




Normally synchronous [client blocks until the reply arrives]
May be reliable
 Reply is ACK
Asynchronous may be alternative
 Client may retrieve the reply later.
Mostly implemented over UDP but not TCP



Acknowledgements are redundant since requests are followed by
replies
Establishing a connection involves two extra pairs of messages in
addition to the pair required for a request and a reply
Flow control is redundant for the majority of invocations, which pass
only small arguments and results
16
The request-reply protocol
Client
doOperation
Server
Request
message
(wait)
Reply
message
getRequest
select object
execute
method
sendReply
(continuation)
17
public byte[] doOperation (RemoteObjectRef o, int methodId, byte[] arguments)
sends a request message to the remote object and receives the reply.
The arguments specify the remote object, the method to be invoked and the
arguments of that method.
public byte[] getRequest ();
acquires a client request via the server port.
public void sendReply (byte[] reply, InetAddress clientHost, int clientPort);
sends the reply message reply to the client at its Internet address and port.
Request-reply message structure
messageType
int (0=Request, 1= Reply)
requestId
int
Contains message identifier
objectReference
RemoteObjectRef
methodId
int or Method
arguments
array of bytes
information to be transmitted in a request or reply message
32 bits
32 bits
Internet address
port number
18
32 bits
time
32 bits
object number
Representation of Remote Object Reference
interface of
remote object
Request-reply message structure
 Fields:
 1. Message Type
 Request or reply
 2. A doOperation generates requestId for each message

A server copies in the reply message
 3 Remote object reference
 marshalled
 4. methodId
 Message Identifier:
 requestId increasing sequence of U_integers (reset to zero)
 Identfier for the sender process, for example its port and internet
address
19
4,294,967,295
The request-reply protocol

Failure model
Problems:







Omission failures. (and failure of processes—crash failure)
No guarantee of delivery in order.
Timeout
 doOperation return exception when repeatedly issued requests are all
timeout
Duplicate request messages:
 filter out duplicates by requestID
 if the server has not yet sent the reply, transmit the reply after finishing
operation execution
Lost Reply Messages
 If the server has already sent the reply, execute the operation again to
obtain the result. Note idempotent operation, e.g., add an element to a
set, and a contrary example, append an item to a sequence
History
 History: server contains a record of reply messages that have been
transmitted to avoid re-execution of operations –memory cost
20
HTTP: an example of a request – reply protocol



HTTP used by web browsers to make requests to web
servers and to receive replies from them.
Client requests specify a URL that includes the DNS
hostname of a web server, an optional port number and
an identification of a resource.
HTTP specifies:







Messages.
Methods.
Arguments.
Results.
Rules for representing data in messages.
Fixed set of methods, unlike other RMI
Content negotiation and password style authentication
21
HTTP: an example of a request – reply protocol


If over TCP
Each client-server interaction consists of the following steps





Persistent connection (http1.1, RFC 2616)



Connections that remain open over a series of request-reply exchanges
between client and server
Closing
Marshalling



The client requests and the server accepts a connection at the default server
port or at a port specified in the URL
The client sends a request message to the server
The server sends a reply message to the client
The connection is closed
Request and replies are marshalled into messages as ASCII text string
Resources are represented as byte sequences and may be compressed
 MIME (Multipurpose Internet Mail Extensions) to send text, images etc,
in emails
HTTP Methods

22
GET, HEAD, POST, PUT, DELETE, OPTIONS, TRACE
HTTP - methods








GET: Requests the resource whose URL is given as argument.
HEAD: Similar to GET, but no data returned.
POST: Can deal with data supplied with the request.
PUT: data supplied in the request is stored with the given URL as
its identifier either as modification of the an existing resource or
as a new resource.
DELETE: Server deletes resource identified by the URL.
OPTIONS: Server supplies client with list of methods that it
allows.
TRACE: Server sends the request back.
Each client request specifies the name of a method to be applied
to a resource at the server and the URL of that resource.
23
HTTP request / reply messages
 Request/Reply:
 Header: for example acceptable content type
 The message body contains data associated with the
URL specified in the request
method
URL or pathname
GET
//www.dcs.qmw.ac.uk/index.html
HTTP version
headers message body
HTTP/ 1.1
Contents of HTTP request message
HTTP version
HTTP/1.1
status code reason headers message body
200
/303
OK
Contents of HTTP reply message
24
resource data
Assignment

Install LAM-MPI on Linux

25
Home Assignment 2 [30/12/08]
Objectives of this lecture




26
To study the general characteristics of interprocess
communication and the particular characteristics of both
datagram and stream communication in the Internet.
To be able to write Java applications that use the Internet
protocols and Java serialization.
To be aware of the design issues for Request-Reply
protocols and how collections of data objects may be
represented in messages (RMI and language integration
are left until Chapter 5).
To be able to use the Java API to IP multicast and to
consider the main options for reliability and ordering in
group communication.
Where are we ?







27
Introduction
The API for the Internet protocols
External data representation and marshalling
Client-Server communication
Group communication [multicast operation]
Case study: interprocess communication in UNIX
Summary
The characteristics of Multicast

Fault tolerance based on replicated services


Client request are multicast to all the members of the group, each of
which performs an identical operation
Finding the discovery servers in spontaneous networking

Multicast message can be used by servers and clients to locate available
discovery services to register their interfaces or to look up the
interfaces of other services
Service Interface provides an entry point that consumers use to
access the functionality exposed by the application [network addressable]
28
The characteristics of Multicast

Fault tolerance based on replicated services


Finding the discovery servers in spontaneous networking


Multicast message can be used by servers and clients to locate available
discovery services to register their interfaces or to look up the
interfaces of other services
Better performance through replicated data


Client request are multicast to all the members of the group, each of
which performs an identical operation
Data are replicated to increase the performance of a service, e.g., Web
Cache. Each time the data changes, the new value is multicast to the
processes managing the replicas
[bandwidth]
Propagation of event notification

29
Multicast to a group may be used to notify processes when something
happens, e.g., the Jini system uses multicast to inform interested clients
when new lookup services advertise their existence. [newsgroup]
IP Multicast Protocol– group communication

A multicast group is specified by a class D Internet address




The membership of a group is dynamic





It is possible to send datagram to a multicast group without being a member and join
or leave the group at any time.
IPv4


Built on top of IP
Sender is unaware of the identities of the individual recipients
Available only via UDP
Multicast routers
[Internet]
 use the broadcast capability of the local network [Ethernet]
 MTTL (time to live) - specify the number of routers a multicast message is allowed
to pass.
[to limit the distance of propagation]
Multicast address allocation
 Permanent group – exist even without members 224.0.0.1 to 224.0.0.255
 Temporary group – the other addresses, set TTL to a small value
Temporary groups, free multicast address, avoid accidents, X IP multicast
Failure model: due to UDP, [no guarantees, omission failure –unreliable multicast]
Java API to IP multicast
30
Multicast peer joins a group and sends and
receives datagrams
import java.net.*;
import java.io.*;
public class MulticastPeer{
public static void main(String args[]){
// args give message contents & destination multicast group (e.g. "228.5.6.7")
MulticastSocket s =null;
try {
InetAddress group = InetAddress.getByName(args[1]);
s = new MulticastSocket(6789);
s.joinGroup(group);
byte [] m = args[0].getBytes();
DatagramPacket messageOut =
new DatagramPacket(m, m.length, group, 6789);
s.send(messageOut);
// this figure continued on the next slide
31
Multicast peers example… continued
// get messages from others in group
byte[] buffer = new byte[1000];
for(int i=0; i< 3; i++) {
DatagramPacket messageIn =
new DatagramPacket(buffer, buffer.length);
s.receive(messageIn);
System.out.println("Received:" + new String(messageIn.getData()));
}
s.leaveGroup(group);
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(s != null) s.close();}
}
}
32
Reliability and ordering of multicast

Failures



Some examples of the effects of reliability and ordering




Router failure prevent all recipients beyond it receiving the message
Members of a group receive the same array of messages in different
orders
Fault tolerance based on replicated services
 if one of them misses a request, it will become inconsistent with the
others
Finding the discovery servers in spontaneous networking
 an occasional lost request is not an issue when locating a discovery
server
Replicated Data [lost messages, ordering]—replication technique
Reliable multicast or unreliable multicast?

33
According to application’s requirement
Where are we ?







34
Introduction
The API for the Internet protocols
External data representation and marshalling
Client-Server communication
Group communication
Case study: interprocess communication in UNIX –
FOR SELF STUDY
Summary
Summary

Two alternative building blocks



Marshalling


Base on UDP or TCP
Multicast

35
CORBA’s CDR and Java serialization
Request-Reply protocol


Datagram Socket: based on UDP, efficient but suffer from
failures
Stream Socket: based on TCP, reliable but expensive
IP multicast is a simple multicast protocol
Home Assignment 2
Q.1 From the book, answer the following
questions
a) 4.1
b) 4.2
c) 4.5
d) 4.6
e) 4.7
f) 4.25
g)
36
4.27
Quiz 02
Reading Material
Home Assignment 2: Learn MPI Programming
http://www.netlib.org/utk/papers/mpi-book/mpi-book.html
Discussion date: 1st January, 2009
37
Extra Slides for Reference
38
UNIX socket

Datagram communication





Datagram Socket
Bind
Sendto
recvfrom
Stream communication




stream socket , bind
Accept
Connect
Write and read
39
Sockets used for datagrams
Sending a message
Receiving a message
s = socket(AF_INET, SOCK_DGRAM, 0)
s = socket(AF_INET, SOCK_DGRAM, 0)
bind(s, ClientAddress)
bind(s, ServerAddress)
sendto(s, "message", ServerAddress)
amount = recvfrom(s, buffer, from)
ServerAddress and ClientAddress are socket addresses
40
Sockets used for streams
Requesting a connection
s = socket(AF_INET, SOCK_STREAM,0)
connect(s, ServerAddress)
Listening and accepting a connection
s = socket(AF_INET, SOCK_STREAM,0)
bind(s, ServerAddress);
listen(s,5);
sNew = accept(s, ClientAddress);
write(s, "message", length)
n = read(sNew, buffer, amount)
ServerAddress and ClientAddress are socket addresses
41