Transcript Transport Layer Support for Highly Available Network Services

Migratory TCP (MTCP)
Transport Layer Support for HighlyAvailable Network Services
Kiran Srinivasan
DisCoLab
Division of Computer and Information Sciences
Rutgers University
Advisor: Liviu Iftode
M. S. Thesis Defense, 09/24/2001
1
Services Rather Than Servers




Client oblivious to a given server’s location
Client is interested only in the content provided
and QoS
A Service -> set of equivalent servers
Internet services


Most services are based on TCP
Clients demand both high-availability and performance
M. S. Thesis Defense, 09/24/2001
2
TCP-based Internet Services

Adverse conditions


TCP response


Core network congestion, overloaded servers, failed
servers or under DoS attack
Network delays => packet loss => retransmission
TCP limitations


Creates an implicit binding of service to a server !
Clients cannot change to another server for sustained
service
M. S. Thesis Defense, 09/24/2001
3
Migratory TCP



Transport layer protocol that supports dynamic
connection migration
Extension to TCP
A client connection can transparently migrate to different
servers during its lifetime




Modified server applications cooperate to support the handoff
Client applications do not change
Servers can be geographically distributed
Does not depend on application-specific info
M. S. Thesis Defense, 09/24/2001
4
Contributions





Designed a transport layer protocol that enables
dynamic connection migration
Implemented a prototype on FreeBSD
Demonstrated the utilization of our protocol in
building highly-available services
Conducted preliminary performance evaluation
F. Sultan, K. Srinivasan, L. Iftode - “Transport
Layer Support for Highly-Available Network
Services ”, HotOS 2001
M. S. Thesis Defense, 09/24/2001
5
Outline







Motivation
Protocol Design
Prototype Implementation
Preliminary Performance Evaluation
Protocol Utilization
Related Work
Conclusions and Future Work
M. S. Thesis Defense, 09/24/2001
6
The Migration Model
Server 1 -
Client
overloaded
congestion
Server 2
M. S. Thesis Defense, 09/24/2001
7
Triggers and Initiators
Server 1
Client
Server 2
M. S. Thesis Defense, 09/24/2001
8
Design Issues

How to resume the service at the new server?




Without the involvement of the client application
Without full migration of the server application
With low overhead
What is the state that needs to be transferred?


Application state
Protocol state
M. S. Thesis Defense, 09/24/2001
9
Contract Between Server Application
and MTCP

Application




Define fine-grained per-connection application state
Export per-connection application state snapshot
periodically to the kernel
After migration, import per-connection state to resume
service
MTCP


Transfer the per-connection state
Synchronize the application state with the protocol state
M. S. Thesis Defense, 09/24/2001
10
Server Application Interface

export_state



import_state




Export the state snapshot at the origin server
Many times periodically
Import the state snapshot at the destination server
Enables the destination server to resume from the
last snapshot
Only once at the destination server
Light-weight connection migration
M. S. Thesis Defense, 09/24/2001
11
Server Application Example
while ((s = accept(ssock)) != -1)
{
if (import_state(s, &state))
num = state
else
num = 0
recv(s, &buf)
state = ++ num
export_state(s, &state)
recv(s, &buf)
state = ++ num
export_state(s, &state)
...
}
M. S. Thesis Defense, 09/24/2001
12
Contract Between Server Application
and MTCP

Application




Define fine-grained per-connection application state
Export per-connection application state snapshot
periodically
After migration, import per-connection state to resume
service
MTCP


Transfer the per-connection state
Synchronize the application state with the
protocol state
M. S. Thesis Defense, 09/24/2001
13
MTCP Protocol
Server 1
Server 2
Connections
Connections
Application
Protocol
M. S. Thesis Defense, 09/24/2001
14
State Synchronization Example
Application
1
po
Ex
rt
while ((s = accept(ssock)) != -1)
{
if (import_state(s, &state))
num = state
else
num = 0
recv(s, &buf)
state = ++ num
export_state(s, &state)
recv(s, &buf)
state = ++ num
export_state(s, &state)
...
}
3 2
M. S. Thesis Defense, 09/24/2001
MTCP
1
15
State Synchronization Example
Application
V
2
REC
while ((s=accept(ssock)) != -1)
{
if (import_state(s, &state))
num = state
else
num = 0
recv(s, &buf)
state = ++num
export_state(s, &state)
recv(s, &buf)
state = ++num
export_state(s, &state)
...
}
MTCP
4 3
M. S. Thesis Defense, 09/24/2001
1
Out of Sync
16
State Synchronization Example
Application
Application
Application
2
2
3 2
1
MTCP
3 2
1
M. S. Thesis Defense, 09/24/2001
T
MTCP
R
PO
EX
REC
V
1
3 2
MTCP
2
17
Log-Based State Synchronization




Logs are maintained in the protocol
At export time, protocol discards logs (sync-ed)
Logs form part of the state to be transferred
during migration
Resumption of service starts from the last state
snapshot
M. S. Thesis Defense, 09/24/2001
18
Implementation




Modified the TCP/IP stack in FreeBSD kernel
Lazy connection migration mechanism
Evaluation on synthetic benchmarks
Target applications


HTTP server
PostgreSQL front-end
M. S. Thesis Defense, 09/24/2001
19
Lazy Connection Migration
Server 1
Client
Server 2
M. S. Thesis Defense, 09/24/2001
20
Implementation: Per-Connection State
Application
snap
Application
Write
Protocol
Write
Buffer
not ack’ed
M. S. Thesis Defense, 09/24/2001
21
Implementation: Per-Connection State
Application
snap
Read pointer
Protocol
X
Read before snap
ack’ed
Read
Buffer
Read
since snap
From network
M. S. Thesis Defense, 09/24/2001
22
Preliminary Performance Evaluation

Experimental setup



Two servers and a client: P III 233MHz, 256 MB RAM
Servers are connected by a dedicated network link
Benchmarks


Microbenchmark
Simple web server like application
M. S. Thesis Defense, 09/24/2001
23
Microbenchmark
• One migration averaged over 200 runs for each state size
• All times are in microseconds
• The state size is in bytes
M. S. Thesis Defense, 09/24/2001
24
Microbenchmark
The time for migration varies linearly with state size
M. S. Thesis Defense, 09/24/2001
25
Stream Server Experiment




Server application sends a stream of 256K bytes in
chunks of 1K each
Server performance degrades after a threshold
Degradation is emulated by means of delays in the server
Migration policy module inserted into kernel at the client


Smoothed rate estimator
Migration occurs when the estimated rate drops to 70% of max.
observed rate
M. S. Thesis Defense, 09/24/2001
26
Stream Server Experiment Results
Effective rate is almost equal to the best rate from the server
M. S. Thesis Defense, 09/24/2001
27
Implementation Summary


Addition of ~5000 lines of kernel code
Changes to TCP finite state machine


Addition of two logical TCP states: Migrating and
Blackhole
Time spent on the design and implementation
was 9 months
M. S. Thesis Defense, 09/24/2001
28
Protocol Utilization


For high availability
For performance through load balancing schemes


Servers trigger migrations based on a load balancing
policy
Fault tolerance

Eager transfer of connection state information
M. S. Thesis Defense, 09/24/2001
29
Related Work

Classes of related work
TCP
 Fault tolerance
 Process migration

M. S. Thesis Defense, 09/24/2001
30
TCP Related Works

HTTP server fail-over by connection migration [Snoeren ‘00]


Fault-tolerant TCP [Alvisi ‘00]



soft TCP and HTTP state maintained at back-up server
failure masking
persistent connections across server crashes
Stream Control Transmission Protocol (SCTP) - RFC 2960

multi-homing (many IP addresses / endpoint) ensures connectivity in
the face of network failure
M. S. Thesis Defense, 09/24/2001
31
TCP Related Works (cont’d)

MSOCKS [Bhagwat `98]


Proxy based “TCP splicing” for mobile clients with
multiple interfaces
Indirect TCP [Bakre ‘95]

Connection handoffs between MSRs allow mobile
hosts to maintain open connections with a fixed
host
M. S. Thesis Defense, 09/24/2001
32
Fault Tolerance Related Works


Log-based rollback recovery
The NonStop kernel [Bartlett ’81]
M. S. Thesis Defense, 09/24/2001
33
Process Migration Related Works




Condor [Litzkow ‘88]
Locus [Popek ‘81]
MOSIX [Barak ‘85]
Sprite [Ousterhout ‘88]
M. S. Thesis Defense, 09/24/2001
34
Conclusions






We provide a transport layer protocol that enables
building of highly-available services
The transport layer protocol allows dynamic server
endpoint connection migration
We implemented a prototype in a FreeBSD kernel
We designed an API for the server applications
Preliminary performance evaluation on a web serverlike application achieved constant rate in the presence
of server degradation
http://discolab.rutgers.edu/mtcp
M. S. Thesis Defense, 09/24/2001
35
Future Work




Investigate migration policies
Alternate state transfer implementations
Recursive connection migration mechanism
Future applications
M. S. Thesis Defense, 09/24/2001
36
Acknowledgment


Florin – for contributing to the design and
implementation of the project
Other members of DisCo Lab
M. S. Thesis Defense, 09/24/2001
37
Thank You
Demo in CoRE 336 !
M. S. Thesis Defense, 09/24/2001
38