Transcript Stream Processing in PNEs

Stream Processing in
PNEs
George Porter
Edge Services Session
Winter Retreat - 2004
Overview


Streams prevalent in the edge network
PNEs—general platform for packet processing
(roughly layers 2-4) in the edge



We need a general mechanism for stream processing
(roughly layer 7) that is integrated with the current PNE
design and architecture
I am proposing an extension to PNEs to specify streams
and a mechanism for executing that specification at high
speeds in a general way from the PNE interface
This mechanism provides a clear separation
between the transport of packets and the
structure of the protocol
Packet-vs.-stream processing: what’s the
difference?


Packet Processing
 Ethernet, IP, and TCP
headers in every packet
 Decisions are stateless
between packets
 Resilient to loss, reorder,
duplication
Stream Processing





Assumes reliable transport
Sequence of variable length
data units over TCP
Simple for endhosts, since
O/S delivers in order
Loss, reorder, dups a problem
for in-network elements
PNEs must track data unit
boundaries
Ethernet
IP
TCP
Ethernet
…
Data
Ethernet
IP
TCP
iSCSI1
Data
Data1
IP
TCP
Data
Ethernet
IP
TCP
Data1
iSCSI2
Data2
Examples of observed phenomenon
(observed on OASIS testbed)
header1
data1
header2
data2
Original Stream:
1)
2)
3)
(overlaps with first
And last part of header)
Need for stream processing
Necessary to track any layer-7 protocol
that exists on top of TCP
 Storage



Web/P2P



iSCSI: storage virtualization, load-balancing,
security, caching, in-network optimizations
HTTP pipelining: load-balancing
P2P: HTTP often used as a transport protocol
Measurement/Monitoring

Tracking state of observed protocols over time
PNE mechanism for tracking ADUs
H
H
StreamTracker
Packet flow (fast path)
Packet(s) containing
ADU boundary
Priority
Queue
Computed
ADU size
FixedLogic
(iscsi)
Paramaterized
by specification
PNE mechanism for tracking ADUs
StreamTracker
Packet flow (fast path)
Packet(s) containing
ADU boundary
Priority
Queue
Computed
ADU size
VariableLogic
(http)
Paramaterized
by specification
Evaluation Platform

Deployed onto testbed consisting of





20 Pentium-500 Linux machines
100 Mbit ethernet-based network
iSCSI source and target through Intel drivers
100MB ramdisk on target / 30gig IDE Seagate
drive
Code deployment


Baseline: Click modular router running in
Kernel
“Tracking”: Click router in kernel with custom
elements
Results: #pkts examined

Only a small number of packets are examined in
detail:
Workload
Packets
examined
Total
Packets
Percent
Unpack and compile Apache
2272
42850
5.30%
Resize two ~80Kb images
15
266
5.64%
Store/Read 20Mb MPEG file
595
37207
1.60%
FS caching does not effect above measurements
Results: overhead of packet examination

Overhead of examining packets in depth is large,
but infrequent:
Workload
Stock Click
CPU
cycles/pkt
Tracking
CPU
Cycles/pkt
Percent
Increase
iSCSI regression Test
1430
1871
30.8%
iSCSI regression Test (1% loss)
1328
1937
45.9%
Unpack and compile Apache
1591
2104
32.2%
Next Steps
Implement in PNE software artifact
 Write compiler for specification language
 Deployment




(see poster for deployment overview)
“VideoCollective”, “VideoCollective with
sharing”, “VideoCollective with collaborative
sharing”
Measurement of storage statistics in above
deployment