Transcript PNE05 - BNRG - University of California, Berkeley

The Computer is the Network:
The Emergence of Programmable
Network Elements
Randy H. Katz
Computer Science Division
Electrical Engineering and Computer Science Department
University of California, Berkeley
Berkeley, CA 94720-1776
1
Presentation Outline
•
•
•
•
•
Motivation
OASIS Project
RouterVM
COPS
Summary and Conclusions
2
Presentation Outline
•
•
•
•
•
Motivation
OASIS Project
RouterVM
COPS
Summary and Conclusions
3
Managing Edge Network
Services and Applications
• Not shrink wrap software—but cascaded “appliances”
• Data Center in-a-box blade servers, network storage
• Brittle to traffic surges and shifts, yielding network
disruption
Edge Network
Blades
Server
Server
Server
Server
Traffic
Shaper
Load
Balancer
IDS
Firewall
Wide
Area
Network
Egress
Checker
Edge Network Middleboxes
4
Appliances Proliferate:
Management Nightmare!
Packeteer PacketShaper
Network Appliance NetCache F5 Networks BIG-IP LoadBalancer
Localized content delivery platform
Web server load balancer
Traffic monitor and shaper
Ingrian i225
Cisco SN 5420
SSL offload appliance
IP-SAN storage gateway
NetScreen 500
Extreme Networks SummitPx1
Firewall and VPN
L2-L7 application switch
Nortel Alteon Switched Firewall
CheckPoint firewall and L7 switch
Cisco IDS 4250-XL
Intrusion detection system
5
Network Support for Tiered
Applications
Server
Server
LAN
Server
Database
Tier
Server
App
Tier
LAN
Server
Server
Web
Tier
Datacenter Network(s)
Load
Balancer
LAN
Firewall
Wide
Area
Network
Egress
Checker
Load
Server
Server
Balancer
Unified
Server
Server
+
LAN Servers
Server Servers
Server
Firewall
on
on
+
Demand Demand
Configure servers,
Egress
storage, connectivity
Blades
Checker
net functionality as needed
Wide
Area
Network
6
“The Computer is the Network”
• Emergence of Programmable Network Elements
– Network components where net services/applications execute
– Virtualization (hosts, storage, nets) and flow filtering (blocking,
delaying)
• Computation-in-the-Network is NOT Unlimited
– Packet handling complexity limited by latency/processing overhead
– NOT arbitrary per packet programming (aka active networking)
– Allocate general computation like proxies to network blades
• Beyond Per Packet Processing: Network Flows
– Managing/configuring network for performance and resilience
– Adaptation based on Observe (Monitor), Analyze (Detect, Diagnose),
Act (Redirect, Reallocate, Balance, Throttle)
7
Key Technical Challenge:
Network Reliability
• Berkeley Campus Network
– Unanticipated traffic surges render the network
unmanageable (and may cause routers to fail)
– Denial of service attacks, latest worm, or the newest file
sharing protocol largely indistinguishable
– In-band control channel is starved, making it difficult to
manage and recover the network
• Berkeley EECS Department Network (12/04)
– Suspected denial-of-service attack against DNS
– Poorly implemented/configured spam appliance adds to DNS
overload
– Traffic surges render it impossible to access Web or mount
file systems
• Network problems contribute to brittleness of
distributed systems
8
Presentation Outline
•
•
•
•
•
Motivation
OASIS Project
RouterVM
COPS
Summary and Conclusions
9
Overlays and
Active
Services for
Inter-networked
Storage
10
Vision
• Better network management of services/applications
to achieve good performance and resilience even in
the face of network stress
– Self-aware network environment
– Observing and responding to traffic changes
– While sustaining the ability to control the network
• Packet flow manipulations at L4-L7 made possible by
new programmable network elements
• New service building blocks
–
–
–
–
Flow extraction
Packet annotation
Packet translation
Resource management
11
Generic Network Element
Architecture
Buffers
Buffers
CP
CP
CP
CP
Classification
Processor
“Tag”
Mem
CP
CP
CP
AP
Rules &
Programs
Interconnection
Fabric
Output Ports
Input Ports
Buffers
Action
Processor
12
OASIS Framework
Applications
NPUs,
Blades
cPredicates
Packet
Processing
Software
Software
Router
Linux-based,
Click, etc.
Network Service Protection
Packet/Flow Segregation
Control
Plane
SAN, FW, IDS,
Traffic Shaper,
A-Layer
GeneralizedL7Filtering
Switch, …
RouterVM and Composition
+
Net
Event
Apps
Manager
Hardware
Assist for
Generalized
Inspection
TCAM
FPGA
State
Machine
Extended Router
Deeper Inspection
Generalized Actions
“Basic” Router
Edge
Core
Forwarding,
IP Filtering
13
E.g., Application-Specific QoS
for Three-Tier Systems
• Composable building blocks to build web services
– Web containers, various app/ejb containers, persistent state via
automatically managed DB pools
• Problem: Open control loop/requests driven by users
– Flash traffic, increased workload can overload components of the
web service
– Hard to provision; hard to make performance guarantees;
seemingly broken behavior to the end user
• TrafficOperation Classification
– Ants: Lightweight processing, touch few components
– Elephants: Heavyweight processing, cross-layer, many components
– Hard to distinguish statically, but can be determined through
statistical techniques
– Correlate request patterns with measured system parameters
(web + db CPU, disk activity, OS process switches, etc.) to
identify elephants vs. ants
14
Identifying RuBIS Elephant Flows for
Admission Control/QoS
No
Network
Visibility
HTTP
Header
Visibility
SLOW
DOWN
SLOW
DOWN
S
e
r
v
e
r
s
S
e
r
v
e
r
s
15
Request Time Distribution with
Preferential Scheduling
Uncontrolled
Blocking phenomenon
Session time goes up
But worst case goes way down
Controlled
total requests
correlated URLs
req/sec (avg)
session time (avg)
max request time
stock adm control
756137
1143264
112521
105964
462
782
670
872
154.7
32.7
16
Presentation Outline
•
•
•
•
•
Motivation
OASIS Project
RouterVM
COPS
Summary and Conclusions
17
RouterVM
• High-level specification environment for
describing packet processing
• Virtualized: abstracted view of underlying
hardware resources of target PNEs
– Portable across diverse architectures
– Simulate before deployment
• Services, policies, and standard routing
functions managed thru composed packet filters
– Generalized packet filter: trigger + action bundle, cascaded,
allows new services and policies to be implemented /
configured thru GUI
– New services can be implemented without new code through
library building blocks
Mel Tsai
18
Extended Router Architecture
• Virtualized components representative of a “common”
router implementation
• Structure independent of particular hardware
Virtual line card
instantiated for every
port required by
application
Virtual backplane
shuttles packets
between line cards
CPU handles
routing protocols
& mgmt tasks
Blue “standard” components
Yellow components added &
configured per-application
Filters are key to
flexibility
Compute engines
perform complex,
high-latency
processing on
flows
Mel Tsai
19
Generalized Packet Filters
• Examples:
• Key to flexibility
– Extend router “filter” concept
– Small # of GPF building blocks yield
large number of apps
» Compose/parameterize filters
» No code writing
– Supports flexible classification,
computation, and actions
– Executed in numeric order
– How “complete” is this formalization?
Packet
filter 1
Packet
filter 2
Packet
filter n
Default
filter
L2
L2Switching
Switching
Engine
Enginew/ARP
w/ARP
– Traffic shaping and monitoring
– L7 traffic detection
(Kazaa, HTTP, AIM, POP3, etc.)
– QoS and packet scheduling
– NAT
– Intrusion detection
– Protocol conversion (e.g. IPv6)
– Content caching
– Load balancing
– Router/server health monitoring
– Storage
– Fibre Channel to IP
– iSCSI
– XML preprocessing
– TCP offload (TOE)
– Mobile host management, 802.11
– Encryption/compression. VPNs
– Multicast, Overlays, DHTs
Mel Tsai
20
GPF Example
A Server Load Balancer and L7 Traffic Detector
Control
Processor
GPF 5:
GPF 10:
SLB
P2P
Servers
Ext. IP = 24.0.5.6
…
10.0.0.1
IP Router
Engine
Backplane
QoS Module
L2 Switching
Engine w/ARP
10.0.0.2
To Clients
10.35.x.x
21
GPF Example
A Server Load Balancer and L7 Traffic Detector
Control
Processor
GPF 5:
GPF 10:
SLB
P2P
Servers
Ext. IP = 24.0.5.6
…
10.0.0.1
name algorithm flowid sip smask dip dmask proto action1 action2 action3 -
L2 Switching
Engine w/ARP
Server Load Balancer
equal flowsIP Router
sip, sport Engine
any
any
24.0.5.6
255.255.255.255
udp, tcp
slb nat 10.0.0.1, 10.0.0.2
log connections, file = log.txt
tag “skip Yahoo Messenger
Filter”
Backplane
QoS Module
GPF 5 Setup
10.0.0.2
To Clients
10.35.x.x
22
GPF Example
A Server Load Balancer and L7 Traffic Detector
Control
Processor
GPF 5:
GPF 10:
SLB
P2P
Ext. IP = 24.0.5.6
…
GPF 10 Setup
L2 Switching
Engine w/ARP
IP Router
Engine
name type pattern -
timeout flowid sip smask dip dmask proto action1 action2 -
Yahoo Messenger Filter
yahoomessenger
^(ymsg|ypns|yhoo).?.?.?.?.?.
?.?(w|t).*\xc0\x80
10 min
sip, dip, sport, dport
any
any
10.35.0.0
255.255.0.0
tcp
limit 1 kbps
email root
10.0.0.1
Backplane
QoS Module
Servers
10.0.0.2
To Clients
10.35.x.x
23
GPF “Fill-in” Specification
RouterVM Generalized Packet Filter (type L7)
“Packet filter” as high-level,
programmable building-block
for network appliance apps
Traditional Filter
FILTER 19 SETUP
Classification
Parameters
Action
NAME SIP SMASK DIP DMASK PROTO SRC PORT DST PORT VLAN ACTION -
example
any
255.255.255.255
10.0.0.0
255.255.255.0
tcp,udp
any
80
default
drop
24
GPF Action Language
•
Basic set of assignments, evaluations, expressions,
control-flow operators, and “physical” actions on
packets/flows
–
–
–
–
Control-flow: If-then-else, if-not
Evaluation: ==, <=, >=, !=
Packet flow control: Allow, unallow, drop, skip filter, jump filter
Packet manipulation: Field rewriting (ip_src == blah,
tcp_src = blah), truncation, header additions
– Actions: NAT, loadbalance, ratelimit, (perhaps others)
– Meta actions: packet generation, logging, statistics gathering
•
Higher level of abstraction than C/C++ or packet
processing language
26
Implemented GPF Libraries
• Basic Filter
– Simple L2-L4 header classifications
– Any RouterVM actions
• L7 Filter
– Adds regular expressions, TCP termination, ADU reconstruction
• NAT Filter
– Adds a few more capabilities beyond the simple NAT action that
is available to all GPFs
• Content Caching
– Builds on the L7 filter functionality
• WAN Link Compression
– Relatively simple to specify, but requires lots of computation
• IP-to-FC Gateway
– Requires its own table format & processing
• XML Preprocessing
– Not very well documented, and difficulty is unknown…
27
GPF Expressiveness Analysis
Expressiveness at the app layers depends on the
breadth of GPF library and GPFs for specific apps
28
GPF Performance: Complex Filters
L2-L4 Headers
“Retreat”
25 bytes of char ‘X’
“Retreat”
25 bytes of char ‘X’
“Retreat”
Padding with ‘X’
Lesson: try to use
start-of-buffer
indicators ^ and
avoid *’s…
Many apps can be
identified with
simple start-ofbuffer expressions
Regex involves
payload copying,
which might be
avoidable
29
Presentation Outline
•
•
•
•
•
Motivation
OASIS Project
RouterVM
COPS
Summary and Conclusions
30
Observations and Motivations
• Internet reasonably robust to point problems
like link and router failures (“fail stop”)
• Successfully operates under a wide range of
loading conditions and over diverse
technologies
• During 9/11/01, Internet worked reasonable
well, under heavy traffic conditions and with
some major facilities failures in Lower
Manhattan
31
Why and How Networks Fail
• Complex phenomenology of failure
• Recent Berkeley experience suggests that traffic
surges render enterprise networks unusable
• Indirect effects of DoS traffic on network
infrastructure: role of unexpected traffic patterns
– Cisco Express Forwarding: random IP addresses flood route
cache forcing all traffic to go through router slow path—high
CPU utilization yields inability to manage router table updates
– Route Summarization: powerful misconfigured peer overwhelms
weaker peer with too many router table entries
– SNMP DoS attack: overwhelm SNMP ports on routers
– DNS attack: response-response loops in DNS queries generate
traffic overload
32
COPS
Checking
Observing
Protecting
Services
33
Check
• Checkable Protocols: Maintain invariants and
techniques for checking and enforcing
protocols
– Listen & Whisper: well-formed BGP behavior
– Traffic Rate Control: Self-Verifiable Core Stateless Fair
Queuing (SV-CSFQ)
• Existing work requires changes to protocol end
points or routers on the path
– Difficult to retrofit checkability to existing protocols
without embedded processing in PNEs
– Building blocks for new protocols
» Observable protocol behavior
» Cryptographic techniques
» Statistical methods
34
Protect
• Protect Crucial Services
– Minimize and mitigate effects of attacks and traffic
surges
– Classify traffic into good, bad, and ugly (suspicious)
» Good: standing patterns and operator-tunable policies
» Bad: evolves faster, harder to characterize
» Ugly: that which cannot immediately be determined as
good or bad
– Filter the bad, slow the suspicious, maintain resources for
the good (e.g., control traffic)
» Sufficient to reduce false positives
» Some suspicious-looking good traffic may be slowed
down, but won’t be blocked
35
Observe
• Observation (and Action) Points
– Network points where control is exercised, traffic
classified, resources allocated
– Routers + End Hosts + Inspection-and-Action Boxes
(iBoxes)
» Prototyped on commercial PNEs
» Placed at Internet and Server edges of enterprise net
» Cascaded with existing routers to extend their
functionality
36
iBox Placement for
Observation and Action
I
R
Internet or
WAN Edge
R
E
E
E
I
R
I
R
R
Distribution
Tier
E
Access
Tier
E
R
I
E
E
E
Access
Tier
R
Network Services
End Hosts
iBoxes strategically placed near
entry/exit points within the
Enterprise network
I
E
E
Server
End Hosts
E
E
Access
Tier
User
End Hosts
37
Annotation Layer:
Between Routing and Transport
External Traffic
Label
Packet
Network
Services
Internal
Router
iBox
iBox
Boundary
Router
Action: Mark
packets
Detect load and trigger action:
Slow traffic with “external” labels
Internal Traffic
Enterprise Network
Label
Packet
• Marking vs. rewriting approach
Packet
– E.g., mark packets as internally vs. externally sourced using IP
header options
• Prioritize internal vs. external access to services
solves some but not all traffic surge problems
38
Annotation Layer:
iBox Piggybacked Control Plane
• Problem: Control plane starvation
• Use A-layer for iBox-to-iBox communication
–
–
–
–
–
Passively piggyback on existing flows
“Busy” parts of network have lots of control plane b/w
Actively inject control packets to less active parts
Embedded control info authenticated and sent redundantly
Priority given to packets w/control when net under stress
• Network monitoring and statistics collection and
dissemination subsystem currently being
designed and prototyped
39
Observe and Protect
iBoxes implemented on
commercial PNEs
– Don’t: route or implement (full)
protocol stacks
– Do: protect routers and shield
network services
» Classify packets
» Extract flows
» Redirect traffic
» Log, count, collect stats
» Filter/shape traffic
40
Presentation Outline
•
•
•
•
•
Motivation
OASIS Project
RouterVM
COPS
Summary and Conclusions
42
OASIS
• Processing-in-the-Network is real
– Networking plus processing in switched and routed
infrastructures
– Configuration and management of packet processing cast
onto programmable network elements (network appliances
and blades)
• Unifying Framework
– Methods to specify functionality and processing
» RouterVM: Filtering, Redirecting, Transformation
» cPredicates: Control extraction and execution based
on packet applications content
• Application-specific network processing based
on session extraction
43
COPS
• PNEs: foundation of a pervasive infrastructure
for observation and action at the network level
– iBoxes Observation and Action points
– Annotation Layer for marking and control
• Check-Observe-Protect paradigm for
protecting critical resources when network is
under stress
• Functionality eventually migrates into future
generations of routers
– E.g., Blades embedded in routers
44
The Computer is the
Network:
The Emergence of
Programmable
Network Elements
Randy H. Katz
Thank You!
45