Transcript Communication - Princeton University

Frenetic: Programming
Software Defined Networks
Jennifer Rexford
Princeton University
http://www.frenetic-lang.org/
Joint with Nate Foster, David Walker, Rob Harrison, Chris Monsanto,
Cole Schlesinger, Mike Freedman, Mark Reitblatt, Joshua Reich
Traditional Networks
Management Plane
Monitors traffic,
configures policy
Control Plane (software)
Tracks topology; computes
routes; modifies data plane
Data Plane (hardware)
Forwards, filters, buffers, tags,
rate-limits; collects statistics
2
Software Defined Networking (SDN)
Logically-centralized control
Smart,
slow
API to the data plane
(e.g., OpenFlow)
Dumb,
fast
Switches
3
Momentum
• Everyone has signed on
– Google, Facebook,
Microsoft, Yahoo, Verizon,
Deutsche Telekom
• New applications
– Host mobility
– Server load balancing
– Network virtualization
– Dynamic access control
– Energy-efficiency
• Real deployments
Programming OpenFlow Networks
• The Good
– Simple data plane abstraction
– Logically-centralized architecture
– Direct control over switch policies
• The Bad
– Low-level programming interface
– Functionality tied to hardware
– Explicit resource control
• The Ugly
Images by Billy Perkins
– Non-modular, non-compositional
– Programmer faced with challenging
distributed programming problem
5
Language-Based Abstractions
• Benefits
–
–
–
–
–
Modularity
Portability
Efficiency
Assurance
Simplicity
Simple, high-level abstractions are crucial for achieving
the vision of software-defined networking.
OpenFlow Networks
7
Data-Plane: Simple Packet Handling
• Simple packet-handling rules
– Pattern: match packet header bits
– Actions: drop, forward, modify, send to controller
– Priority: disambiguate overlapping patterns
– Counters: #bytes and #packets
1. src=1.2.*.*, dest=3.4.5.*  drop
2. src = *.*.*.*, dest=3.4.*.*  forward(2)
3. src=10.1.2.3, dest=*.*.*.*  send to controller
8
Controller: Programmability
Application
Network OS
Events from switches
Topology changes,
Traffic statistics,
Arriving packets
Commands to switches
(Un)install rules,
Query statistics,
Send packets
9
E.g.: Server Load Balancing
• Pre-install load-balancing policy
• Split traffic based on source IP
src=0*
src=1*
Seamless Mobility/Migration
• See host sending traffic at new location
• Modify rules to reroute the traffic
11
Programming Abstractions for
Software Defined Networks
12
Three Main Abstractions
Composing modules
Reading
state
Writing
policies
OpenFlow
Switches
13
Reading State: Multiple Rules
• Traffic counters
– Switch counts bytes and packets matching a rule
– Controller application polls the counters
• Multiple rules
– E.g., Web server traffic except for source 1.2.3.4
1. srcip = 1.2.3.4, srcport = 80
2. srcport = 80
• Solution: predicates
– E.g., (srcip != 1.2.3.4) && (srcport == 80)
– Run-time system translates into switch patterns
14
Reading State: Unfolding Rules
• Limited number of rules
– Switches have limited space for rules
– Cannot install all possible patterns
• Must add new rules as traffic arrives
– E.g., histogram of traffic by IP address
– … packet arrives from source 5.6.7.8
1. srcip = 1.2.3.4
1. srcip = 1.2.3.4
2. srcip = 5.6.7.8
• Solution: dynamic unfolding
– Programmer specifies GroupBy(srcip)
– Run-time system dynamically adds rules
15
Reading: Extra Unexpected Events
• Common programming idiom
–First packet goes to the controller
–Controller application installs rules
packets
16
Reading: Extra Unexpected Events
• More packets arrive before rules installed?
–Multiple packets reach the controller
packets
17
Reading: Extra Unexpected Events
• Solution: suppress extra events
–Programmer specifies “Limit(1)”
–Run-time system hides the extra events
not seen by
application
packets
18
Frenetic SQL-Like Query Language
• Get what you ask for
– Nothing more
– Nothing less
• SQL-like query language
– Familiar abstraction
– Returns a stream
– Intuitive cost model
• Minimize controller overhead
– Filter using high-level patterns
– Limit the # of values returned
– Aggregate by #/size of packets
Traffic Monitoring
Select(bytes) *
Where(in:2 & srcport:80) *
GroupBy([dstmac]) *
Every(60)
Learning Host Location
Select(packets) *
GroupBy([srcmac]) *
SplitWhen([inport]) *
Limit(1)
19
Composition: Multiple Modules
• Networks have multiple policies
–Routing
–Traffic monitoring
–Access control
• Challenges
–Common set of rules in the switches
–Processing the same packets
• OpenFlow API is not modular
–Programmer must combine the logic
20
Composition: Simple Repeater
Simple Repeater
def switch_join(switch):
# Repeat Port 1 to Port 2
p1 = {in:1}
a1 = [out:2]
install(switch, p1, DEFAULT, a1)
# Repeat Port 2 to Port 1
p2 = {in:2}
a2 = [out:1]
install(switch, p2, DEFAULT, a2)
Controller
1
2
When a switch joins the network, install two forwarding rules.
Composition: Web Traffic Monitor
Monitor “port 80” traffic
def switch_join(switch)):
# Web traffic from Internet
p = {in:2, srcport:80}
install(switch, p, DEFAULT, [])
query_stats(switch, p)
1
def stats_in(switch, p, bytes, …)
print bytes
sleep(30)
query_stats(switch, p)
2
Web traffic
When a switch joins the network, install one monitoring rule.
22
Composition: Repeater + Monitor
Repeater + Monitor
def switch_join(switch):
pat1 = {in:1}
pat2 = {in:2}
pat2web = {inport:2, srcport:80}
install(switch, pat1, DEFAULT, None, [out:2])
install(switch, pat2web, HIGH, None, [out:1])
install(switch, pat2, DEFAULT, None, [out:1])
query_stats(switch, pat2web)
def stats_in(switch, xid, pattern, packets, bytes):
print bytes
sleep(30)
query_stats(switch, pattern)
Must think about both tasks at the same time.
Composition: Frenetic is Modular
# Static repeating between ports 1 and 2
def repeater():
rules=[Rule(in:1, [out:2]),
Rule(in:2, [out:1])]
register(rules)
Repeater
# Monitoring Web traffic
def web_monitor():
q = (Select(bytes) *
Where(in:2 & srcport:80) *
Every(30))
q >> Print()
Repeater + Monitor
Monitor
# Composition of two separate modules
def main():
repeater()
web_monitor()
24
Composition: Reactive Run-Time
• Microflow-based
– Send first packet to
the controller
– Install rule if possible
• Check all policies
– Accumulate actions to
perform on packet
• Check all queries
– If no matches: install a
rule to handle remaining
packets of the flow
25
Composition: Proactive [POPL’12]
• Proactive, wildcard rules
– Keep packets in the “fast path”
• “Cross-product” of predicates
in:1
in:2
*
X
in:2 & srcport=80
*
=
in:1
in:2 & srcport=80
in:2
*
• Translate predicates into rules
– Convert each predicate to one or more rules
– Minimize to produce a smaller set of rules
• Reactive specialization
– Dynamically expanding the policy as packets arrive
26
Writing Policy: Avoiding Disruption
Writing Policy: Avoiding Disruption
Reasons
• Routine maintenance
• Unexpected failure
• Traffic engineering
• Fine-grained security
Invariants
• No forwarding loops
• No black holes
• Access control
• Traffic waypointing
Writing Policy: Traffic Engineering
• Shortest-path routing
–Controller computes shortest paths
–… based on preconfigured link weights
1
1
1
1
3
29
Writing Policy: Traffic Engineering
• Transient loop
–Update top switch to forward down
–… while bottom switch still forwards up
15
1
1
1
3
30
Writing Policy: Path for a New Flow
• Rules along a path installed out of order?
–Packets reach a switch before the rules do
packets
Must think about all possible packet and event orderings.
31
Writing Policy: Update Semantics
• Per-packet consistency
– Every packet is processed by
– … policy P1 or policy P2,
– … but not a mixture of the two
– E.g., access control, no loops
or blackholes during routing change
P1
P2
• Per-flow consistency
– Sets of related packets are processed by
– … policy P1 or policy P2,
– … but not a mixture of the two
– E.g., server load balancing, in-order delivery, …
Writing Policy: Policy Update
• Simple abstraction
– Update the entire configuration at once
– E.g., per_packet_update(P2)
P1
• Cheap verification
– If P1 and P2 satisfy an invariant
– Then the invariant always holds
• Run-time system handles the rest
P2
– Constructing schedule of low-level updates
– Applying optimizations to limit the number of rules
– Using only OpenFlow commands!
33
Writing Policy: Two-Phase Update
• Version numbers
– Stamp packet with a version number (e.g., VLAN tag)
• Unobservable updates
– Add rules for P2 in the interior
– … matching on version # P2
• One-touch updates
– Add rules to stamp packets
with version # P2 at the edge
• Remove old rules
– Wait for some time, then
remove all version # P1 rules
34
Writing Policy: Optimizations
• Avoid two-phase commit
– Naïve version touches every switch
– Doubles rule space requirements
• Limit scope of two-phase commit
– Affects only a portion of the traffic
– Affects only a portion of the topology
• Simple policy changes
– Extension: strictly adds paths
– Retraction: strictly removes paths
• Run-time system applies optimizations
35
Frenetic Abstractions
Policy Composition
Consistent
Updates
SQL-like
queries
OpenFlow
Switches
36
Ongoing Work
• Network virtualization
– Applications see abstract topology
– E.g., one big switch
37
Ongoing Work
• Network virtualization
– Applications see abstract topology
– E.g., one big switch
• Joint host-network management
– Measurement and control
– … through local host agent
38
Ongoing Work
• Network virtualization
– Applications see abstract topology
– E.g., one big switch
• Joint host-network management
– Measurement and control
– … through local host agent
• Policy transformation
– Spread rules over many switches
– E.g., distributed firewall/load-balancer
39
Related Work
• Programming languages
– FRP: Yampa, FrTime, Flask, Nettle
– Streaming: StreamIt, CQL, Esterel, Brooklet, GigaScope
– Network protocols: NDLog
• OpenFlow
– Language: FML, SNAC, Resonance
– Controllers: ONIX, Nettle, FlowVisor, RouteFlow
– Testing: MiniNet, NICE, FlowChecker, OF-Rewind,
OFLOPS
• OpenFlow standardization
– http://www.openflow.org/
– https://www.opennetworking.org/
40
Conclusion
• SDN is exciting
–Enables innovation
–Simplifies management
–Rethinks networking
• SDN is happening
–Practice: useful APIs and good industry traction
–Principles: start of higher-level abstractions
• Great research opportunity
–Practical impact on future networks
–Placing networking on a strong foundation
41
Concern
Assembly Languages
x86
NOX
Programming Languages
Java/ML
Resource
Management
Move values
to/from register
Declare/use
variables
Modularity
Unregulated
calling
conventions
Calling
conventions
managed
automatically
Consistency
Inconsistent
memory model
Consistent (?)
memory model
Portability
Hardware
dependent
Hardware
independent
Frenetic
Concern
Assembly Languages
Programming Languages
x86
NOX
Java
Frenetic
Resource
Management
Move values
to/from register
(Un)Install
policy
rule-by-rule
Declare/use
variables
Declare network
policy
Modularity
Unregulated
calling
conventions
Unregulated
use of network
flow space
Calling
conventions
managed
automatically
Flow space
managed
automatically
Consistent (?)
memory model
Consistent global
policies
Hardware
independent
Hardware
Independent
Consistency
Portability
Inconsistent
Inconsistent
memory model global policies
Hardware
dependent
Hardware
dependent
Thanks to My Frenetic Collaborators
Nate Foster
Rob Harrison
Dave Walker
Mike Freedman
Chris Monsanto
Alec Story
Mark Reitblatt
Josh Reich
44