Transcript MPLS-TE Doesn't Scale

MPLS-TE Doesn’t Scale
Adrian Farrel
Old Dog Consulting
[email protected]
www.mpls2007.com
OLD DOG CONSULTING
Is the Sky Falling?

The only way to get your attention is to be alarmist

MPLS-TE is perfectly functional in today’s networks

But:


MPLS-TE will not scale indefinitely
The problem is the well-known “full mesh” or
“n-squared” problem

The number of LSPs scales as the square of the number
of PEs
2
OLD DOG CONSULTING
What Do We Want to Achieve?

MPLS-TE is an important feature for many SPs




Allow traffic to be groomed
Optimize use of network resources
Provide quality of service guaranties
Carriers look to provide edge-to-edge tunnels across
their core networks





Differentiated Services
VPNs
VLANS and pseudowires
Multimedia content distribution
Normal IP traffic
3
OLD DOG CONSULTING
What is the Scope of the Problem?

Consider a service provider network with 1000 PEs





This is not outrageously large
Such a network may be broken into areas or ASes
Consider a full mesh of PE-PE TE-LSPs
Consider parallel tunnels for different services, QoS
levels, and for protection
May give rise to multiples of 999,000 LSPs in the core


What is the capacity of a core LSR?
What is the capacity of a management system?
4
OLD DOG CONSULTING
What Are the Scaling Limits?

Management



NMS
 How many LSPs can the NMS process
Management protocols
 Reporting on large numbers of LSPs may overload the
management network
LSR issues



Memory capacity
 Per LSP data requirements
CPU capacity – largely an RSVP-TE protocol issue
 Degradation of LSP setup times
 Soft state addressed by Refresh Reduction
MPLS forwarding plane
 Number of labels (Only 1048559 per interface)
5
OLD DOG CONSULTING
The Snowflake Topology


Example network for analysis
Meshed core of P nodes





PE
Called P1 nodes
Each Pi+1 node connected to
P2
just one Pi node
PE nodes connected to just one
P1
Pn node
Well-defined connectivity and
symmetry allows many important
metrics to be computed
Number of levels & number of nodes per level may be varied




We
We
We
We
can
can
can
can
vary
vary
vary
vary
the
the
the
the
number of P1 nodes
ratio of Pi+1 to Pi
value n
number of PE nodes per Pn node
6
OLD DOG CONSULTING
Analysing the Snowflake Topology

Define





Discover




Pn a node at the nth level (level 1 is core)
Sn the number of nodes at the nth level
Mn the multiplier at the nth level (how many Pn+1 nodes are connected to
a Pn node)
Ln number of LSPs seen by a Pn node
LPE = 2*(SPE - 1)
L2 = M2*(2*SPE - M2 - 1)
L1 = M1*M2*(2*SPE – M2*(M1 + 1))
Practical numbers





S1 = 10, M1 = 10, and M2 = 20
SPE = 2000
LPE = 3998
L2 = 79580
L1 = 756000
7
OLD DOG CONSULTING
The Ladder Topology


Example network for analysis
Core of P1 nodes looks
like a ladder


Symmetrical trees subtended
to core



Similar to many national
networks
Each Pi+1 node connected to just one Pi node
Each PE node connected to just one P node
Again:


Well-defined connectivity and symmetry allows many important
metrics to be computed
Number of levels & number of nodes per level may be varied
8
OLD DOG CONSULTING
Analysing the Ladder Topology

Same definitions as for snowflake network


Discover




E the number of subtended edge nodes (PEs) to each spar-node
(E = M1*M2)
LPE = 2*(SPE - 1)
L2 = 2*M2*(SPE - 1) - M2*(M2 - 1)
L1 ≈ E*E*S1*S1/2 + E*E*S1 + 3*E*E - E*M2
Practical numbers






S 1 = 10, M1 = 10, and M2 = 20
E = 200
SPE = 2000
LPE = 3998
L2 = 79580
L1 = 2516000
9
OLD DOG CONSULTING
Option 1 – Solve a Different Problem!

If a full mesh of PE-PE LSPs is too big, don’t build it!




The suggestion is to build a full mesh of Pn-to-Pn LSPs, and
perform routing or routing-based MPLS between Pn and PE
Scaling improves from O(10002)
to O(1002)
But we lose functionality




This is the bottom line if we don’t fix the problem
Why did we want a PE-PE mesh?
How do we handle private address spaces?
What if the traffic is not routable?
This may simply not be good enough to provide the function
10
OLD DOG CONSULTING
Option 2 – LSP Hierarchies

Well-known, core MPLS function





Label stacks
Forwarding Adjacencies (RFC 4206)
Configured or automatic grooming
Possible to build a full or partial
mesh of hierarchical tunnels
For example connect all P2 nodes


Each P2 node must encapsulate each PE-PE LSP in the
correct tunnel
Each P1 node only sees the P2-P2 tunnels
11
OLD DOG CONSULTING
Scaling Properties of Hierarchies - Snowflake



Note that PE-PE tunnels don’t help
P1-P1 tunnels are also no benefit (core is fully meshed)
P2 nodes see all PE-PE LSPs and new tunnels


Situation at P1 nodes is much better


L2 = M2*(2*SPE - M2 - 1) + 2*(S2 - 1)
L1 = M1*(2*S2 - M1 - 1)
Numbers (S1 = 10, M1 = 10, and M2 = 20)
SPE
LPE
L2
L1

Flat
2-Level
2000
3998
79580
756000
Hierarchy
2000
3998
79778
1890
Maybe insert another layer (P3 ) to increase the scaling?

L3 remains high
12
OLD DOG CONSULTING
Scaling Properties of Hierarchies - Ladder


Note that PE-PE tunnels don’t help
But P1-P1 tunnels are good because core is not fully-meshed


Another level of hierarchy is also possible


L1 ≈ S1*S1/2 + 2*S1 + 2*E*E*(S1 - 1) - E*M2 - 2
Add a mesh of P2-P2 tunnels
 L1 = S1*S1/2 + 2*S1 + 2*M1*M1*S1 - M1(M1 + 1) – 2
 L2 = 2*M2*(S(PE) - 1) - M2*(M2 - 1) + 2*(S(1)*M(1) - 1)
Numbers (S 1 = 10, M1 = 10, and M2 = 20)
Flat
SPE
LPE
L2
L1
2000
3998
79580
2516000
2-Level
Hierarchy
2000
3998
79580
716060
13
3-Level
Hierarchy
2000
3998
79778
1958
OLD DOG CONSULTING
Issues and Drawbacks for Hierarchies

Scaling is not good enough!


Management burden


Plan and operate a secondary mesh
 Effectively the same burden as managing PEs or a layered
network
 Possible to consider auto-mesh techniques
Fast Reroute protection is a problem


Impact on layer adjacent to PEs is negligible
 Actually impact is slightly negative
FRR struggles to protect tunnel end-points
Not obvious how to arrange the hierarchy when the network is
not symmetrical

E.g., some PEs closer to the core
14
OLD DOG CONSULTING
Option 3 – Multipoint-to-Point LSPs





LSPs merge automatically as they converge on the destination
Reduces the number of LSPs toward the egress
Other LSP properties (e.g.,
bandwidth) must be cumulative
TE is still possible, but
de-merge is not considered
Should count “LSP state” not number of LSPs


New definition
 Xn the amount of LSP state
held at each Pn node
For flat and hierarchical networks:
 Each LSP adds one state at ingress or egress
 Each LSP adds two states at each transit node
15
OLD DOG CONSULTING
Scaling Properties of MP2P LSPs - Snowflake
XPE = 2*(SPE - 1)
X2 = SPE*(M2 + 1)
X1 = M1*M2*(S1 - 2) + SPE*(M1 + 1)
 Numbers (S1 = 10, M1 = 10, and M2 = 20)
SPE
XPE
X2
X1
Flat 2-Level Hierarchy
2000
2000
3998
3998
159160
159358
1512000
3780
16
P2MP
2000
3998
42000
23600
OLD DOG CONSULTING
Scaling Properties of MP2P LSPs - Ladder
XPE = 2*(SPE - 1)
X2 = (M2 + 1)*S1*E
X1 ≤ (4 + M1)*S1*E - M1*E
 Numbers (S1 = 10, M1 = 10, and M2 = 20)
Flat
SPE
XPE
X2
X1
2000
3998
159160
5032000
2-Level
Hierarchy
2000
3998
159160
1433998
3-Level
Hierarchy
2000
3998
159358
3898
17
P2MP
2000
3998
42000
26000
OLD DOG CONSULTING
Issues and Drawbacks for MP2P LSPs

Clear scaling benefits



Better than flat networks
Only thing that improves the situation adjacent to PEs
But…






Data plane support
 This will only ever be a packet/frame/cell technology
Control plane support
 RSVP does have MP2P support
 RSVP-TE features not yet specified or implemented
De-aggregation and disambiguation
 May be necessary to use label stack so that egress can detect sender
of data
OAM may be more complex and require source labels
New management applications needed
FRR still to be designed
18
OLD DOG CONSULTING
Other Topics for Investigation

Cost-effectiveness of the network




Fast Reroute



Revenue only generated by PEs
K = S(PE)/(S(1)+S(2) + ... + S(n))
Many ways to improve scaling reduce cost-effectiveness
What are the implications of FRR to scaling?
Can scaling contributions be designed that can be protected by
FRR?
Point-to-multipoint


What are the scaling properties of P2MP MPLS-TE?
Domain boundaries (in particular AS boundaries)


Boundaries such as at area and AS borders cause constrictions
How can we reduce the number of LSPs seen by ABRs and ASBRs?
19
OLD DOG CONSULTING
Conclusions, Next Steps, and References

MPLS-TE is not a scaling issue today

But it won’t scale arbitrarily

We need to plan now for tomorrow’s scalability
Hierarchical LSPs are not as good as expected
MP2P LSPs may offer a better solution
More research and implementation is needed

draft-ietf-mpls-te-scaling-analysis-01.txt






Seisho Yaukawa (NTT)
Adrian Farrel (Old Dog Consulting)
Olufemi Komolafe (Cisco Systems)
20
OLD DOG CONSULTING
Questions?
[email protected]
21
OLD DOG CONSULTING