Transcript PPT Version

Benchmarking Methodology
for Routing Experts 101
Al
Morton
bmwg chair
Crash Course Syllabus





Who we are
What we do
What we’ve accomplished
What we’re working on that might
interest you
Where we’re headed…
2
Who are we?
3
What we do in BMWG
Attributes of a BENCHMARK
 a standardized test that serves as a basis for
performance evaluation and comparison.
 Unambiguous method, repeatable
 Reveal valuable characteristics to users and
provide insights to improve internetworking
technology
 that permit cross-vendor comparison with
externally observable measurements
4
Lab Test Configurations:
DUT=Device Under Test
RFC 2544
Benchmarking Methodology
March 1999
+------------+
|
|
+------------| tester
|<-------------+
|
|
|
|
|
+------------+
|
|
|
|
+------------+
|
|
|
|
|
+----------->|
DUT
|--------------+
|
|
+------------+
Figure 1
+--------+
+------------+
+----------+
|
|
|
|
|
|
| sender |-------->|
DUT
|--------->| receiver |
|
|
|
|
|
|
+--------+
+------------+
+----------+
Figure 2
5
What BMWG’s
Benchmarks are NOT:



Functional tests
Evaluated with PASS/FAIL criteria
Performed on live networks, or performed
with live network traffic
– Assigned IPv4 address space for Benchmarking
198.18.0.0/15 - This block has been allocated for
use in benchmark tests of network interconnect
devices. Its use is documented in [RFC2544].
– About to request dedicated IPv6 space
– Want to talk Live Network Measurements?
PMOL meets tomorrow..
6
Benchmarking Past






RFC 1242, Benchmarking Terminology
for Network Interconnection Devices
LAN Switches, Firewalls, ATM, F.Relay
IP Multicast
Single Router OSPF Control Plane
Convergence
BGP Control Plane Converge. (Terms)
Resource Reservation (Terms)
7
Benchmarking Present

IGP Dataplane Convergence Time
– More than the usual number of DISCUSSs


Accelerated Stress Benchmarking
Protection Switching Technologies
– Methodology for MPLS-FRR


IPv6 at Publication Requested
IPsec devices
8
Work Proposal Summary Matrix
Work Area
>
Criteria \/
Proposal
In Scope of
Charter?
(acm)
Draft(s)
SIP
Mcast VPN
Scalability
LDP
Converg
MPLS
BFD
WLAN
switch
RSVPTE
Y
Y
Y
Y
Y
Y
Y
Y
Y
No
Overlap
w/802.11
T
Y
1+
Draft
gives
motivat.
1+1
Draft
gives
motivat.
Y
1+1
Sig. Support
at meetings
Sig. Support
on List
Testing to
support Dev?
Y
1+
Y
1+1
Y
Interest
when first
proposed
9
How can You help the
good old BMWG?



Review Drafts!
Early Review >> Late Review
Establish a more formal process?
10