Transcript TR41.4-05-08-008-M-TIA-921PerformanceEvaluation

TIA-921 (PN-3-0062) and ITU-T G.NIMM
Network Model for Evaluating Multimedia
Transmission Performance Over Internet Protocol
Jack Douglass, Spirent
Chair TIA TR30.3
August 2005
TIA TR41.4-05-08-008
TIA-921 (PN-3-0062) and ITU-T G.NIMM
Types of DUTs
IP Phones
POTS over IP
Media Gateways
MGC
User Agents
Call Agents
Media Servers
Video
Gatekeepers Application
Servers Edge Routers
Gateways
Modems
Fax
IP Network Impairment Emulator
Based on TIA-921 and ITU-T G.NIMM
Source Device
Under Test
Ethernet
IP Network
Impairment Emulator
Ethernet
Destination
Device Under
Test
TIA-921 (PN-3-0062) and ITU-T G.NIMM
Network Model for Evaluating Multimedia Transmission Performance
TIA-921 / ITU-T G.NIMM
Network Impairments and Parameters
that Affect Voice and Video Quality
•
•
•
•
•
•
•
•
•
Network Architecture
Types of Access Links
QoS controlled Edge Routing
MTU Size
Packet Loss (Frame Loss)
Out of order packets
One Way Delay (Latency)
Variable Delays (Jitter)
Background Traffic
(Occupancy)
•
•
•
•
•
•
Timing Drift
Route Flapping
Signaling protocol mismatches
Network faults
Link Failures
Voice Only Impairments
–
–
–
–
Echo
Voice coding algorithms
A/D and D/A Conversion
Noise – Circuit and External
• Video Only Impairments
– Video coding algorithms
– Variable and Fixed Frame Rates
– Video Complexity
TIA-921and ITU-T G.NIMM
IP Network Model
Source
Device A
LAN A
Local Access
B
Local Access
A
64 kbit/s
*128 kbit/s
256 kbit/s
1000BaseX
*384 kbit/s
* 100BaseT Switch
512 kbit/s
100BaseT Hub
*768 kbit/s
10BaseT
*T1 (1.536 kbit/s)
* WLAN (~4 Mbit/s) E1 (1.920 kbit/s)
---------------------E3 (34 Mbit/s)
Occupancy level
*T3 (44 Mbit/s)
Packet loss
ADSL (~256 kbit/s)
*Cable (~256 kbit/s)
Fiber (1-10 Gbit/s)
-------------------Occupancy level
QoS edge router
Core IP Network
Route flapping
One-way delay
Jitter
Packet loss
64 kbit/s
*128 kbit/s
256 kbit/s
*384 kbit/s
512 kbit/s
*768 kbit/s
*T1 (1.536 kbit/s)
E1 (1.920 kbit/s)
E3 (34 Mbit/s)
*T3 (44 Mbit/s)
ADSL (~2 Mbit/s)
*Cable (~3 Mbit/s)
Fiber (1-10 Gbit/s)
-------------------Occupancy level
QoS edge router
* Case used in impairment tables
LAN B
1000BaseX
* 100BaseT Switch
100BaseT Hub
10BaseT
* WLAN (~4 Mbit/s)
---------------------Occupancy level
Packet loss
Destination
Device B
TIA-921and ITU-T G.NIMM
IP Network Model
Direction of Test
Source
Dev ice A
Local
Access A
LAN A
LAN data rate
LAN occupancy
Local
Access B
Core IP Netw ork
Access data rates
Access occupancy
MTU size
Route f lapping
Link f ailure
One-way delay
Jitter
Packet loss
LAN B
Access data rates
Access occupancy
MTU size
• Typically test are run in one direction
• Tests may be run in both directions
• Impairments occur in both directions
LAN data rate
LAN occupancy
Destination
Dev iceB
TIA-921 and ITU-T G.NIMM
Test Profiles Based on QoS Classes
Service Test
Profiles
QoS
Class
(Y.1541)
Applications (Examples)
Node
Mechanisms
Well-Managed IP
Network
(Profile A)
0
High quality video and VoIP,
VTC (Real-time applications,
loss sensitive, jitter sensitive,
high interaction)
Strict QoS,
guaranteed no
over
subscription on
links
Constrained routing and
distance
Best-Effort
IP Network
(Profile B)
1
VoIP, VTC
(Real-time applications, jitter
sensitive, interactive)
Separate queue
with preferential
servicing, traffic
grooming
Less constrained routing
and distances
Unmanaged
IP Network,
Internet
(Profile C)
2
Lower quality video and
VoIP, signaling, transaction
data (highly interactive)
Separate queue,
drop priority
3
Transaction data, interactive
4
Short transactions, bulk data
(low loss)
Long queue,
drop priority
5
Traditional Internet
applications (default IP
networks)
Separate queue
(lowest priority)
Network Techniques
Constrained routing and
distance
Less constrained routing
and distances
Any route/path
Any route/path
• Statistically based models for different QoS Classes
TIA-921 and ITU-T G.NIMM
Test Profiles Based on QoS (Y.1541) Classes
Different test profiles for different Service Level Agreements (SLAs)
Impairment Type
Units
Range
Impairment Type
Units
Range
Impairment Type
Units
Range
Jitter
ms
0 to +/50
Jitter
ms
0 to +/75
Jitter
ms
0 to +/250
One Way Average
Latency
ms
20 to
60
One Way Average
Latency
ms
50 to
200
One Way Average
Latency
ms
50 to
800
Sequential Packet
Loss
# of
sequential
packets
Sequential
Packet Loss
# of
sequential
packets
2 to 5
Sequential
Packet Loss
# of
sequential
packets
2 to
500
< 10-3
Rate of
Sequential Loss
sec-1
< 10-1
Rate of Sequential
Loss
Rate of
Sequential Loss
sec-1
sec-1
0 to 2
0 to 20
0 to
0.05
Random Packet
Loss
%
Random Packet
Loss
Random Packet
Loss
%
%
0 to 0.1
0 to 20
0 to
0.001
Out of Sequence
Packets
%
%
Out of Sequence
Packets
%
Out of Sequence
Packets
Random
loss
only
Profile A
Well Managed Network
Impairment Ranges
Table 2
Profile B
Best Effort Managed Network
Impairment Ranges
Table 3
Profile C
Un-Managed Network
Impairment Ranges
Table 4
TIA-921 and ITU-T G.NIMM
8 Impairment Severity Combinations
Severity=>
Impairment
A
B
C
D
E
F
G
H
Units
Profile A LOO
%
50
30
15
5
0
0
0
0
Profile B LOO
%
5
25
30
25
10
5
0
0
Profile C LOO
%
5
5
10
15
20
25
15
5
Source Location (A) Parameters
LAN A Occupancy
%
1
2
3
5
8
12
16
20
Access A Occupancy
%
0
1
2
4
8
15
30
50
512
512
1508
1508
1508
1508
1508
1508
MTU A
bytes
Core Network Impairments
Route flap interval
seconds
0
3600
1800
900
480
240
120
60
Route flap D delay
ms
0
2
4
8
16
32
64
128
Delay
ms
4
8
16
32
64
128
256
512
Jitter
ms
5
10
15
20
30
50
75
100
Link fail interval
seconds
0
3600
1800
900
480
240
120
60
Link fail duration
ms
0
64
128
256
400
800
1600
3000
Packet loss
%
0
0.01
0.02
0.04
0.1
0.2
0.5
1
Destination Location (B) Parameters
Access B Occupancy
MTU B
LAN B Occupancy
%
bytes
%
0
1
2
4
8
15
30
50
512
512
1508
1508
1508
1508
1508
1508
1
2
3
5
8
12
16
20
Table 10: Impairment Severity Combinations
TIA-921 and ITU-T G.NIMM
LAN and Access Rates
Home Rates
LAN Rate
LOO
Mbit/s
%
Access Rate
LOO
Toward core
From core
kbit/s
kbit/s
Represents
%
Represents
4
75
802.11b, 10BaseT hub
128
768
40
Cable, ADSL
20
25
802.11g, 100BaseT hub
384
1536
50
Cable, ADSL
384
3000
10
Cable, ADSL
Table 5: LAN Rates for Home Locations
Table 7: Access Rates for Home Locations
Business Rates
LAN Rate
LOO
Mbit/s
%
Represents
4
20
802.11b, 10BaseT
20
20
802.11g, 100BaseT hub
100
60
100 BaseT switched, Gbit Ethernet
Table 6: LAN Rates for Business Locations
Access Rate
LOO
Toward core
From core
kbit/s
kbit/s
Represents
%
384
1536
40
ADSL entry
384
3000
15
ADSL premium
1536
1536
40
T1
43000
43000
5
T3
Table 8: Access Rates for Business Locations
LAN and Access Rates (New Revision of Standard/Recomendation) will change as network conditions change
133 LAN/Access Rate Combinations
Testing with Fixed Levels of IP Impairments
Fixed
Packet
Jitter
Fixed
Jitter
&
Packet
Loss
• IP Impairment network emulators exist today that provide
fixed or semi-fixed levels of IP impairments.
• Real IP Network Impairments Vary Over Time significantly
over time and are bursty in nature
TIA-921 and ITU-T G.NIMM
Statistically Based IP Network Impairment Conditions (ICs) based on SLAs
Call
Quality
Metric
IC– 96A
Probability
(LOO = 0.00256% )
Impairment
Condition
TIA-921 and ITU-T G.NIMM
Statistically Based IP Network Impairment Conditions (ICs) based on SLAs
Call
Quality
Metric
IC– 96E
Probability
(LOO = 0.01025% )
Impairment
Condition
TIA-921 and ITU-T G.NIMM
Statistically Based IP Network Impairment Conditions (ICs) based on SLAs
Call
Quality
Metric
Route Flap
Link Failure
Impairment
Condition
IC– 96H
Probability
(LOO = 0.00256%)
TIA-921 and ITU-T G.NIMM
Statistically Based IP Network Impairment Conditions (ICs) based on SLAs
• Each Impairment Condition has a Likely-hood of Occurrence
(LOO)
– Based on real IP Network Statistics
– Based on Network Architecture
– Based on Classes of Service
• Definitive set of test conditions
– 1064 Impairment Combinations
– Tests may be automatically
– Typically 2 min per Impairment Condition
• Approximately 1.5 days to run complete set of tests
– Impairment Condition may be correlated with appropriate quality
measurement
• Voice Quality Metric (MOS, PESQ, PSQM, R-Factor)
• Video Quality Metric
TIA-921 and ITU-T G.NIMM
Example of Network Model Coverage (NMC) Curve
Mean Opinion Score - Listening Quality vs. TIA-921 Network Model Coverage
(G.711 PLC, 40 ms jitter buffer)
4.5
4
3.5
MOS-LQ
3
2.5
Test Profile A
Test Profile B
Test Profile C
2
1.5
1
0.5
0
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% Network Model Coverage
Test Results from 1064 Statistically based IP Network Impairment Combinations
TIA-921 and ITU-T G.NIMM
Example of Network Model Coverage (NMC) Curve
R-Factor - Listening Quality vs. TIA-921 Network Model Coverage
(G.711 PLC, 40 ms. jitter buffer)
100
90
80
70
R-LQ
60
Test Profile A
Test Profile B
Test Profile C
50
40
30
20
10
0
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% Network Model
Test Results from 1064 Statistically based IP Network Impairment Combinations
History of TR30.3
Network Model Standards
• TIA/EIA 496A-1989: Interface Between Data Circuit Terminating
Equipment (DCE) and the Public Switched Telephone Network
– Included Network Model for Evaluating Modem Performance
• TIA/EIA TSB 37A-1994: Telephone Network Transmission Model
for Evaluating Analog Modem Performance, which became ITU-T
Recommendation V.56bis-1995
• EIA/TIA TSB 38-1994: Test Procedures for Evaluation of 2-Wire 4
Kilohertz Voice Band Duplex Modems, which became ITU-T
Recommendation V.56ter-1996
• ANSI/TIA/EIA 3700-1999: Telephone Network Transmission
Model for Evaluating Analog Modem Performance
• ANSI/TIA/EIA 793 -2000: North American Telephone Network
Transmission Model for Evaluating Analog Client and Digitally
Connected Server Modems
• ANSI/TIA 876 – 2002: North American Network Access
Transmission Model for Evaluating xDSL Modem Performance
TR30.3 Liaison Activities
• Liaisons activities between TR30.3 and appropriate
committees to develop TIA-921 (PN-3-0062), Network
Model for Evaluating Multimedia Transmission
Performance Over Internet Protocol
– TR41.4
– TR45.5
– ITU-T SG12
– ITU-T SG9
– ITU-T SG15
• Working with Operating Companies and Service
providers to provide statistically accurate IP network
impairment model
TIA-921
Target Audience for Network Model
•
•
•
•
•
•
Network Equipment Manufacturers
Operating Companies
Service Providers
Design Engineers
Test houses
Magazines and product reviewers