Transcript Nairobi, Kenya, 26 th July 2010

ITU-T Workshop on
Delivering Good Quality Telecommunication Service
in a Safe Environment in Africa
(Nairobi, Kenya, 26th July 2010)
Benchmarked Key Performance
Indicators
Crisphine J. Ogongo,
Engineer/Compliance
Communications Commission of
Kenya
Nairobi - Kenya 26 - 27 July 2010
Content
ITU/ETSI Standards
QoS Regulation And Comparison
With Int’l Stds
Conclusion
Questions
Nairobi - Kenya 26 - 27 July 2010
ITU-T/ETSI STANDARDS
Nairobi - Kenya 26 - 27 July 2010
Definition
Quality of service (QoS):
The collective effect of service
performances, which determine the
degree of satisfaction of a user of the
service (ITU-T E.800)
Nairobi - Kenya 26 - 27 July 2010
What QoS Really Is!
Network Performance Parameters translated
to;
system design, configuration, operation and
maintenance
Quality of service parameter influenced by;
Statistics e.g «Call Block Rate»
User/customer requirements
Individual experience e.g «inaccessibility»-
User opinion/requirements feed back in to
the network planning process to alter
planned performance and/or practical
operational standards
Nairobi - Kenya 26 - 27 July 2010
ITU-T QOS Parameters with Targets
Call Set Up Time/Post Dialing Delay
Call Release Delay
Answer Signal Delay
End to End Blocking
Handover Success Rate/Unsuccessful
Handover
Speech Quality
Multimedia QoS
Nairobi - Kenya 26 - 27 July 2010
Call Set up time
Call type
F-M
M-F
M-M
Authentication/
ciphering
Paging/alerting
0.0
2.5
2.5
4.0
0.0
4.0
Routing number transfer
2.0
0.0
2.0
Local connection
3.0
3.0
3.0
Toll connection
5.0
5.0
5.0
International connection
8.0
8.0
8.0
Post-selection delay
TOTAL
Definition
Time interval between the end of
dialing by the user and the
reception
by
him
of
the
appropriate tone or recorded
announcement, or the abandon of
the call without tone.
PLMN
F-M
M-F
PSTN
M-M
Normal Load
High Load
Local connection
9
5.5
Mean
11.5 3 s
Toll connection
11
7.5
13.5 5s
8s
7.5 s
12 s
International connection
14
10.5
16.5 8 s
11 s
12 s
16.5 s
Nairobi - Kenya 26 - 27 July 2010
95%
6s
Mean
4.5 s
95%
9s
Answer Signal Delay
PLMN
Connection
Call type
F-M
M-F
Definition
M-M
Local connection
1.0
1.0
1.25
Toll connection
1.75
1.75
2.0
International connection
2.75
2.75
3.0
PSTN
Time interval between the
establishment
of
a
connection between calling
and called users, and the
detection of an answer
signal at the originating
exchange.
Normal Load
High Load
Mean
Mean
95%
95%
Local connection
0.75 sec 1.5 sec 1.0 sec 2.0 sec
Toll connection
1.5 sec
3.0 sec 2.0 sec 4.0 sec
International connection
2.0 sec
5.0 sec 3.3 sec 6.5 sec
Nairobi - Kenya 26 - 27 July 2010
Call Release Delay
Call release
delay (secs)
Call type
Calling, or called,
party clears
Definition
Connection Type
F-M, M-F, M-M
PSTN
1.0 sec
0.4 – 1sec
Call release delay is defined as the time
interval from the instant the first bit of the
DISCONNECT message is passed by the user
terminal which terminated the call to the
access signaling system, until the last bit of
the RELEASE message is received by the same
terminal (indicating that the terminals can
initiate/receive a new call).
Nairobi - Kenya 26 - 27 July 2010
End to End Blocking
Definition: - The probability that any call
attempt will be unsuccessful due to a lack
of network resources
PLMN
PSTN
1990
Improvement
On radio Channel
5-10%
1%
PLMN to Fixed
1%
0.5%
Normal Load
High Load
Local
2%
3%
Toll Connection
3%
4.5%
Int’l Connection
5%
7.5%
Nairobi - Kenya 26 - 27 July 2010
Unsuccessful Handovers
Is the probability that a handover attempt fails because of
lack of radio resources in the target cell, or because of a lack
of free resources for establishing the new network connection.
The failure condition is based either on a specified time
interval since the handover request was first issued or on a
threshold on signal strength
Call type
Probability of unsuccessful land cellular
handover
Nairobi - Kenya 26 - 27 July 2010
F-M, M-F, M-M
0.5%
Speech Quality
PESQ: - Perceptual evaluation of speech quality
ITU-T P.862
An objective method for end-to-end speech
quality assessment of narrow-band telephone
networks and speech codecs
PESQ compares an original signal X(t) with a
degraded signal Y(t) that is the result of passing
X(t) through a communications system. The
output of PESQ is a prediction of the perceived
quality that would be given to Y(t) by subjects
in a subjective listening test(6).
Provides Speech Quality raw values -0.5 to 4.5
Nairobi - Kenya 26 - 27 July 2010
Speech Quality
PESQ: - Perceptual evaluation of speech quality ITU-T P.862
An objective method for end-to-end speech quality assessment
of narrow-band telephone networks and speech codecs
Original
Signal X (t)
Communication
Network
Y(t)
PESQ
Algorithm
• perceived
Speech
quality
Original Signal X(t)
Original Signal X(t)
Subjects
in
Listening
Test
Nairobi - Kenya 26 - 27 July 2010
Perceived
Speech
quality
Speech Quality cont’d
ITU-T P.862.1 - Mapping function for transforming P.862 raw result scores
to MOS
The mapping ensures a domain rescaling from –0.5 ... 4.5 to 1.02 ... 4.56
5
4.5
4
Mapped P.862
3.5
3
2.5
2
1.5
1
0.5
0
–1
0
1
2
P.862
Nairobi - Kenya 26 - 27 July 2010
3
4
5
P.862.1_F1
Speech Quality Cont’d
Speech quality depends on Transmission
Rating Factor R
The R-value is a measure of a quality perception to be expected by the
average user when communicating via the connection under consideration
high
quality
medium
quality
low
quality
Area not
recommended
linear quality scale
100
90
80
70
60
50
Overall
Rating "R"
T1211030-99
Nairobi - Kenya 26 - 27 July 2010
Speech Quality Cont’d
Transmission Rating Factor
conversion to MOS equivalent
R-value range
Equivalent
MOS
Values
90  R < 100
5
80  R < 90
4
70  R < 80
3
60  R < 70
2
50  R < 60
1
Conversion of R-values into MOS
Speech
transmissio
n
quality
category
Excellent
Good
Fair
Bad
Poor
User satisfaction
Very satisfied
Satisfied
Some users dissatisfied
Many users dissatisfied
Nearly all users dissatisfied
MOS = MOS = 1+0.035R + R(R-60)(100-R)7.10⁻⁶
Nairobi - Kenya 26 - 27 July 2010
MOS User Satisfaction Levels
Typical values used in
RValue
MOS
Value
Comment
Country
R≤0
1
Nearly all users dissatisfied
R = 55
2.835
Many users dissatisfied
Kenya
R = 60
3.1
Some users dissatisfied
Nigeria/Kenya in 3yrs
R = 70
3.597
Some users dissatisfied
R = 80
4.024
Satisfied
R≥
100
4.5
Very satisfied Satisfied
MOS = MOS = 1+0.035R + R(R-60)(100-R)7.10⁻⁶
Nairobi - Kenya 26 - 27 July 2010
Data/Multimedia Parameters Critical
to Users
Delay variations
Occurs at the transport layer in packetized data systems
due to the inherent variable arrival times of individual
packets
Solved through buffering
Delay
the time taken to establish a particular service from the
initial user request and the time to receive specific
information once the service is established.
Packet Loss
Of packets or bits during transmission
Includes coding degradation
Nairobi - Kenya 26 - 27 July 2010
Multimedia (Audio and Video) QoS
Parameters and targets
1-way
delay
Delay
Variation
Information
Loss
(Packet
Loss RatioPLR)
Audio Conversational Voice
<150 ms
< 1 ms
< 3%
Audio Voice Messaging
< 1 ms
< 3%
High Quality Streaming Audio
<1s
playback
< 10 s
<< 1 ms
< 1%
Videophone (2-way)
< 150 ms
< 1%
Videophone (1-way)
< 10 s
< 1%
KPI
Nairobi - Kenya 26 - 27 July 2010
Multimedia QoS Parameters
KPI
Symmetry
T-Values
Delay
Web-browsing
– HTML
1-way
~10 KB
Bulk data
transfer/retrieval
Transaction services –
high priority e.g.
e-commerce
Command/control
1-way
2-way
10 KB-10
MB
< 10 KB
P< 2 s /page
A<4s
/page
P < 15 s
A < 60 s
P<2s
A< 4 s
2-way
~ 1 KB
< 250 ms
Still image
1-way
< 100 KB
Interactive games
2-way
< 1 KB
P < 15 s
A < 60 s
< 200 ms
Telnet
2-way
< 1 KB
< 200 ms
E-mail (server access)
1-way
< 10 KB
E-mail (server to
server transfer)
1-way
< 10 KB
P< 2 s
A< 4 s
Can be
several
minutes
Nairobi - Kenya 26 - 27 July 2010
No delay variation or
information Loss
Key
P-preferred
A-Acceptable
T-typical
QOS REGULATION IN AFRICA AND
COMPARISON WITH INTERNATIONAL
STANDARDS
Case studies
Kenya
Nigeria
Uganda
Tanzania
Nairobi - Kenya 26 - 27 July 2010
Status of Country QoS Regulation
Nigeria
Tanzania Uganda Kenya
QoS regulated
√
√
√
√
QoS Parameters and Targets
√
√
√
√
Regulations
x
√
x
x
Guidelines
√
x
√
X
Licenses
x
x
x
√
Objective parameters
√
√
√
√
Subjective Parameters
√
√
√
√
PSTN Network
x
√
√
√
Cellular Mobile Network
√
√
√
√
Internet
x
√
√
√
Data (Leased Lines)
x
√
√
x
Nairobi - Kenya 26 - 27 July 2010
Objective KPIs and Targets
Benchmarks
ITU
Targets
Kenyan
Targets
Nigeria
Tanzania
Uganda
Completed Call (%)
-
90
90
>99
99
Speech Quality (MOS)
%age with Good SQ
4.0 ≤
MOS ≥
4.5
95%>2.7
& 3.1 in 3
yrs
98 >2.0
MOS
>95
95
2
2
<3
2
-
2
KPI
Call Drop Rate (%)
-
Call Block Rate (%)
1
10
-
Call Set Up Time (s)
varied
13.5
< 10
90
90
-
-
Call Set Up Success Rate
(%)
-
Handover Success Rate (%)
99.5%
85
90
-
-
Call release delay
1s
-
-
2s
-
Multimedia (IP) QoS
Varied
ITU
-
-
-
Nairobi - Kenya 26 - 27 July 2010
Subjective QoS parameters
Account complaint rate
Account complaint resolution time
Disconnection complaint rate
Disconnection complaint resolution time
Miscellaneous complaint rate
Miscellaneous complaint resolution time
Fault report rate
Fault repair time
Billing Accuracy
Service supply time
Call centre answer success ratio
Call centre answer time
Complaint resolution
Nairobi - Kenya 26 - 27 July 2010
Conclusion
ITU-T recommendations have covered many
QoS issues but very few targets have been
specified
Where ITU-T targets are specified, they
should be held as minimum thresholds not
negotiable downwards
Many regulators monitor QoS parameters
Targets adopted by regulators vary widely
Nairobi - Kenya 26 - 27 July 2010
Questions
How many regulators monitor QoS
performance of providers?
What are the KPI’s monitored?
How comprehensive are KPIs in addressing
customers requirements?
How do the targets compare with
international standards or best practices?
Is there need for African Regulators to
benchmark and adopt similar and adequate
parameters and targets?
Nairobi - Kenya 26 - 27 July 2010
Thank You
Nairobi - Kenya 26 - 27 July 2010