Transcript voice quality measurment in VoIP networks

Voice over IP and Voice
Quality Measurement
Dr. Lingfen Sun
School of Computing, Communications and
Electronics
University of Plymouth
Outline of Talk




Introduction
VoIP Networks
What is QoS or Perceived QoS?
How to Measure/Predict Voice Quality?



Subjective
Objective (intrusive and non-intrusive methods)
QoS Prediction and Control Research in
Plymouth
4/2/2005
SoCCE, UoP
2
Introduction – the problem

Internet Protocol (IP) networks



4/2/2005
On a steep slope of innovation – long term carriers
of all traffic including voice traffic.
IP is now the “universal” communications protocol
because it facilitates convergence of networks and
the ability to offer multiple services on the same
networks.
Not designed to carry real-time traffic, such as
voice and video, because of their variable
characteristics (e.g. delay, delay variation and
packet loss) . These have adverse effects on voice
quality.
SoCCE, UoP
3
Introduction – Voice Quality in IP
networks

User perceived quality is the key QoS metric
in VoIP applications - The end-user of a VoIP
service expects:



voice quality to be as good as in traditional
networks, and
the service to be as reliable.
This is not the case at present. This makes it
necessary to be able to predict/measure, and
if appropriate, control voice quality in order to
deliver the desired QoS.
4/2/2005
SoCCE, UoP
4
VoIP Network and Perceived QoS
IP phone
Gateway
Gateway
SCN
IP Network
IP softphone
SCN
SCN: Switched
Communication
Networks (PSTN,
ISDN, GSM …)
Network QoS
Perceived QoS


Network QoS
Perceived QoS is measured from ‘mouth to ear’, i.e. end-to-end and
depends on the performance of IP network and terminal/gateway.
4/2/2005
SoCCE, UoP
5
VoIP – New Applications
Mobile
network
PSTN
IP Network
/MPLS
MGW
GW
Dual-mode
handset
IP access
network
AP
DSLAM
VoWLAN
IAD
IAD: Integrated Access Device
DSLAM: DSL Access
Multiplexer
MGW: Media Gateway
MPLS: Multi-protocol Label
Switching
4/2/2005
Enterprise LAN
VoDSL
SoCCE, UoP
6
VoIP Protocol Stack
Application layer
Audio
/video
RTP RTCP SIP
H.323
Transport layer
UDP
TCP
Network layer
IP
Physical layer
e.g. Ethernet/SDH
4/2/2005
SoCCE, UoP
7
What is QoS?



The ISO standard defines QoS as a concept
for specifying how “good” the offered
networking services are. QoS can be
characterised by a number of specific
parameters.
For Multimedia Communication System
(MCS), QoS concept can be extended to
“User QoS” or “Perceived QoS”.
For VoIP, Perceived QoS – user perceived
voice quality (e.g. MOS)
4/2/2005
SoCCE, UoP
8
Factors affect voice quality
End-to-end perceived voice quality (MOS)
Sender
Receiver
Encoder Packetizer
IP Network
Depacketizer
Jitter
buffer
Decoder
Voice
source
Voice
receiver
coding distortion
codec delay
delay
packet loss
network delay
delay
buffer-delay
buffer-loss
jitter
codec
impairment
delay
• Other impairments: echo, sidetone, background noise …
• Other factors: language, gender, FEC, packet loss concealment
4/2/2005
SoCCE, UoP
9
Inter-relationships between the QoS
Parameters [1]
Network Packet
Loss
Network Jitter
Overall
Packet
Loss
Jitter
Buffers
4/2/2005
Perceived
Quality
Overall
Delay
Network Delay
Network Factors
Codec
Performance
Application Factors
SoCCE, UoP
QoS Service
Level
10
QoS parameters [1]
QoS Service Class
4/2/2005
SERVICE
Codec Performance, VAD, Frames per
Packet, Jitter Buffer, Codec Delay,
FEC (Redundancy)
APPLICATION
Max Packet Loss, Max Mean delay,
Max Delay Variation
TRANSPORT
SoCCE, UoP
11
Key QoS parameters and how they arise

Packet Loss




End-to-end Delay





Network packet loss (as a result of congestion or
rerouting in the IP network)
Late arrival loss (dropped at receiver)
Link failures and system errors.
Network delay (transmission and queuing delay)
Buffer delay
Codec processing delay
Packetizing/depacketizing delay
Jitter (delay variation)

4/2/2005
Caused by queuing delays within the IP network
SoCCE, UoP
12
Delay impact on multimedia quality [7]
Interactive
Responsive
Timely
Non-critical
Packet Loss
5%
Conversational
voice and video
0%
100 msec
Voice/video
messaging
1 sec
Streaming
audio/video
10 sec
Fax
Delay
100 sec
For VoIP applications, delay < 150 ms, imperceptible,
delay > 400 ms, quality unacceptable for most users.
4/2/2005
SoCCE, UoP
13
How to Enhance QoS?

Application-level QoS mechanisms




Packet loss compensation (e.g. FEC, loss
concealment)
Jitter compensation (e.g. buffer algorithms)
Adaptive source coding …
Network-level QoS mechanisms



4/2/2005
How to guarantee IP network performance
Diffserv (Differentiated Services)
Intserv (Integrated Services) …
SoCCE, UoP
14
How to Measure Voice Quality?

Why need to measure voice quality?


For QoS monitoring and/or control
purposes to ensure that the technical and
commercial requirements (e.g. SLA) are
met.
How to measure voice quality?


4/2/2005
Subjective methods (e.g. MOS)
Objective methods (e.g. PESQ or E-model)
SoCCE, UoP
15
Subjective or objective
measurement

Subjective Voice Quality Measurement




Subjective listening tests by a group of people
Provides a benchmark for objective test methods
Expensive and time-consuming
Objective Speech Quality Measurement



4/2/2005
Repeatable, automatic, and predicts subjective score
Suitable for online quality measurement/monitoring
Can be used for intrusive and Non-intrusive
measurements.
SoCCE, UoP
16
Voice quality measurement
MOS-LQ
MOS-LQ
Reference speech
Intrusive
Measurement
(e.g. PESQ)
Gateway
Gateway
SCN
SCN: Switched Comm. Networks
(PSTN, ISDN, GSM …)
Degraded speech
SCN
IP Network
(e.g. loss, delay, jitter)
Non-intrusive
Measurement
(parameter-based
e.g. E-model)
Non-intrusive
Measurement
(signal-based
e.g. P.563)
MOSc
MOS-LQ
MOS-LQ: MOS-Listening quality
MOSc: Conversational MOS score
4/2/2005
SoCCE, UoP
17
Voice quality measurement (cont.)
Voice quality
measurement
Calibration
Subjective
methods
Non-intrusive
methods
Parameter-based
methods
4/2/2005
Objective
methods
Intrusive
methods
Signal - based
methods
SoCCE, UoP
Comparison-based
methods
18
Subjective voice quality measurement

Mean Opinion Score (MOS)





The most widely used subjective measure of voice quality.
Provides a direct link to voice quality as perceived by the end user.
Gives average opinion of quality based on asking people to grade
the quality of speech on a five-point scale: Excellent, Good, Fair,
Poor and Bad.
Slow, time-consuming, expensive, not repeatable and cannot be
used to monitor voice quality on-line in a large network.
Different Categories of MOS Test (ITU P.800[2])


4/2/2005
Absolute Category Rating (ACR): only listen to the degraded
speech signals (most commonly used)
Degradation Category Rating (DCR): rate annoyance or
degradation level between the reference and degraded signal
SoCCE, UoP
19
MOS Test Based on ACR
Category
Speech Quality
5
Excellent
4
Good
3
Fair
2
Poor
1
Bad
Absolute Category Rating (ACR)
4/2/2005
SoCCE, UoP
20
MOS Test based on DCR
Category
5
4
3
2
1
Degradation level
Inaudible
Audible but not
annoying
Slightly annoying
Annoying
Very annoying
Degradation Category Rating (DCR)
4/2/2005
SoCCE, UoP
21
Online MOS Test Website



http://www.tech.plymouth.ac.uk/spmc/people/lfsun/m
os
This is our research on subjective tests. The aim is to
provide a more efficient method to carry out
subjective tests compared to standard MOS test (e.g.
ITU P.800).
Standard MOS measurement requires a stringent test
requirement (e.g. sound proof room, a large number
of subjects, test procedures). Thus, it is very time
consuming, expensive, and difficult to organise a test.
4/2/2005
SoCCE, UoP
22
Objective voice quality measurement




Automated measure of speech quality using an
appropriate model.
Conventional methods, e.g. SNR-based approach,
are not appropriate as they fail to reveal quality as
perceived by the end user.
Emerging methods for voice quality prediction are
based on models of human auditory perception or
psychologically-derived computational models.
Can be intrusive (e.g. ITU P.862, PESQ [3]) or Nonintrusive (e.g. ITU P.563 [4] formerly P.SEAM) .
4/2/2005
SoCCE, UoP
23
Intrusive measurement
Reference signal/speech
PESQ quality score
System
under test
(MOS)
PESQ
Degraded signal/speech



PESQ (Perceptual Evaluation of Speech Quality), ITU P.862,
Feb, 2001
Intrusive (active) test, listening-only quality
uses test stimuli, such as speech signal
4/2/2005
SoCCE, UoP
24
Perceptual Evaluation of Speech Quality


Transforms the original and degraded speech signals into a
psychophysical representation that approximates human
perception.
Calculates their perceptual distance and maps this into an
objective MOS score.
4/2/2005
SoCCE, UoP
25
PESQ (perceptual difference)
Loss position
PESQ
reference speech
degraded speech
PSQM
4/2/2005
SoCCE, UoP
26
OPTICOM- Opera system
Opera system "Digital Ear“
http://www.opticom.de
Perceptual Voice/Audio
Quality
PESQ/PSQM/PEAQ
4/2/2005
SoCCE, UoP
27
Non-intrusive measurement




Non-intrusive (passive) test
Output-based (speech signal based) or
parameter-based
Low accuracy if compared to the
intrusive methods
Adequate for real-time, online
monitoring purposes
4/2/2005
SoCCE, UoP
28
Non-intrusive Speech Quality Prediction
Gateway
PSTN
Signal-based
method
MOS


IP
T1/E1
IP Network
Signal-based
method
Parameterbased method
MOS
MOS
Signal-based (output-based): to predict/measure voice
quality directly from degraded speech signal (e.g. from
T1/E1).
Parameter-based: to predict/measure voice quality directly
from IP network impairment parameters (e.g. loss, delay,
jitter).
4/2/2005
SoCCE, UoP
29
Signal based (output-based) Method
From T1/E1 link
or end terminal

Speech
Pre-processing

speech feature
parameters extract/
analysis
Speech quality
model
MOS
4/2/2005


Assess/predict speech
quality non-intrusively from
degraded speech signal
only
Need to extract speech
features (e.g. unnaturalness
voice, noises, time clipping)
Mapping to MOS via quality
prediction model
ITU P.563 – May 2004
(single-end, signal-based or
output-based)
SoCCE, UoP
30
Parameter based Method
IP packets

RTP header /
network parameter
analysis

Parameters (e.g.
loss, jitter, delay)
Quality prediction
models (e.g. NN or
non-linear models)

Access/predict speech
quality from IP network
impairments (e.g. loss,
delay) and codec etc.
Neural network model,
non-linear regression
model, ITU-T E-model [5]
External or built-in
approach (be located
before/after jitter buffer)
MOS
4/2/2005
SoCCE, UoP
31
E-model (ITU G.107, G.108)




Computational model – can be used to compute the
“Mouth-to-ear” transmission quality.
Overall Transmission Quality Rating given by model
is referred to as the R factor. R lies in the range 0100 and can be mapped to MOS.
Designed for network planning, but may be used for
non-intrusive quality monitoring/measurement.
Based on the principle that “Psychological factors on
the psychological scale are additive”
4/2/2005
SoCCE, UoP
32
E-model equation
R  R0  I d  I s  Ie  A





Ro: base R value (noise level)
Id: impairments that are delayed with respect to speech
(e.g. talker/listener echo and absolute delay)
Is: impairments that occur simultaneously with speech
(e.g. quantization noise, received speech level and
sidetone level)
Ie: equipment impairment (e.g. codec, packet loss, jitter)
A: Advantage factor (e.g. 0 for wireline and 10 for GSM)
4/2/2005
SoCCE, UoP
33
Loss model - maps loss to Ie
Curve is CODEC
dependant
Ie (packet loss)
50
40
30
20
10
0
0
5
10
15
Packet Loss Rate
4/2/2005
SoCCE, UoP
34
Delay model
30
R Factor 20
Reduction
10
0
0
100
200
300
400
End to end delay (ms)
4/2/2005
SoCCE, UoP
35
E-model (a simplified version)
Delay (d)
Delay model
Id
MOS
RMOS
Packet loss rate
Codec type
Loss model
Ie
R  93.2  I d  Ie
4/2/2005
SoCCE, UoP
36
E-model (R factor) and MOS
TIA 2000
4/2/2005
SoCCE, UoP
37
Extended E-model

Simplified E-model


consider only effects from codec, packet
loss (random packet loss) and end-to-end
delay.
Extended E-model [6]

Further consider burst loss effects (e.g. 2-state
Gilbert model, 3 or 4 states Markov models)
Further consider recency effects.

Telchemy (http://www.telchemy.com/)

4/2/2005
SoCCE, UoP
38
Burst Loss vs. Random Loss
Burst packet loss
Non-bursty packet loss
Packet lost
4/2/2005
Packet received
SoCCE, UoP
39
“Recency” Effect [6]
60 second call
MOS 3.82
MOS 3.28
MOS 3.18
“Good” 4.3MOS
4/2/2005
“Bad” 1.8 MOS
(3dB SNR)
SoCCE, UoP
Source AT&T
T1A1.7/98-031
40
Extended E Model [6]
Packet
Loss
Jitter
Codec
type
Loss
Model
Jitter
Model
Burst
model
User
R Factor
Codec
Model
Delay, measured
using RTCP
4/2/2005
Ie
Network
R Factor
Delay
model
SoCCE, UoP
Recency
model
41
VQmon – Embedded Monitoring[6]
Gateway
Gateway
IP
Network
QoS
metrics
NMS
VQmon Agent
embedded into
VoIP Gateway
Telchemy (http://www.telchemy.com/)
4/2/2005
SoCCE, UoP
42
Voice and Video quality Assessment in
Psytechnics




Psytechnics – spin off from BT
http://www.psytechnics.com
Intrusive model (e.g. PESQ)
Non-intrusive model




4/2/2005
psyVoIP (parameter-based)
E-model
NiQA (signal-based)
CCI (Call Clarity Index)/INMD (In-service Nonintrusive Measurement Device)
SoCCE, UoP
43
QoS Prediction and Control Research in Plymouth

Aims and objectives



To research and develop novel, generic methods
for objective measurement, prediction and control
of user-perceived quality.
To apply the methods to real world problems in
communications, audio and healthcare.
Examples



4/2/2005
Non-intrusive voice quality prediction and
measurement for VoIP
QoS prediction and control for wireless VoIP
Multimedia quality prediction (voice, audio and
video)
SoCCE, UoP
44
Signal Processing & Multimedia
Communications Group
Research within the Group is concerned with the
development of novel, generic signal and information
processing methods and their applications to real world
problems.
Main application areas:
 Multimedia communications – quality of service
prediction and control
 Audio – sound synthesis, audio quality assessment
 Biomedicine – intelligent biosignal analysis, biomedical
informatics, decision support.
4/2/2005
SoCCE, UoP
45
About my PhD project
End-to-end perceived voice quality (MOS)
Encoder Packetizer
Voice
source
IP Network
Depacketizer
Jitter
buffer
Receiver
Sender
Decoder
Voice
receiver
Non-intrusive
measurement
MOS


To develop novel and efficient method/models for non-intrusive quality
prediction,
To apply the models for perceptual optimization control( e.g. buffer optimization
and adaptive sender-bit-rate QoS control)
4/2/2005
SoCCE, UoP
46
A New Methodology
Intrusive method
MOS(PESQ)
delay
Reference speech
PESQ
E-model
Measured
MOSc
Degraded speech
VoIP Network
(packet loss, delay, codec …)
Non-intrusive
method
New model
(regression or ANN models)
Predicted MOSc


Based on intrusive quality measurement (e.g. PESQ) to predict voice quality nonintrusively which avoids subjective tests.
A generic method which can be easily applied to audio, image and video.
4/2/2005
SoCCE, UoP
47
Two Non-intrusive Models


Artificial neural network models for
predicting listening and conversational
voice quality
Simplified regression models to predict
voice quality
4/2/2005
SoCCE, UoP
48
Three Applications



Voice quality monitoring/prediction for
real Internet VoIP traces
Perceived voice quality driven jitter
buffer optimization
Perceived voice quality driven QoS
control (combined adaptive sender-bitrate and priority marking control)
4/2/2005
SoCCE, UoP
49
References
1.
2.
3.
4.
5.
6.
7.
M. Buckley, End-to-end QoS control in VoIP systems, Workshop on
QoS and user perceived transmission quality in evolving networks,
Oct. 2002.
ITU-T Rec. P.800, Methods for subjective determination of
transmission quality, Aug.1996.
ITU-T Rec. P. 862, Perceptual evaluation of speech quality (PESQ),
an objective method for end-to-end speech quality assessment of
narrow-band telephone networks and speech codecs, Feb. 2001
ITU-T Rec. P.563, Single-ended method for objective speech quality
assessment in narrow-band telephony applications, May 2004.
ITU-T Recommendation G.107, The E-model, a computational model
for use in transmission planning, 2000.
A. Clark, Modeling the Effects of Burst Packet Loss and Recency on
Subjective Voice Quality, 2nd IPTel Workshop, 2001, pp.123 – 127.
H. Schink, Characterising end to end quality of service in TIPHON
systems, IP Networking & Mediacom Workshop, April 2001.
4/2/2005
SoCCE, UoP
50
References






L Sun and E Ifeachor, "New Models for Perceived Voice Quality
Prediction and their Applications in Playout Buffer Optimization for
VoIP Networks“ Proceedings of IEEE ICC 2004, Paris, France, June
2004, pp.1478 - 1483.
Z Qiao, L Sun, N Heilemann and E Ifeachor "A New Method for VoIP
Quality of Service Control Based on Combined Adaptive Sender Rate
and Priority Marking“ Proceedings of IEEE ICC 2004, Paris, France,
June 2004, pp.1473 - 1477.
L Sun and E Ifeachor, "New Methods for Voice Quality Evaluation for
IP Networks" Proceedings of the 18th International Teletraffic
Congress (ITC18), Berlin, Germany, 31 Aug - 5 Sep 2003, pp. 1201 1210.
L Sun and E Ifeachor, "Prediction of Perceived Conversational Speech
Quality and Effects of Playout Buffer Algorithms“, Proceedings of
IEEE ICC 2003, Anchorage, USA, May 2003, pp. 1- 6.
L Sun and E Ifeachor, "Perceived Speech Quality Prediction for Voice
over IP-based Networks" Proceedings of IEEE ICC 2002, New York,
USA, April 2002, pp.2573-2577.
L Sun, G Wade, B Lines and E Ifeachor, "Impact of Packet Loss
Location on Perceived Speech Quality“, Proceedings of 2nd IPTelephony Workshop (IPTEL '01), New York, April 2001, pp.114-122.
4/2/2005
SoCCE, UoP
51
Contact details:



SPMC Group website:
http://www.tech.plymouth.ac.uk/spmc
Professor Emmanuel Ifeachor, Head of
Group,
E-mail:[email protected]
Dr. Lingfen Sun
E-mail:[email protected]
Homepage:
http://www.tech.plymouth.ac.uk/spmc/people/lfsun/
4/2/2005
SoCCE, UoP
52
Thank you!
4/2/2005
SoCCE, UoP
53