Transcript VoIP on the iPhone

VoIP on the iPhone: Imagine the Possibilities
Jan Linden, VP of Engineering
VoIP on the iPhone
Outline
•
•
•
•
Market trends
Technology trends
Mobile VoIP challenges
Solutions and recommendations
Evolution of IP Voice and
Video Processing
Advanced Mobile and
Multimedia Capabilities
Solutions Designed
Specifically for VoIP
PSTN Solutions
Adapted for VoIP
Mobile Market Trends
• Fixed Mobile Convergence
– Mobile phone increasingly perceived as a computer
• Advanced operating systems such as Apple OSX are making this
a reality
– Users expect to access the same applications available across
multiple platforms
• Increased accessibility and affordability
– Network operators are introducing new pricing models to
encourage uptake of data driven services and applications
• IP networks enable more cost effective transport
– Handset prices declining
• iPhone 3G half the price of 1st Generation
Mobile Technology Trends
• Migration from legacy cellular to next-gen networks
– Allows for faster data connection
• Growth of Smartphone adoption
– iPhone and other dual-mode phones provide greater flexibility in
connecting to available networks
– All-in-one devices able to run multiple applications efficiently
• Consumers can place calls and receive data simultaneously
• Development of 3rd Party Applications
– Operating systems, like the Apple OS X, more conducive to
advanced 3rd party applications
– VoIP over mobile in high demand
The iPhone as a VoIP Platform
• iPhone most VoIP friendly phone on market…
–
–
–
–
–
Open APIs for easy development
Sufficient CPU resources
Support of wideband codecs
Intuitive UI makes for simple navigation between applications
Demand for 3rd party apps is higher than other phones
• …but strengths present unique challenges
– Running too many applications can drain resources
– Usage scenarios can negatively impact voice quality
With the right voice processing software, developing VoIP applications is a cinch!
Technology Challenges – Network Connections
• IP Networks
– Latency
– Jitter
– Packet loss
• Wireless connections
– Low signal strength
– Interference from users in
same or neighboring
frequencies
– Available bandwidth varies
– WiFi access point
bottlenecks
Technology Challenges
• Device Limitations
– Limited processing power
– Battery life
– Data intensive applications can
cause jitter
• Mobile Environments
– Tend to have excessive
background noise
– Hands Free operation
enhances acoustic echo
Impact of IP Networks
Delay
Packet Loss
Network Jitter
Impact of IP Networks – Latency
• Major effect is “stepping on each other’s talk”
• Usage scenario affects annoyance factor – higher
delay can be tolerated for mobile devices
• Long delays make echo more annoying
001100111
1111010
0101101
1011100
Sources of Latency
•
•
•
•
•
•
•
Codec
Capture
Playout
Network delay
Jitter buffer
OS interaction
Transcoding
A/D
A/D
PrePreprocessin
Processing
g
Speech
Speech
encoding
Encoding
IPIP
interface
Interface
IP
IP Network
Network
D/A
D/A
PostPostprocessin
Processing
g
Speech
Speech
Decoding
decoding
Jitter
Jitter
Butter
buffer
Mean Opinion Score
Impact of Latency on Voice Quality
4
3
2
1
0
250
500
One-way transmission time [ms]
750
Data from ITU-T G.114
• ITU-T (G.114) recommends:
– Less than 150 ms one-way delay for most applications (up to 400
ms acceptable in special cases)
• Users have become accustomed to longer delays
– Still, low delay very important for high quality
Impact of IP Networks – Packet Loss and Jitter
• Packet Loss
– Occurs due to flushed buffers in network nodes
– Same effect if packets are too late to be used
– Smooth concealment necessary
• Network Jitter
– Transmission time differs for each packet
– Jitter buffer necessary to ensure continuous playout
– Trade-off between delay and quality
Technology Challenges – Mobile Environments
• Acoustic Echo
– Speakerphone operation common for iPhones
– Micropohone and speaker close
– High delay in VoIP makes echo more prominent
• Background Noise
– Environments inherently noisy
– Usage scenarios can make
differentiating between speech and
noise difficult
Technology Challenges – Devices
• Limited Resources
– Size limits processing power
– Battery life also limited
• VoIP Friendliness
– Many smartphones limit access to
soundcard
– Device buffers add latency
– Clock drift
• Too much of a Good Thing
– Running too many applications can drain
resources
– Multiple applications accessing network
can result in jitter
VoIP Design Considerations
Speech Quality
Time to Market
Ease of Use
Quality
Cost
Cost
Signaling
Flexibility
Network
Impairments
Infrastructure
Power
Consumption
Device
Considerations
Features
Mobile VoIP Design
Considerations
Speech Codec
Coping with Network
Degredation
Environment – Background
Noise, Room Acoustics,
etc.
Codec
Hardware
Network
VoIP Design
Challenges
Echo Cancellation
Echo
Additional Voice
Processing Components
Power Consumption
Power
Hardware Issues
(Processor, OS,
Acoustics, etc.)
Voice
Environment
Speech Codec
• Many conflicting
parameters affect choice
of codec
• Determines upper limit of
quality
•
Complexity
Memory
Delay
Speech Codec
Support of several codecs necessary
– Interoperability
Input Signal
Robutness
Bit-rate
– Usage scenario
•
Packet-loss
Robustness
IPR issues a significant concern
Quality
Sampling
Rate
Quality
Audio Bandwidth Effect on Quality
CD -Audio
GIPS Wideband Speech
GIPS Narrowband Speech
Std Wideband Speech
(e.g. G.722.2)
Std Narrowband Speech
Implementation (PSTN)
200 - 3400 Hz
3,4
4 kHz
6.4-7.0 kHz
8 kHz
22.1 kHz
Frequency
• Better than PSTN quality is achievable in VoIP
– Utilizing full 0 – 4 kHz band in narrowband
– Wideband coding offers more natural and crispier
voice
iPhone Specific Challenges
• Applications cannot run in the background
• Audio HAL APIs (regular APIs) not available on iPhone. (As on
Mac.) Very iPhone specific solution had to be designed
• Must consider change of audio route (e.g. headset in/out) and
handle this
• Wideband audio not available via built-in mic
Recommendations
• IP Networks
– No need for dedicated bandwidth if solution includes high
performance:
• Bandwidth management
• Robust codec framework
– Low latency achieved by efficient jitter buffer
• Audio
– High quality audio is crucial
• Users expect the same quality they experience on the PC
• Optimized Solutions
– Efficiency for ARM processors on iPhones
– Reduce CPU constraint and increase battery life
Recommendations
• Adaptive Jitter Buffer and PLC
– Manage up to 30% packet loss
– Adapt to network busts and excessive mobile jitter
• Codecs Designed for IP Networks
– Work in tandem with jitter buffer to overcome packet loss
– Can deliver Wideband experience
– Robust and bit-rate adaptive codecs necessary
• Efficient use of the total available bandwidth
• GIPS iLBC already in the iPhone, GIPS iSAC and other wideband
codecs provide even better user experience
• Aggressive Voice Quality Enhancement
– Detect and cancel background noise
– Suppress acoustic echo
Recommendations
• VoiceEngine Mobile for iPhone
– Comprehensive package of voice processing components
– Handles delay, jitter, packet loss and mobile environments
– Optimized for iPhone and Smartphones
– Efficient solution reduces CPU demands
– Flexible and easy to integrate