Transcript Creating Resource-Efficient V2oIP Applications for Low

Creating Resource-Efficient V2oIP Applications
for Low-MHz Mobile Processors
Fred Wydler
VP VoIP Products
SPIRIT DSP
About SPIRIT
Acknowledged Communications Expert
 10 years in global embedded voice
software business
Selected SPIRIT
Customers:
 200+ first-class customers, including
largest OEMs and software vendors
 140 people, 70 engineers, 15 PhDs
 100+ millions embedded (HW) voice
channels powered
 Multiple international “Product of the
year” awards:
…and 200+ more
3
Agenda
 VoIP on mobile: new rules of the game
 To be IMS ready is the goal
 Video communication – must-have functionality
 Mobile VoIP. Quality wins. Wideband coming
 Where the troubles come from
4
Agenda
 VoIP on mobile: new rules of the game
 To be IMS ready is the goal
 Video communication – must-have functionality
 Mobile VoIP. Quality wins. Wideband coming
 Where the troubles come from
5
Mobile VoIP: Start of a New Era
WiFi phones
Game Consoles
Mobile phones
VoIP
Set-top boxes
Wireless
residential
gateways
Portable players
6
Demands of Today and Tomorrow
IMS Compliance
Multimedia
accelerators
Requirements grow
higher and deeper
Real-time video calls
Time-to-market
Mobile form factor
High quality echo free
voice
7
Basic layer –
traditional
requirements
Agenda
 VoIP on mobile: new rules of the game
 To be IMS ready is the goal
 Video communication – must-have functionality
 Mobile VoIP. Quality wins. Wideband coming
 Where the troubles come from
8
Each Service over Each Access Network
IMS idea goes to the real life fast
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
ü
9
Voice
Instant messaging
Video calls
Content sharing
Video sharing
Interactive video
3D multimedia
Mp3 playback
Gaming
Broadcasting
PPT, POC
Cellular
(GSM, W-CDMA, CDMA200)
Data channels
(GPRS, EDGE, EV-DO, HSPDA)
Wireless IP
(WiFi, WiMAX)
VoIP Evolution
Are you 100% sure of your Voice Engine supplier?
Full IMS
Client
Market Demands
Defined
Quality Level
Video
Engine
Voice
Engine
Voice
Engine
Application
Enabler
Video
Engine
Voice
Engine
Time
10
Application
Enabler
Video
Engine
Voice
Engine
Defined
Quality Level
Application
Enabler
Video
Engine
Voice
Engine
Agenda
 VoIP on mobile: new rules of the game
 To be IMS ready is the goal
 Video communication – must-have functionality
 Mobile VoIP. Quality wins. Wideband coming
 Where the troubles come from
11
Video Means See not Guess
 Video performance in terms of resource limitations of mobile
platforms
– Regular RISC processor inside offers up to 15fps on a PDA display
– Adding HW accelerator significantly improves frame rate
 Video packet loss
- Video PLC
 Packets’ delivery conditions do not allow smooth playback
– Automatic adaptation to the channel bandwidth (ARS)
– Intelligent CPU load control/playback rate control
– Audio/Video synchronization
voice, audio, video
12
Hardware Accelerators Welcome
Integrating hardware accelerators into a complete solution
 Standard integration
(OpenMAX™)
 Custom integration
Application Layer
Call Manager (SIP, H.323, H.324M)
TeamSpirit® Mobile Voice + Video Engine
Media Transport (RTP/RTCP/SRTP)
Thin Layer
Hardware Accelerator
Already deployed by:
13
MPEG.4
H.264
H.263+
AMR
G.723.1
AEC
Voice&Video Framework
VoIP Speech
Audio/Video
Codecs
Synchronization
NetJet™
DTMF
(Jitter Buffer, PLC,
“playout rate”/”clock
drift” adaptation)
CPU Load Control
Video RTP Packetizers
RealDuplex™
(Speech Enhancement)
Agenda
 VoIP on mobile: new rules of the game
 To be IMS ready is the goal
 Video communication – must-have functionality
 Mobile VoIP. Quality wins. Wideband coming
 Where the troubles come from
14
Quality as Competitive Advantage
Customer satisfaction is the goal
 Mobile VoIP is a commodity (lots of providers offer VoIP services)
 Zero switching costs while changing VoIP provider
(caused by poor user experience: voice breaks, delays, echo)
 To tie up the customer VoIP providers need to pursue voice quality
3.9MOS
Average real conversation voice
quality delivered by typical VoIP
service providers nowadays
15
vs.
4.32MOS
Voice quality delivered
by TeamSpirit®
for VoIP calls over WiFi
Wideband Voice Engine
Customer’s demand – wideband communication
 Wideband AEC
 Wideband speech codec
 Wideband adapted algorithms
16
Agenda
 VoIP on mobile: new rules of the game
 To be IMS ready is the goal
 Video communication – must-have functionality
 Mobile VoIP. Quality wins. Wideband coming
 Where the troubles come from
17
Where the Troubles Come From
 Network challenges
 Voice processing challenges
 Device challenges
 Integration challenges
18
Why Engine, not Codec?
Network challenges
 Smooth operation in IP networks originally not suited to real-time
voice transmission
–
–
–
–
Network jitter elimination
Packet loss concealment (standard speech codecs)
Delay reduction
Packets re-ordering
 Robustness to WiFi-specific network impairments
– Applying new WiFi standard revisions
– Advanced algorithms handling packet bulk scenario
19
Seamless Network Access
Network challenges
 Handover
– Apply VoIP to different access networks (WiFi, 3G)
– Network evolution to the all-IP network and mobility
– Service provider has to be involved
– Active and automatic handover
 WiFi roaming
20
PSTN Experience to Achieve with VoIP
Voice processing challenges
 Quality issues
–
–
–
–
Voice packet loss robustness
Efficient echo cancellation
Round-trip delay minimization
Clock drift adaptation
 Regular telephony services
–
–
–
–
Call mute
Call hold/resume
N-way conferencing
Call transfer
 Advanced telephony services
– 4x dialing
– Voice mail
– Caller ID
21
Mobile Form Factor to Survive with
Device challenges
 Hardware acoustic differences
– Earpiece mode support
– Microphone and speaker AGC
– Filtering
– Intelligent non-linear processing
 Battery life
– Minimal MHz requirements
– The least number of threads in a system and the lowest memory
footprint
– Effective usage of DTX mode to minimize Tx and Rx power
consumption for the WiFi interface
– Highly-optimized speech and video codecs
– Careful optimization to specific processor architecture
22
VoIP Encapsulation
Integration challenges
 VoIP is a commodity – not everyone is an expert in VoIP
– Integration is the key
–Minimum number of threads
–CPU load control
–Time for debugging
– Cross-platform code (ARM9/9E/11, TI DaVinci, TI C64x, Renesas,
XScale, TI OMAP)
– Support for Windows/Linux OS families
– HW accelerators support
 High-level API – application enabler required
– Signalling (H.324M, H.323, SIP, Skype, XMPP, any proprietary
protocol)
– IMS application compliance
23
Integration is the Key
What’s wrong with components?
− Single components work good, but the complete solution does not
− Single components are optimized, but the complete solution is not
− Integration is difficult to manage
G.7xx
H.*
I/O
G.*
vs.
et
k
or
w
24
Network
N
H.26x
I/O
Complete Solution
Application Layer
Call Manager (SIP, H.323, H.324)
TeamSpirit® Mobile Voice+Video
Media Transport (RTP/RTCP/SRTP)
Video Engine
Voice Engine
Speech Codec Library
(G.7xx, AMR-NB)
Audio/Video
Synch.
NetJet™
(Jitter Buffer, Packet Loss
Concealment, “Clock Drift”
Adaptation)
DTMF
RealDuplex™
(AEC, VAD, AGC, CNG, NS,
Mode Detection)
Video Codec Library
(H.26x, MPEG.4)
RTP Packetizers
Playback Rate Control
N-way
Conferencing
Mixer
Video Packet Loss
Concealment
CPU Load Control
OS/Hardware Dependent IO (Sound/Video IO, Device Detector)
25
Real Life Requirements
Beyond standard specifications
 Interoperability with IMS infrastructure vendors
 Firewall overcoming
 Testing with major operators in USA, Asia, Europe
26
Thank you!
Voice and Video Enable your Application!
Request TeamSpirit® Mobile evaluation SDK
with free SIP Voice&Video phone application in it
running on starting from 200 MHz WinMobile devices
www.SpiritDSP.com
[email protected]
27