Transcript advanced SIP testing

Testing Large Scale
SIP Networks
Ofir Michael
Director VoIP technology, RADCOM.
[email protected]
Agenda
• SIP testing – advanced scenarios
• SIP testing - Real Life Examples
Test Equipment
Requirements
SIP Testing – the basics
• Call simulation
• Call monitoring
o
o
Signaling analysis
Media analysis
• Conformance testing
• Load (stress) generation
Advanced SIP Testing
• Advanced testing
o
o
o
o
• For complex scenarios:
Benchmarking
Performance
Media degradation
Envelope determination
o
o
o
o
o
• In terms of a large network…
Many users
Many call flows
Different codecs
Different networking configuration
NAT, Firewall…
Stress Testing??
• Aimed at determining the operational envelope:
o
o
o
Network elements
Service
Certain concept / design
• Why stress test:
o
o
Vendors: wants to determine performance figures
before shipping
Service Providers: wants to find the problems in the
lab
Stress Testing!
Not just “number of calls”
• Emulate a network wide environment:
o
o
o
Large number of users
Many network elements (IPs)
Emulate NAT
• Excessive call load
• Many profiles / scenarios
• Many different services
What are we looking for
Predicting the element behavior in real life:
• How does the UUT (network element / service)
behave in real life (=stress…)
• Where is the “breaking point”
• What happens at the “breaking point”
o
How are new / existing calls affected?
o
Is the media degraded?
Stress Factors
• Basic call flows (signaling)
• RTP streams
• Media type (Video, Audio)
• Advanced call flows
• SIP services
Signaling Stress Requirements
• Generate and respond to SIP service requests
• Simulate User Agent Client / Server communication
procedures (RFC 3261)
• Emulate behavior of one or many SIP phones
o
different signaling and media parameters for each
Registration Rate
• True emulation of user behaviour
o
Generate calls only when registered
• Mega-blast generation of registration events
• Full configuration of SIP registration procedures
o
Mimic real user behaviour
Advanced Testing Requirements (1)
• Multiple users
o
100,000’s UACs
o
Configurable groups each with different behavior profiles
• Multiple IPs
o
1000s of IPs - both media and signaling
• Multiple MACs
• Multiple VLANs
Advanced Testing Requirements -- 2
• Full SIP message customization
• Video generation
• DTMF generation
• Different call flows:
o
Hold, Consultation Hold, Music on Hold,
3-way Conference, Call redirect, Call reject…
• Erroneous flows
Key Traffic Generation Measurements
• Session success rates and GoS
• SIP method count and response code
• Registration history
• Presence and IM message statistics
• Incoming and outgoing RTP stream counters
Real Life Examples
Common pitfalls
• Registration properties
o
Tendency to forget the credentials
• Registration rates
o
o
Initially takes too long
Server will not withstand load
• Non-standard flows
• Test signaling & media simultaneously
Network Model: Simple Test Lab
Analog call generator
Call
Analysis
IP / ATM / FR
Core 2
UUT
SIP Stress
Generator
Call
Analysis
Core 1
UUT
Test Environment:
•Multi-service solution
•Multi-technology: VoIP & PSTN
SIP Stress
Generator
Network Model: Simple Test Lab (cont.)
• SIP Stress Generator:
o
o
Generate different levels of stress including RTP
Voice and video simulation
• Call Analysis:
o
o
o
Signaling and media analysis
Voice quality (intrusive, non-intrusive and subjective)
RTP transport streams
• Analog Call Generator
o
o
Connect to PSTN interfaces
Measure voice quality, voice distortion
Testing Options
• Performance - under different levels of stress:
o
o
o
Signaling and media quality evaluation
Call handling
Media degradation
• Interoperability
o
o
Different codec models
PSTN and VoIP devices
• Reliability
o
o
Quality evaluation
Stability testing
Firewall, SBC, CSCF Testing
Call
Analysis
Call
Analysis
Test bed 1
Soft-Switch
Test bed 2
Soft-Switch
Public IP
SIP Stress
Generator
Session
Boarder
Controller
Session
Boarder
Controller
SIP Stress
Generator
SP1
• Load scenarios
o
o
o
SP2
BHCA
– 1,000,000s per hour
CPS
– hundreds to thousands
Concurrent calls > 100,000s
SIP Stress
Generator
Firewall, SBC & CSCF: Test Lab
• SIP Stress Generation requirements:
o
Different levels of stress
o
Different services
o
Different media
o
Different call flows
• QoS analysis:
o
Voice quality
o
Transport
Firewall, SBC & CSCF: Testing Needs
• QoS
o
Delays, Jitter, Packet loss, MOS
• Stability (network elements / services)
• Benchmarking and comparisons
Simultaneous test bed operation
Test Bed 1
Traffic Simulator
Traffic Simulator
Test Bed 2
Traffic Simulator
Traffic Simulator
Test Bed 3
Traffic Simulator
Traffic Simulator
Elegant solution
VLAN1
VLAN2
Traffic
Generator
Test Bed 2
VLAN
switch
VLAN3
Traffic
Generator
Test Bed 1
Test Bed 3