Transcript IxVeriWave Turn up Process

Proceed with confidence …
Solutions for wireless & wired testing
IxVeriWave 802.11 Focus
Focused on 802.11 device and network performance
 Device and system build outs
•
•
•

Qualify: Network requirements
Quantify: Application level feature and function support
Verify:
Overall performance
Deployment and interoperability
•
•
•
•
Verification of system requirements
Ensure system deployment is as expected
Provide qualified sign-off criteria
Reduce un-knows and down time
Who Uses IxVeriWave?
Service Provider, Private Network
Operator, IT Department
Network Equipment Manufacturers
Network Device / Clients – Handset,
PDA, Scanner, Patient Monitor,
Camera
Network Infrastructure – Access
Points, Controllers
Development
QA
Interoperability
Wireless System Under Test
Test Traffic
WT20
RF
Proof of
Concept
Vendor
Selection
Deployment
AP
AP
Wireless System Under Test
Test Traffic
Ethernet
Ethernet
Test Traffic
WaveDeploy
WT20
Ethernet
Test Traffic
Network Under Test
WaveDeploy
Ethernet
WaveDeploy
WiFi
AP
AP 3
Monitor Ports
Traffic Generation Ports
Client Test
Bench
AP1
AP 2
DUT
AP 1
WT20
WiFi
Client
AP2
WT20
IxVeriWave Turn up Process
● Each VeriWave Stateful Client implements

Carrier deference

Collision sensing

Random back-off

Retransmission

Collision emulation
● VeriWave client is a highly stateful entity

Each client supports the full MAC per 802.11 standard

802.1X supplicant supports full EAP stack per client

Upper layer protocols (e.g., DHCP and TCP) are also implemented
using independent protocol state machines

All state machines for all clients run concurrently

Every client independently updates its state based on channel
conditions e.g. 802.11g clients using protected-mode when .11b is
detected
● Multi-level scheduler enables optimum channel
utilization, multiple traffic flows per client, and mixing
of Constant Bit Rate voice/video traffic with Best
Effort data traffic
● All multiplexed on to a single PHY
IxVeriWave Products at a Glance
 WaveTest
• WLAN Traffic Generator / Analyzer
• Lab testing:
 Access Point testing
 Controller testing
 Client testing
 WaveDeploy
• Handheld Field Tool
• Software with a hardware option
• Site Assessment and Readiness
Tool
802.11ac Deployment
• Carriers and Cable companies have been early
adopters of 802.11ac
• The new 802.11ac will provide
– Higher bandwidth rates (1.73Gb @ 80M channel spacing
vs legacy 450Mb@ MIMO MCS 23)
– Operates at 5.0Ghz freq. , less interference than 2.4Ghz
– Designed to support HD and other residential applications
– Eventually will support Multi-User MIMO (MU-MIMO)
Carriers and Cable Companies
• IxVeriWave sold it’s first carrier system in 2007
to Verizon in Waltham, MA, USA
• Since then we have placed systems with most
major carriers in the US and abroad, including:
–
–
–
–
–
–
–
AT&T
Deutches Telecom
Cablevision
China Unicom
TW Cable
BT
etc.etc.
- Bell Canada
- Telefonica
- Shaw
- KoreanTel
- Vodaphone
- LGU
- China Mobile
- Telstra
- Telus
- PCCW
- O2
- BSkyB
- China Telecom
- Verizon
- Orange
- TelcomArgentina
- VideoCom
Sample Customers – From Silicon to
Deployment
Silicon &
Software
Providers
Mobile
Devices
Network
Infrastructure
Service Provider &
End User
IxVeriWave Turn up Process
Recommended Four Part Test Plan
 IxVeriWave has created a test plan to support:
•
•
•
•
Benchmarking
Quality of Experience
Over The Air (OTA) Testing
Site Analysis (pre/post deployment)
 The goal:





Define the characteristics of the network infrastructure elements
Fully appreciate the “actual” capabilities of AP + Controller network
Identify any performance issues that will impact design
Identify any devices issues (features supported etc.)
Base your network design on known conditions NOT paper
specifications
Advantages of Using IxVeriWave
 Technical Benefits
•
•
•
•
•
•
Confirm configuration expectations
Evaluate feature and functional performance
Ensure repeatable test criteria across test procedures
Reduce performance analysis cycle
Replicate real world client eco systems prior to deployment
Evaluate measurable performance results
 Financial Benefits
•
•
•
•
•
Select the best configuration for the requirements
Clearly define performance capabilities, know what your deploying
Increase corporate efficiency via proper configuration qualification
Reduce field complaints from unidentified issues
Reduce operational costs by deploying a known/robust network
At the end of the day
● Comprehensive report
generation
– Management-level reports are
created automatically by Automated
Tests and scripts
– Eliminates tedious numbercrunching and graphing
● Highly detailed reports
– Test details
• Date, version numbers, etc.
– Measured results
– Expected results
– Explains how to interpret results
Part 1 Benchmarking

Simple, accurate benchmark testing
•
Common configuration, reporting
•
PDF report summarizing test results
Integration & Parametric
Testing
 Per industry standards or published methodology
• Benchmarking
 Throughput
 Packet loss
 Client capacity
 TCP Goodput
 Verify your configuration
Part One-Throughput Sample
Typical a/b/g AP performance
802.11 n Throughput Sample
Enterprise 802.11n AP performance
IxVeriWave Turn up Process
Benchmarking
 You can not tell performance by brand name
 The suggested benchmarking tests have been
selected because they quickly verify performance
 Network design should then be based on
performance and not mathematical calculations
 The goal:





Define the characteristics of the network infrastructure elements
Fully appreciate the “actual” capabilities of AP + Controller network
Identify any performance issues that will impact design
Identify any devices issues (features supported etc)
Base your network design on know conditions NOT paper specifications
Part 2 – Create Client Eco System
• Representative device and
user profiles for key wired
& WLAN environments
• WaveQoE accurately
replicates complex
interactions of clients,
servers and traffic
• WaveQoE measures and
reports key application
metrics affecting end-user
Quality of Experience
(QoE)
• Provides a triple play
performance assessment
of your infrastructure
Defining Eco System Traffic
The IxVeriWave designed and engineered radio is the main WLAN Tx/Rx engine.
The IxVeriWave radio allows the user the flexibility to define device level metrics
to create device profiles and specific eco systems.
Part 2 – Test to Specific SLA’s
Define required SLA’s
Voice
MOS > 4.2
View Pass/Fail per Traffic Type
MPEG Video (unicast)
Delay Factor < 70 mSec
Media Loss Rate < 3%
FTP
Goodput > 100 kbps
Tranfer time < 1 Sec
HTTP
Goodput > 500 kbps
Tranfer time < 100 mSec
MPEG Video (multicast)
Delay Factor < 50 mSec
Media Loss Rate < 1%
UDP
Latency < 4 Sec
Jitter < 100 mSec
Packet Loss < 10%
System or Network
Under
Test
Traffic type
Achieved SLA
Pass/Fail
Voice
4.3
Pass
Unicast video
DF=80 mSec
MLF=3%
Fail
FTP
Goodput=100 kbps
Transfer time=2 Sec
Fail
HTTP
Goodput=600 kbps
Transfer time=80 mSec
Pass
Multicast
video
DF=40 mSec
MLF=1%
Pass
UDP
Latency=3 Sec
Jitter=100 mSec
Loss=8%
Pass
Different Settings Different Results
IxVeriWave Turn up Process
Quality of Experience
 Allows a controlled examination of the WLAN
devices ability to support Triple Play
 Evaluate the mission critical support capabilities of
the network equipment being deployed
 Confirm proper operation of the configuration to be
deployed in the network
 The goal:





Define the characteristics of the network infrastructure elements
Fully appreciate the “actual” capabilities of AP + Controller network
Identify any performance issues that will impact design
Identify any devices issues (features supported etc)
Base your network design on know conditions NOT paper specifications
WaveAgent™
 WaveAgent
•
Measures and reports end-user
experience on the DUT (forwarding
rate, loss, latency, MOS, etc.)
•
Very small footprint (<64kbytes)
does not interfere with normal
operation of DUT
•
Supported OS’s: Windows Mobile 6,
Windows XP, Windows Vista, Linux
2.6 kernel
Small SW utility loaded on
remote client
WiFi
Eco-system
traffic
Ethernet
 Benefits
Client hosting
WaveAgent
•
Enables performance testing of the client’s 802.11 subsystem (device driver and
integration) independently of the application
•
Enables performance testing during silicon qualification (on eval boards), post driver
integration on client hardware, and on integrated client during troubleshooting
•
Works over any network interface (Ethernet, WLAN, MoCA, WiMAX, LTE, etc.)
•
Allows testing to occur earlier in the development cycle
Part 3 – OTA Interoperability Testing
● Interoperability testing & troubleshooting with real client devices (laptops,
mobile phones, scanners, patient monitors, etc.)
● Powered by WaveAgent installed on each device
● Setup and test run within less than 5 minutes
● Results are reported over time
● WaveInsite measures

UDP throughput, TCP Goodput, Loss

Voice MOS and R-Value

All results compared to user defined Service Level Requirements (SLRs)
IxVeriWave Turn up Process
Over The Air (OTA)
 Test the network configuration in a controlled and
repeatable test environment
 Verify the capabilities of the AP device with a variety
of client devices
 Confirm interoperability of the AP-Client device
 The goal:





Define the characteristics of the network infrastructure elements
Fully appreciate the “actual” capabilities of AP + Controller network
Identify any performance issues that will impact design
Identify any devices issues (features supported etc.)
Base your network design on know conditions NOT paper specifications
WaveDeploy Pro

Reported metrics:
•
Power – RSSI
•
Co-channel interference
•
UDP downstream/upstream Max forwarding
rate
•
TCP downstream/upstream Max Goodput
•
Voice MOS Upstream, Voice MOS
Downstream
•
Video MDI
•
Loss & jitter
•
Client Roaming time (driver and OS
dependent)

Set Service Level Agreements for each
client / service type

Single Pass Site Survey - all measurements
are conducted and recorded during a single
pass

HeatWave maps and individual measurements at every test stop available at the
conclusion of site survey

Generates PDF reports

Maximum number of mobile devices participating in single survey = 10
Off the shelf server
(could be a laptop)
IxVeriWave Turn up Process
Network Assessment
 WaveDeploy will allow the user to quickly verify the
performance of the target network
 Performance metrics derived from the testing of the
AP provides data for SLA settings in WaveDeploy
 The assessment provides RF to application details
 The goal:





Define the characteristics of the network infrastructure elements
Fully appreciate the “actual” capabilities of AP + Controller network
Identify any performance issues that will impact design
Identify any devices issues (features supported etc.)
Base your network design on know conditions NOT paper specifications
WLAN Networks:
Getting to Best-in-Class
A Tale of Two Airports
WLAN Networks:
so what’s the catch
Beijing and Guangzhou
• VeriWave began providing Vendor selection services to
China’s three major carriers in 2008
– Technology selection and integration department
– Network and system test department
– Network operations department
• China deployed 400,000 hotspots in 2009-2010
• Carrier 1 deployed Guangzhou
–
–
–
–
Utilized IxVeriWave to do AP selection testing
Defined SLA’s prior to deployment
Identify configuration issues
Confirm proper operation of features/functions
• Carrier 2 deployed Beijing
– Designed from vendor specifications
– Tested via IxChariot and Airmagnet
– Deployed newer version of Carrier 1’s AP (same NEM)
Defining a WiFi Network
•
•
WiFi Networks Testing begins at the network level
IxVeriWave provides purpose build equipment to address all
of the test requirements of a robust network
• WaveTest: WT-20/90
–
–
–
–
•
Benchmarking AP’s
Confirming configurations (AP, DHCP Servers, Controllers)
Identify configuration issues
Confirm proper operation of features/functions
WaveDeploy Pro
– Quickly confirm proper performance in the deployed network
– Ensure application level user experience meets users requirements
– Test against required SLA (pass/fail)
•
It is quicker to test and deploy than deploy and then test
Testing Deployed Networks
Beijing Terminal 1
Sample Testing of the Airport Wifi
Guangzhou Gate B216
Signal Strength –
Great in both terminals for both tests
Guangzhou Terminal B216
Beijing Terminal 1
RSSI indicated that both terminals had strong signal levels!
The Results – Beijing - PEK
Beijing only worked
for locations right
next to access point
This is a dropout of HTTP
The Results – Beijing - PEK
Some areas
have rates as
low as 4kbits
per second!
The Result - Guangzhou
Lowest HTTP goodput for
Guangzhou is 348Kbits/s
Greater Quality of
Experience in
Guangzhou – more
consistent application
performance in all
areas with no HTTP
dropouts
The Result - Guangzhou
802.11 n AP deployed by carrier
The Results – Beijing - PEK
New version of 802.11n AP deployed in PEK
WaveTest - Application Results
Guangzhou
• Meets expected SLA
• Meets expected voice support
• Meets expected video support
• Meets expected data rates
Meets expected client mgmt
Beijing
• Poor overall performance
• Limited voice support
• No video support
• Poor in terminal roaming
• Very low data rates supported
• Poor client management
Gauging the Environment
 Beijing vs Guangzhou
•
RF levels would imply the networks are the same
•
•
WaveDeploy verified poor performance in Beijing
•
•
Some co-channel interference noted
WaveTest lab testing identified the issues
•
•
•
•
•
Yet users in Beijing were complaining
Poor overall performance
Poor data management
Configuration issues
NEM verified that the new devices OEM had not performed benchmark testing
Resolving and identify the issue required benchmarking of the AP
 Financial Benefits
•
•
•
•
•
Select the best configuration for the network requirement
Clearly define performance capabilities, know what your deploying
Increase corporate efficiency via proper configuration qualification
Reduce field complaints from unidentified issues
Reduce operational costs by deploying a known/robust network
Advantages of Using IxVeriWave
 Technical Benefits
•
•
•
•
•
•
Confirm configuration expectations
Evaluate feature and functional performance
Ensure repeatable test criteria across test procedures
Reduce performance analysis cycle
Replicate real world client eco systems prior to deployment
Evaluate measurable performance results
 Financial Benefits
•
•
•
•
•
Select the best configuration for the requirements
Clearly define performance capabilities, know what your deploying
Increase corporate efficiency via proper configuration qualification
Reduce field complaints from unidentified issues
Reduce operational costs by deploying a known/robust network
Thank You
Demonstration