Transcript The Meta-CLI Model solution

Contributions to Automated
Testing of Network Service
Interactions
Rudy Deca1 ,Omar Mahrez1,
Omar Cherkaoui1, Yvon Savaria2 and
Doug Slone3
1University
of Québec at Montréal, 2Ecole Polytechnique de Montréal,
3Cisco Systems, Inc.
Automated Testing of
Network Service
Interactions
Notere 2005, Ottawa/Gatineau, Canada
Contents




The problem context
The Meta-CLI Model
The ScriptMaker tool
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
2
The Problem context
Causes:
Connectionless
stateless
networks
Connectionoriented
circuit-switching
stateful
network
services
More services
more features
more parameters
More
environment
states
Effects:
Network
convergence
Increase
in the
number
of states
Hybrid,
contradictory
network service
characteristics
New
Solutions
required
More combinations
more interactions
more configurations
more tests
required
3
Contents


The problem context
The Meta-CLI Model

General presentation

•



Structure
Operations
Abstracting the environment
The ScriptMaker tool
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
4
The Meta-CLI Model solution

Abstracts:




the network service and device configuration
states
the environment states
the dynamic changes of the connectionless,
stateless IP protocols
Benefits:


Automated Testing of
Network Service
Interactions
Automated adaptation to environment changes
Efficient generation and modification of
configurations, test scripts and tests for
service interactions.
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
5
Modelling a service configuration
as a conceptual tree structure
service_1
device_1
Created
Modifie
d
service_1
Meta-CLI modelling
Command
mode
M
Configuration
commands and
parameters
Command
P
Parameter
R
Arcs:

Hierarchical
relationships

Inclusion relationships
Command
mode
N
Command
Q
Parameter
S
Nodes:

Configuration command modes

Configuration commands

Configuration parameters
6
Contents


The problem context
The Meta-CLI Model

General presentation
•
Structure

Operations





Configuration
Validation
Abstracting the environment
The ScriptMaker tool
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
7
Meta-CLI Model Operations

Operation types:



Distribution:



Configuration
Validation
Single device
Multiple devices of a network
Operation attributes:



Automated Testing of
Network Service
Interactions
Atomicity
Order (causal, sequential)
Granularity (varying from parameters to
services)
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
8
Configuring services using the
Meta-CLI Model
The main idea of the Meta-CLI configuration: compose the
device and network service configuration models.
Conversion
Network
service
Meta-CLI Model
Device
configuration
record
Automated Testing of
Network Service
Interactions
Composition
Capsule
Conversion
Construct
Configuration Construct +
Retroversion
Device
configuration
record +
9
Generic service configuration
operations








Instantiate service configurations on devices.
Instantiate service configuration parameter values.
Instantiate service configuration constraints.
Adapt service configuration to device and interface
environment.
Adapt service configuration to device and interface
role.
Determine environment support for various feature
combinations
Determine compatibility of services and features.
Determine environment support for service and
feature combinations.
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
10
Examples of validation
dependencies
device_1
device_2
Command
Command
Command
Command
Value1
Value2
Value5
Value6
Q
P
T
P
Dependency T-Q
Parameter
Parameter
R
Value3
Dep.
U-R
U
Value4
Parameter
S
Value7
Parameter
V
Value8
Dependency S-R
Validation dependencies between:
 configuration commands: T depends on Q
 configuration parameters: S and U depend on R
11
Network service and feature
interactions

Mutual exclusion, due to:




Influence, due to :



Competition for resources
Accessing the same parameters/variables
Incompatible goals
Scale factor
Performance factor
Service combination constraints:


Common supporting environment
Existence of mandatory parameters
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
12
Contents


The problem context
The Meta-CLI Model
•



General presentation
Abstracting the environment
The ScriptMaker tool
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
13
Environment heterogeneity
solution
The configuration environment:
 network-level component





network topology,
technology,
Protocol,
Role.
equipment-level component.

hardware components




software components
 operating system,

Automated Testing of
Network Service
Interactions
device type,
interface type,
medium type, etc.
software version, etc.
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
14
Adapt CLI commands to
different environments
service_1
Command P is instantiated on device_1
with one parameter (R) and on device_2
with two parameters (R,S).
env1,
env2
Command
P
env1, env2
Paramet.
R
Paramet.
S
env1,
env2
env2
device_1 + service_1
device_2 + service_1
device_2
device_1
env2
env1
Command
P
Command
P
Parameter
R
Parameter
R
Parameter
S
15
Adapt CLI commands to different
device and interface roles
service_1
role1, role2
Command P
role1, role2
device_1
device_1
role1
Command P is instantiated:
 on device_1 with parameter R and
 on device_2 with parameter S,
based on the different roles assumed
by the two devices.
Paramet. R
Paramet. S
role1
role2
+ service_1
device_2 + service_1
device_2
role2
Command
P
Command
P
Parameter
R
Parameter
S
16
Addition of a network service to
a device configuration



The generic network service
model is accessed from the
repository.
A service instance model is
generated and adapted to the
given environment
and then composed with the
device configuration model.
17
Modification of a network
service in a device configuration




The network service is already configured on an
equipment when an environment change occurs.
The network service model instance in the old
environment is extracted from the device
configuration model.
The generic network service model is accessed
from the repository.
A new instance model is


generated for the new environment and
composed with the device configuration model.
Determining environment support
for various feature combinations
service_1
Service
features
env1, env2
Command P
env1, env2
Feature R
Feature S
env1, env2
env2
Environment
instances
service_1
P
P
P
P
Feature R
NP
P
NP
P
Feature S
NP
NP
P
P
env1
S
S
NS
NS
env2
S
S
S
S
P = is present
S = supports
NP = is not present
NS = does not support
The information is obtained by querying the tree model.
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
19
Contents



The problem context
The Meta-CLI Model
The ScriptMaker tool

•
•
•

Functionality
Architecture
Generation scenario
Conversion scenario
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
20
Functionality of the ScriptMaker
tool



Aimed to regression testing.
Provides an effective, automated and easy way to
generate/modify testing scripts with test cases featuring
various combinations of network services and features.
Solves the problem of the existence of a large number of :




combinations among existing network services;
features and parameters of the services;
environment states in which the service parameters are
configured differently.
Uses a modular approach for components:




Service configurations
Traffic
Routing, topology and connection among router interfaces
Configurations validation
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
21
Contents



The problem context
The Meta-CLI Model
The ScriptMaker tool
•

•
•

Functionality
Architecture
Generation scenario
Conversion scenario
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
22
Architecture of the ScriptMaker
tool
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
23
Contents



The problem context
The Meta-CLI Model
The ScriptMaker tool
•
•

•

Functionality
Architecture
Generation scenario
Conversion scenario
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
24
The script generation scenario
Script Manager Block
Environment
(descriptor)
Script
Feature name(s)
OK
OK
Script
Feature name(s)
Environment (descr.)
Test Selection
Block
Selected
Generic TC, TS,
Signaling
Test Filling Block
Templates
Test Binding
Block
Templates
Generic
service
capsule
stubs
Automated Testing of
Network Service
Interactions
Selected Specific
TC, TS,
EnviRonment
Specific
CLI
service
code
Meta-CLI Model
Block
25
The test selection block
Test
Selector
Feature
name(s)
TC, TS,
Template
names
Feature
name(s)
Spec
Archive
Env.
desc.
Selected TC, TS,
Templates
Constr
Test
Organizer
Feature
name(s)
Environment
Selected
TC, TS,
Template
names
Test Bench
Constraints
Environment
Constraint
Database
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
26
The test filling block
Test Filler
Selected Spec CLI auxiliary, validation
commands and TCL Traffic, Topology
commands
Selected generic
stubs Kx, Kv, Trf, Top
Converter
Selected Gen stubs
Kx, Kv, Trf, Top
Selected Spec CLI auxiliary,
validation commands and TCL
Traffic, Topology commands
Environment
Selected Gen CLI
auxiliary, validation
commands and TCL
Traffic, Topology
commands
Code
Generator
Automated Testing of
Network Service
Interactions
Selected Gen CLI
auxiliary, validation
commands and TCL
Traffic, Topology
commands
Instantiator
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
Ks = Service capsule
Kv = Validation capsule
Kx = Auxiliary capsule
27
The Meta-CLI Model block
Meta-CLI Model
manager
Selected generic service
stubs Ks
Selected generic service capsule
Service
Retriever
Selected
specific
CLI service
Environment
descriptor
Selected generic
service capsule
Selected
generic
service stubs
Selected
generic service
capsule
Capsules
(models)
Automated Testing of
Network Service
Interactions
Service
Instantiator
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
28
Contents



The problem context
The Meta-CLI Model
The ScriptMaker tool
•
•
•


Functionality
Architecture
Generation scenario
Conversion scenario
Conclusion
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
29
The script conversion scenario
Environment
Script Manager
Block
New script
Feature name(s)
Old script
Script
Script
Feature name(s)
Environment
Parsing
Block
Generic TC, TS,
Template
services
auxiliaries
validations
topologies
traffics
Test
Selection
Block
Signaling
Selected
Generic TC, TS,
Template
Test Filling
Block
selected services
auxiliaries
validations
topologies
traffics
EnviRonment
service
Automated Testing of
Network Service
Interactions
Filled Selected
Specific TC, TS,
Test Binding
Template
Block
Specific
CLI
service
code
Meta-CLI Model
Block
30
Conclusion

Problem:

New Internet services, hybrid, with numerous:






parameters
features
interactions
heterogeneous environments.
FSMs not always useful.
Solutions:

The Meta-CLI Model:



Generic service configuration
Environment-aware
The ScriptMaker tool:



Automated Testing of
Network Service
Interactions
Automated generation/modification of testing scripts;
Modular design;
Uses the Meta-CLI for abstracting and reusing service
configurations.
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
31
Questions
Automated Testing of
Network Service
Interactions
R. Deca , O. Mahrez, O. Cherkaoui,
Y. Savaria and D. Slone
32