Transcript slides - Department of ECE UCF

Network Management Game
Engin Arslan,
Murat Yuksel,
Mehmet H. Gunes
University of Nevada, Reno
LANMAN 2011
North Carolina
Outline
• Motivation
• Related Work
• Network Management Game (NMG) Framework
• User Experiments & Results
• Conclusion
Motivation
• High demand for multimedia applications
(VoIP, IPTV, teleconferencing, Youtube)
• ISPs have to meet customer demand
Service Level Agreement (SLA)
• Network management and automated configuration of largescale networks is a crucial issue for ISPs
• ISPs generally trust experienced administrators to manage
network and for better Traffic Engineering
Training Network Administrators
• Network administrator training is a long-term process
• Exposing inexperienced administrators to the network is too
risky
• Current practice to train is apprenticeship
Can we train the network administrators using a
game-like environment rather than months of
years of apprenticeship?
Related Work
• Training by virtualized game-like environment

Pilot training

Investor training

Commander training
• Compeau et al. : End-user training and learning
Deborah Compeau, Lorne Olfman, Maung Sei, and Jane Webster. 1995. End-user training and learning. Commun. ACM 38, 7
• Chatham et al. : Games for training
Ralph E. Chatham. 2007. Games for training. Commun. ACM 50, 7 (July 2007), 36-43.
• Network administrator programs: Cisco Certification
Framework
Network
Configuration
Display traffic
3
7
2
1
6
Traffic traces
Graphical User
Interface
Change link
weight
4
Simulation
Engine
(NS-2)
5
Calculate
new routes
Block diagram of Network Management Game (NMG) components.
Network Simulator (NS-2)
NS-2
System
Configuration
Output
• No real time interactivity
Run simulation See the results
• Necessitates adequate level of TCL scripting
• Not designed for training purpose
Simulator-GUI Interaction
• Concurrency is challenging
Run the simulation engine for a time period then
animate in GUI before the engine continues
Slowdown animator – chose this approach
• GUI-Engine interaction is achieved via TCP port
Animator opens a socket to send simulation traces
GUI opens a socket to send commands
Sample Message: $ns $n1 $n2 2 
set weight of link between n1 and n2 to 2
NMG Screenshot
User Goal
• Increase Overall Throughput by manipulating
link weights within a given time period
B
1Mb/s
E
A
3Mb/s
1Mbps
1Mb/s
C
4Mb/s
D
3Mb/s
3Mbps
User Goal
User Experiments
We conducted 2 user experiments
• Training without Mastery
 No specific skills targeted
 No success level obligated
• Training with Mastery
 Two skills are targeted to train
 Success level obligated
Introduction| Related Work | NMG Framework | User Experiments| Conclusion
Training without Mastery
• 5 training scenarios
• For every scenario, user has fixed 3-5 minutes
to maximize overall throughput
• 8 users attended
• Took around 45 minutes for each user
• User performance evaluated for failure and no
failure cases
User Experiment
Failure scenarios
No failure scenarios
Tutorial
6
7
Before Training
1
2
3
Training
4
5
6’
7’
After Training
No Failure Case
Before
Training (Mbps)
Ratio to
After
Optimal (%) Training (Mbps)
Ratio to
Optimal (%)
No Player
6
66.6
6
66.6
Genetic Algorithms
-
-
6.8
75.5
Random Recursive Search
-
-
8.5
94.4
Users (Average)
7.11
79
8.6
95.5
Optimal
9
100
9
100
P-test
value :0.0002
Before Training
After Training
16% increase
Failure Case
Before
Training
(Mbps)
Ratio to
Optimal (%)
After
Training (Mbps)
Ratio to
Optimal (%)
No Player
4
30.7
5
38
Genetic Algorithms
-
-
7.9
60.7
8
61.5
10.01
77
13
100
Random Recursive Search
Users (Average)
Optimal
Users- outperform
heuristic
solutions
9.73
P-test value:74.8
0.27
13
Before Training
100
After Training
2.2% increase
Training with Mastery
• Two skills are targeted
High bandwidth path selection
Decoupling of flows
•
•
•
•
7 training scenarios 7 levels
Success level is obligated to advance next level
5 users attended
Took 2-3 hours on average per user
Introduction| Related Work | NMG Framework | User Experiments| Conclusion
Training with Mastery
Tutorial
8
Before Training
1
2
3
4
5
6
7
Training
Introduction| Related Work | NMG Framework | User Experiments| Conclusion
8’
After Training
Results of Training with Mastery
P-test value: 0.00001
Introduction| Related Work | NMG Framework | User Experiments| Conclusion
Conclusion
• Performance of a person in network
management can be improved via our tool
16% improvement  first user experiment
13%- 21% improvement second user
experiment
• People outperform heuristic algorithms in case of
dynamism in network
• Targeting skills and designing specific scenarios
for skills lead better training
 Success level of second user training
Introduction| Related Work | NMG Framework | User Experiments| Conclusion
Future Work
• Extend for large scale networks
• Extend quantity and quality of test cases
• Using different metrics in addition to throughput
such as delay or loss
• Improve for investment based simulations (whatif scenario)
• Simulate multiple link failure (disastrous scenario)
Thank you!
For offline questions: [email protected]
Related Work
• Ye et al. :Large-scale network parameter
configuration using an on-line simulation framework
Tao Ye, Hema T. Kaur, Shivkumar Kalyanaraman, and Murat Yuksel. 2008. Large-scale network parameter
configuration using an on-line simulation framework. IEEE/ACM Trans. Netw
• Gonen et al. :Trans-Algorithmic search for automated
network
management
and
configuration
B. Gonen,
etal. Probabilistic
Trans-Algorithmic
search
for automated
network
management
and configuration.
In
IEEE International
Workshop
on Management
of Emerging
Networks
and Services
(IEEE MENS
2010
• Wang et al. :IGP weight setting in multimedia ip
networks
R. D. D. Wang, G. Li, “Igp weight setting in multimedia ip networks,”in IEEE Infocom Mini’07, 2007.