Transcript Cisco Services - Courses

Jarrett Fishpaw, Business Operations Manager
Tyler Munger, NMO Lab Intern
UC Santa Cruz, ISM 101: Thursday, February 28, 2013
©
© 2010
2010 Cisco
Cisco and/or
and/or its
its affiliates.
affiliates. All
All rights
rights reserved.
reserved.
Cisco Confidential
1
• Jarrett’s Personal Background
• Cisco Systems Overview
• What is Cisco Services
• Cisco Services Technology Group
• Smart Analytics – ‘Actionable Insight’ TM
• The NMO Lab and Example Data Analytics Projects
• Day in the life of a Cisco Employee
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
2
• B.A. Business Management Economics (2009)
• M.S. Applied Economics and Finance (2010)
• Business Operations Manager, Cisco Systems
Contractor from June 2010 - April 2011
FTE since May 2011
• Mayor of Los Altos (2012)
Joined City Council in 2010
Job Role:
• 50% - Incubation and Innovation of new analytical models
Automated Router Configuration and Simulation
• 50% - Specific projects/Team Administration
Enhanced NPA
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
3
• At Cisco (NASDAQ: CSCO) customers come first and an integral part
of our DNA is creating long-lasting customer partnerships and working
with them to identify their needs and provide solutions that support their
success.
• Husband and wife Len Bosack and Sandy Lerner, both working for
Stanford University, wanted to email each other from their respective
offices located in different buildings but were unable to due to
technological shortcomings.
• A technology had to be invented to deal with disparate local area
protocols; and as a result of solving their challenge - the multi-protocol
router was born.
• Founded:
San Francisco, California, U.S. (1984)
Len Bosack, Sandy Lerner, Richard Troiano, Nicholas Pham
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
4
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
5
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
6
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
7
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
8
Q1 FY'13 Revenue: $11.9 billion
Q1 FY'13 Employee Count: 72,360
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
9
Technical
Assistance
Center
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
10
• Employees Worldwide:
11,400+
• Certified Professionals:
CCIE= 2,000+
• Patents
~700 since 1998
• Cisco Partners
• Services is responsible for 80% of the revenue coming from Cisco Partners
• There are over 280,000 partner employees which provide channel leverage
• Over 68,000 authorized/registered partners
• Provide support in over 180 countries worldwide
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
11
• The Cisco Services Technology Group drives technology
innovation and software development to accelerate the evolution
of our Advanced and Technical services portfolio.
• Our global organization consists of five primary groups:
Technology Development
Support
Business Operations
Globalization
Expertise
• We are accountable for developing, delivering, and supporting the
technologies Cisco Services deploys in our markets.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
12
Who Are These People?
• Think of him or her as a
hybrid of data hacker,
analyst, communicator,
and trusted adviser.
• The combination is
extremely powerful—and
rare.
© 2010 Cisco and/or its affiliates. All rights reserved.
What are their traits?
• Intense curiosity
• Associative thinking
Credit Card fraud – DNA
sequencing
Finance - Pharmaceutical
Trials - Network Reliability
Cisco Confidential
13
Role of Analytics in Complex Networks
Analysts
Competitors
IBM opens a Services Innovation Lab
(SIL)…the lab will accelerate the
expansion of real-time analytics … in both
IBM’s technology services offerings and its
global services delivery capabilities.
(4/2011)
IBM has committed $2 billion in service R&D
Partners & Customers
IT complexity will soon outstrip customer’s IT
management capabilities – Forrestor
Network managers have more information than
ever … Making sense of the information is the
essential challenge… trends will see the
effective transition of development priorities from
network technology to IT analytics
– IDC, 3/2011
© 2010 Cisco and/or its affiliates. All rights reserved.
Smart Analytics represent an excellent
delivery of intellectual capital…This type of
IC scored in the top 3 of partner demands
in the partner focus groups regarding
…desirable service elements
- Market Intelligence Team,
5/2011
Cisco Confidential
14
• Understand network
health
• Set and measure
progress against
network
improvement plans
and change
initiatives
• Compare against
peers in the same
vertical on certain
metrics such as
technology maturity
and operational
maturity
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
15
• Big Data requires:
Hadoop
SAS
R
Denodo
• Domain Experts Collaborating with Statisticians
• ‘Big Data’ combined with Analytics
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
16
Process data using advanced tools, and domain expertise
Techniques:
Data Clustering
Data Mining
Correlation Analysis
Time Series - ARIMA Models
Predictive Forecast
Multivariate Regressions
Cross Sectional Data
Historic Trending
Linear Regression
© 2010 Cisco and/or its affiliates. All rights reserved.
Multivariate Regressions
Curve Fitting
ARIMA – Autoregressive Integrated Moving Average
Cisco Confidential
17
Delivery of Intuitive Insight
• Industry Benchmarking
• Root Cause Analysis
• Predictive Forecasting
• ROI Simulation
• Visualization
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
18
• Projects: Short-term undergraduate internships (1-3 quarters) and Masters/PhD
level research (3+ quarters) at Cisco
•
Wide range of interesting projects from hardware testing to Supply Chain
Management
•
Real world data and problems
•
Potential users (work has a high possibility of actually being used)
• Time: Generally 20 hours a week. Usually work from home and go into Cisco
about once a week
• Resources
•
Networking labs (UCSC and Cisco)
•
Servers
•
Domain experts
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
19
• BS in Computer Science from UCSC (2009)
• Graduate Student in the Technology and Information Management (TIM)
program working under Subhas Desa
• Been part of the NMO lab for 4 years
•
1 year as an undergraduate student
•
3 years as a graduate student
• Research areas
•
Data mining/Information retrieval
•
Software engineering
•
Computer networks
•
Knowledge engineering
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
20
• When a customer has a problem with a Cisco
“We have one ASA with 2 ISP
networking product they open a service request
connections and several site to
site VPN’s set-up to other sites
with Cisco’s support teams
with ASAs. When we test failing
• Cisco support teams receive thousands of service over the VPN’s to use the second
ISP connection the tunnels are
requests every month on a wide variety of Cisco
not coming up. We have
networking products
confirmed that outbound routing
fails over immediately.”
• How do we detect emerging (growing) problems
before they impact a large number of customers?
•
Software bugs
•
Viruses
•
Hardware failures
Typical Customer Service Request
Primary Investigator: Tyler Munger
Advisors: Chris Wong (Cisco), Subhas Desa (UCSC)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
21
1. Data Mining: identify the associated product failure mode for customer
service requests
•
Support Vector Machines (classification)
2. Prediction: generate an alert when a particular failure mode starts
abnormally increasing in frequency (indicating an emerging problem)
•
Double Exponential Smoothing (forecasting)
•
Control Charts (alerts)
3. Software Engineering: develop an easy to use software tool that allows
Cisco Engineers to monitor customer service requests for emerging
problems
•
Weka (data mining)
•
Ruby on Rails (webserver)
•
MySQL (database)
•
Adobe Flex (visualization and GUI)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
22
• Identified 20 key failure modes for a
popular Cisco network security
product (ASA 5505)
• Classified a dataset of ~10,000
service requests by failure mode
using a Support Vector Machine
classifier (SVM)
• Dominant failure modes were
“failover communication between
devices”, “VPN tunnels
disconnected after switchover”,
“configuration not in sync after
switchover”
© 2010 Cisco and/or its affiliates. All rights reserved.
Failure Mode
Frequency
Failover
communication
between devices
1414
VPN tunnels
disconnected after
switchover
922
Configurations not in
sync after
switchover
609
No switchover
450
Hardware
224
…
…
Cisco Confidential
23
• Developed a web-based
software tool for Cisco to
use for monitoring emerging
issues
• Used the tool to identify
emerging problems in
historical customer service
requests
•
Increase in Virtual Private
Networking (VPN)
problems caused by a bug
in a new software release
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
24
• Customer: increased network reliability because potential problems are
identified and resolved before they occur
• Cisco: increased customer satisfaction and reduced support costs
because customers are opening less service requests
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
25
•
Cisco Internet Operating Systems (IOS) is the core piece
of software that runs a wide range of Cisco networking
products (routers, switches, firewalls, etc.)
•
There are hundreds of different IOS software releases
•
•
Older releases are more reliable but have less features
•
Newer releases have more features but are less reliable
How do we recommend the most reliable IOS software
release that has the features that the customer needs?
Primary Investigator: Yali Mu
Advisors: Jarrett Fishpaw/Chris Wong (Cisco), Subhas Desa (UCSC)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
26
Total Number of Bugs
Unfound Bugs
• The reliability of an IOS software
release can measured by the number
of software bugs that have not been
found yet (unfound bugs)
• How do we estimate the number of
unfound bugs for each IOS Software
Release?
1.
Create a bug forecast using historical
bug data
2.
Determine the total number of bugs for
the software release based on the
forecast
3.
Calculate the number of unfound bugs
• Develop an easy to use software tool
that allows Cisco Engineers to easily
select the most reliable software
release for a particular customer
network
© 2010 Cisco and/or its affiliates. All rights reserved.
Bugs Found
Forecast
Cisco Confidential
27
• Created reliability forecasts for over 5,000 IOS software
releases across 36 different product families
• Forecasts were used to create 3 reliability estimates for
each IOS software release
1. Total number of bugs
2. How many unfound bugs
3. When all the bugs will be found
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
28
•
Customers: network reliability improves because they can select the
most reliable IOS software release for their network
•
Cisco: increased customer satisfaction and reduced support costs
because customers have less network problems
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
29
• Many employees have a unique and flexible work schedule
• WebEx Meetings
• Outlook Calendar
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
30
• Each manager has a different style
• Responsiveness to Email and IM’s (Phone/text occasionally)
• Personal discipline to manage multiple projects simultaneously
• Weekly Breakfast Club on our team
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
31
Cisco Business questions
• What does Cisco do?
• Are Cisco products food or internet related?
• Do you use Cisco products?
• What does a Data Scientist do?
• What year was Cisco Founded?
• How large is Cisco (people/revenue)?
• How many authorized partners does Cisco have?
Cisco Certification questions
• Name 2 Cisco networking certifications
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
32