Transcript AT&T’s High Speed Packet Services Overview

Customer Experience &
Network Evolution Plans
Robert Calderbank
VP Research, AT&T Labs
Copyright AT&T 2003
EXPLOIT TECHNICAL INNOVATION
AT&T Labs
The Innovation Engine Behind AT&T’s World-Class Technology
• 6,500 of the world’s best scientists
and engineers
• AT&T’s patent portfolio includes
1,580 granted patents
Middletown, NJ
• 120 years of technology
breakthroughs and product/service
innovation
• Over 80% of our scientists &
technologists hold a PhD or other
advanced degree
• Currently involved with
approximately 90 U.S. & international
universities
AT&T Proprietary, Copyright 2003
Menlo Park, CA
Florham Park, NJ
EXPLOIT TECHNICAL INNOVATION
Directed Research Infrastructure is Accelerating Development of
End To End Solutions
Business Model*: Information and Operations Support to ABS that enables Customer Focused Operations
across all Networks and Services, and across the customer lifecycle. Rapid Response to
transform customer experience.
Business Problems: Unique capability to monitor current market and operational process
leading to dialog with Product and Operations that anticipates/frames the right questions and
collaboratively provides competitive advantage.
Data Integration:
Unique capability to
capture, integrate and
use diverse information
across silos, processes
and organizations at
full AT&T scale
Understand and
(Re)Define the
Problem(s)
Monitor & Control
Anticipate User’s
Needs
Data
Publishing
Create a Solution and
Iterate
“Test and Learn”
Business Solutions:
Unique capability to
build scalable, flexible
prototypes that can be
used immediately and
then improved based
on experience and
evolving needs
Enhance the
Infrastructure
Better/Quicker
Solutions each time.
*Shared across ABS and ACS
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
The Problem:
So many places to look; so little time
• Go to any work center and reps will be using lots and lots of systems
– Provisioning, maintenance, care
• Users want integration (one stop shopping)
– But large systems integration projects are expensive and risky
integration: benefits of integration without the costs
• VirtualRapid
cycle times: hours rather than years
–
• Why are reps using so many systems?
investigation
• Typical
– Log into many systems, and hope you have enough of a key to find something
– Tedious, expensive, often unsuccessful
scenario:
• TypicalCustomer
calls care and expects us to find their records quickly
–
– They don’t know how our databases are organized
– May not know product(s), primary key(s), spelling of their name in our DB(s)
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
VIP Architecture
DBOR: Current “Factory”
MetaSearch
Local Interfaces
External Interfaces
VIP GUI
ETE Process Models
• Simulation/Optimization
• Process Lifecycles
VIP Cache
Custom Views
Data Staging
DB snapshots
Web Crawlers
Detailed ETE Process
Monitor & Control
• Virtual Integration Tool
Direct Access
Data Access
ConnectVu
CARE
LIFE
COLR
PWOT
CSR
MACD
SCOT
Data Sources
Martin
BMP
PIC/CIC
Process Workflow
Process System Support
Current Legacy
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
VIT/VIP Usage
Number of Queries Daily
 Over 10,000 queries/day – LIFE and CSR pulls
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Current “Factory”
Integrated, Automated
“Factory” Process
ETE Process Models
• Simulation/Optimization
• Process Lifecycles
ETE Process Models,
Monitor & Control
Detailed ETE Process
Monitor & Control
• Virtual Integration Tool
Built off DBoR Access
Process Workflow
Process Workflow
Process System Support
Current Legacy
DBoR POR
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
AT&T’s Focus in 2003 and Beyond
Strategy
Reduce cycle time, consolidate
similar functions and systems, deploy workflow,
auto inventory, E-enablement, self-srv, Retire systems
Scrub DBORs, Deploy MPLS, VoIP
Better SLAs
Flexibility & Simplicity
Reliability & Security
Consistant & Predictable
quality of service
Customer Requirements
Basic + Managed
2002 - 2004
Predictive
2005 - 2007
System monitors, correlates
and recommends action
Adaptive
2008 - 2010
System monitors correlates and takes action
Cybernated
2011 - 2014
Cybernated Network - Integrated
Components, dynamically managed
by business rules/policies
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Business Grade Networking
Leveraging Scale
Terabytes/day
DPM
AT&T IP Traffic Growth: Blue
Internet Core Routes: Red
1998
96
97
Voice
98
99
Time
Frame
00
01
IP
AT&T Proprietary, Copyright 2003
BGP Routes
Traffic Crossing the Network and Active BGP Entries
Defects per Million
1999
2000
2001
2002
02
ATM
EXPLOIT TECHNICAL INNOVATION
Reliability and Performance of
AT&T Networks
The “discord checks”
embody the “rules” for
configuring the service
Web
reports
Automation
queries
eNetdb
Abstract
network
database
Discords
Low level
standard
Discords
form (tables)
polled
fixing errors
Customer Acquisition and Growth
• MIS Acquire the Traffic Program
– Analysis of daily usage and content mix
by potential customers, specifically
large content providers such as
Microsoft, Real Networks, and Speedera
• Customer Focused Operations – Signature
Client Program
– Significant contraction of the time to
onboard or migrate a network or
customer to an AT&T network or service
Router config files
Optimization of IP infrastructure in
AT&T MIS being upgraded to #1 ISP
from a preliminary ranking of 9th in
a survey of ISP conducted by
Boardwatch
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Major Applications – Backbone – Netflow Data
%flows/pkts/bytes by port number
Bytes
Flows
HTTP
http
nntp
smtp
0
ftp-data
napster
443
dns
4041
9995
4040
6970
kshell
1755
pop3
web-proxy
27005
napster
2048
5000
host2-ns
ftp-ctrl
1074
1044
6901
1050
1057
1027
1036
1049
6112
6701
2002
1042
2001
1025
28800
snmp
netbios-ns
31501
1672
4000
203
rest
telnet
1075
NNTP
AT&T Proprietary, Copyright 2003
2816
By 49608
4020
vid
771
Customer
EXPLOIT TECHNICAL INNOVATION
Gigascope – Application Layer Monitoring & Analysis
•
Gigascope - next-generation packet monitor
– Non-invasive
– Analyzes packet data at up to OC48 link
speeds
– AT&T’s GSQL language allows rapid
development of new queries
•
Example
– Monitored a particular customer
application with Gigascope to determine:
total number of active users, packet loss
rate, etc.
– Results being used to understand
network impact on application
performance, e.g., impact of packet loss
on user experience
– Loss rate on AT&T backbone is well
within limits
AT&T Proprietary, Copyright 2003
Gigascope
Optical
Splitter
EXPLOIT TECHNICAL INNOVATION
Getting to an Autonomic Network
• Provide predictive applications to intelligently integrate correlate, and act
on network information
– Detection (noticing problems as they occur)
– Diagnosis (identifying where and why the problem occurred)
– Repair (reliable analysis of possible changes to the network)
• A global view of the data is required to make this work
– Topology (routers, links, capacity)
– Traffic (offered load between points in the network)
– Routing (configuration of routing protocols)
• Use a data distribution bus and data warehouse to
– provide real time access to current and historical performance data
– obtain data off the data layer, rather than have each applications poll the network
– provide views of data for query or extract for non real time application needs such as
customer traffic studies
– Link to other DBORs for non-performance data (e.g., INSTAR for IP customer data)
• Use components with open interfaces and open data models, permitting
use of plug and play components at each layer
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Systems Architecture:
Instrumentation, Data, Application Layers
Product/
Sales/
Tier III
NFO/CFO/GNOC
Network Care
Reporting
Anomaly
Detection
Network
Management
(GCFP)
Network/
Customer
Traffic Studies
Capacity
Planning
Reports
Including
lightweight
publish/
subscribe
capability
Data Distribution Bus
Real Time
Performance Data
Capacity
Management
Historical Performance Data
Data Distribution Bus
Data Collectors and Active Probes
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
A Global View of the Data is Critical
What happened on these peering links? A problem or an improvement?
•
Without a network-wide view, see only "effects" of problems (e.g., change in link
load, degradation in performance), not root causes, and have no basis for
knowing how the network will behave after making a change.
IP networks use “hot potato” routing -packets take the “best” exit among
several choices, where “best” is partly
under our control, and partly under
peers’ and users’ independent,
dynamic control
•
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
SQL Slammer Worm: Why so potent
•
At a glance:
–
Installed itself on vulnerable systems
•
•
–
Generated pseudorandom IP addresses
–
Sent worm code to those addresses
Huge installed base of vulnerable code
–
•
•
MSDE software embedded in large number of other applications—130+ apps (e.g.,
Office XP, Visio)
Many systems did not apply available patch
–
Patches very difficult to apply in production systems
–
Many admins unaware of embedded MSDE in there apps
The Worm was built to probe the entire Internet
–
•
Exploited buffer overflow in SQL/MSDE server software
Addresses were generated more uniformly from entire address space than previous
worms like Code Red
The Worm was built for speed
–
cpu did little else but generate addresses and send worm payload
•
Saturated high-speed LANs which amplified its effects
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
What WAS the Worm?
Normal Traffic
Curve for UDP
Sat 1/18/03
“SQL Slammer”
worm strikes
Sat 1/24/03
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Worm Signature
• Used TAP traffic monitors to determine flow signature:
UDP flows of size 404 bytes to port 1434
• TAP infrastructure with “smart sampling” allowed us
to see this traffic accurately all across the network - in
real time!
• Also running in large customer network: was able to
quickly detect hundreds of infected hosts using above
signature and forward to customer for action
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Traffic Effects of the SQL.slammer worm
•
Worm used UDP-based traffic
– Majority of internet applications are TCP based (web, chat, news,
peer-to-peer file sharing)
– UDP traffic does not “back off” under congestion like TCP traffic
does. Thus, UDP traffic can “squeeze” TCP traffic under heavy
load:
• 1st hour after worm: TCP traffic was 22% lower than usual
• 4th hour after worm: TCP traffic was 14% lower than usual
• 24 hours after worm: TCP traffic was back to normal
•
Worm diffused across public and private networks
– Infection was anywhere the affected Microsoft software was
running; did not discriminate by network
– The worm only needed to breach one badly configured firewall to
go on to infect an entire Intranet
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Effects on Other Traffic
Normal WEB Traffic
Curve
Sat 1/18/03
“SQL Slammer”
worm strikes
Sat 1/24/03
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Traffic Effects of the SQL.slammer worm
Difference between “usual” Saturday TCP/web traffic and traffic on 1/25/03
AT&T Proprietary, Copyright 2003
EXPLOIT TECHNICAL INNOVATION
Systems and Networks: Evolving to the
Cybernated Network
Basic
Multiple networks and systems
Managed
Integration of data and actions
through management tools, and
intensive manual analysis
System monitors, correlates and
recommends actions
Adaptive
System monitors, queries as
needed for additional data,
correlates and takes action
Autonomic/
Cybernated
Integrated components,
dynamically managed by network
and business rules
AT&T Proprietary, Copyright 2003
work plan
Predictive
We are
here?
The target
is here
EXPLOIT TECHNICAL INNOVATION