Transcript March, 2008 Workforce Continuity Briefing

Business Continuity &
Disaster Recovery Solutions
Workforce Continuity
Briefing for Virginia Business Recovery Association
GLOBAL CAPABILITY.
PERSONAL ACCOUNTABILITY.
Jonathan Nguyen-Duy
Verizon Business Continuity Services
February 21, 2008
© 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
PROPRIETARY STATEMENT
This document and any attached materials are the sole property of Verizon and are not to be used by you
other than to evaluate Verizon’s service.
This document and any attached materials are not to be disseminated, distributed, or otherwise conveyed
throughout your organization to employees without a need for this information or to any third parties without
the express written permission of Verizon.
The Verizon and Verizon Business names and logos and all other names, logos, and slogans identifying Verizon’s products and services are trademarks and service marks or registered
trademarks and service marks of Verizon Trademark Services LLC or its affiliates in the United States and/or other countries. All other trademarks and service marks are the property of
their respective owners.
2
Agenda
• BCDR/COOP Challenges
– Key issues
– Extended global enterprise
• Verizon Business Services
– Natural disasters
– Pandemic flu
– Remote working
– Workforce continuity
– Practical solutions
• Summary
– Key insights and recommendations
3
History of disaster recovery and business continuity
In the beginning…….
– Mainframe computer-based only
– Technology driven
– Disaster recovery only
– Expensive subscriptions and recovery
– Long term recovery questionable
– Hard sell to management
And now…..
– Business driven
– Disaster recovery and business continuity, both required
– Regulatory requirements
– Enterprise-wide
– Telecommunications requirements critical
– Cost of doing business
4
Verizon Business
BCDR/COOP & The Extended Enterprise
– Expanding perimeters and interdependencies means
more opportunities for interruptions
»Natural disasters
»Cyber threats
»Biological threats
»Change management
»Power outages
»Supply chain dependencies
– Increasing exposure and risks
»Compliance
»Lost revenue, capitalization
»Public trust
– Requires all hazards approach
»Network security
»Physical security
»BCDR/COOP planning
»Applications resilience
»SCM
5
Verizon Business
Business Continuance and Emergency
Management Capabilities
GLOBAL CAPABILITY.
PERSONAL ACCOUNTABILITY.
© 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
Verizon Business
BCEM Program
• Global business continuity planning organization
–
–
–
–
–
–
35,000 employees
321 offices in 75 countries across six continents
Global IP footprint serving 2,700+ cities in 150 countries
More than 200 state-of-the-art data centers in 22 countries
Unique capabilities MERIT Team
Partner with federal agencies and task forces
• Capable of addressing local, regional and international events
– Automated BCP tools
– Integrated multi-functional approach
»Network Operations
»Managed Services
»Security
»Customer Support
»Public Relations
• Certified Business Continuity professionals, processes, and products
– Utilize federal and industry standards
»National Incident Management System
»Disaster Recovery Institute and Business Continuity Institute
7
Verizon Best Practices
ERT and MERITSM Teams
• Verizon Business Emergency Response
Teams (ERT’s) support tactical recovery
operations in each geographic region
• Major Emergency Response Incident Team
SM
(MERIT ) - Verizon’s fully capable hazardous
materials team.
– Technical systems restoration
– Outside plant and facilities engineering
– Customer equipment restoration
– Environmental health and safety
• Capable of entering and recovering in any
hazardous environment – Asbestos, Train
Derailments, Major Flooding, Glycol Spills,
Anthrax Contamination.
8
Verizon Best Practices
Mobile Communications Command Center
• 40-foot and 53-foot self-contained
vehicles.
• 24 individual stations
9
• VoIP telephones, wireless IP telephones,
and both wired and wireless Ethernet
access
MC3 Fleet Deployments
• Free Local, long distance, and
international calls, as well as access the
Internet and send or receive faxes and
televisions
• Troop Deployment, Fort Carson, Colorado, February
– April 2004
• Based in Richardson, Texas. However, a
MC3 can reach either coast within 24
hours.
• Missouri Tornado, Pierce City, Missouri, May 2003
• Hurricane Katrina, New Orleans and Gulf Coast,
August – September 2005
• Hurricanes Charley, Frances, Ivan, and Jeanne,
Florida area, August – October 2004
• Troop Deployment, Fort Hood, Texas, January –
March 2004
• Southern California Fires, San Diego, November
2003
• New York City, 9-11, September 2001
Verizon Best Practices
Workforce Continuity
• Prepared for a wide range of business interruptions
• Enterprise-wide skill set inventory for 247K+ employees
• Mitigation Strategies
– Remote Working
– Management of IT Systems and Applications
– Automation Tools
– Work from Home
– Staggered Shifts
– Hygiene and Cleaning Practices
• Response Strategies
– Prioritization of Critical Business Processes
– Identify the Most Important Functions
– Allocate Resources for Continued Operations
– Contractor/Vendor Support
– Global Work Shifting & Evacuations
10
Verizon Business
Pandemic Response
GLOBAL CAPABILITY.
PERSONAL ACCOUNTABILITY.
© 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
Pandemic Challenges
• Can you operate with minimal contact between staff and between staff and
clients?
– Minimizing contact
– Mitigation via teleworking, Web conferences, etc.
– Effectiveness of remote working
• Can you effectively operate with key staff incapacitated for months or
permanently?
– Loss of personnel
– Develop autonomous decision making processes.
• How will you operate if your supply chains are disrupted?
– Assess and plan for multiple scenarios.
– Build redundant capabilities.
– Precontract & predeploy resources
12
Current WHO Pandemic Alert Phase
• Stage 3 – No or very limited human-to-human transmission
13
Known cases of H5N1 as of February 15, 2008
• 361 cases
• 227 deaths
• 62% Mortality
• Middle East
• China
• SE Asia
• 02/13/08 latest death
reported in Indonesia
– Symptoms Feb. 2nd
– Died Feb. 13th
Source: World Health Organization, 2008
14
Planning for Pandemic Influenza
• Verizon has served on multiple federal, state, and local pandemic
planning programs
• 2005 Government and industry first planning efforts
• 2006 – 2008 National security escalation
– U.S. Chamber of Commerce
– National Telecom Security Advisory Committee
– National Infrastructure Advisory Committee
– DHS
– Verizon
15
Verizon Best Practices
Global Influenza Pandemic Planning
• Verizon Business has developed:
– A Global Influenza Pandemic Plan (“GIPP”)
– Preventative and responsive strategies
»increased absenteeism in the later stages of the pandemic
»quarantine requirements
»worker infection
»care for infected family members
»child care in the wake of school closings
– Pandemic Alert System (complements WHO Alert Phases)
• Verizon Business Pandemic Planning
– 24x7 monitoring – DHS, CDC, WHO etc.
– 50% staff absences
– two weeks at the height of a pandemic wave and lower levels of staff absence
for a few weeks on either side of the peak (approximately 8 week period).
16
Verizon Business – Overall Pandemic Strategy
Stage A
Stage B
Stage C
Stage D
Reports of influenza virus
affecting certain animal
populations, but not
occurring in countries where
Verizon Business facilities
are located.
Influenza virus affecting
animal populations in
countries where Verizon
Business facilities are
located, and there are
confirmed cases by the
WHO of animal to human
transmission.
Influenza virus has mutated
and there are confirmed
cases by the WHO of human
to human transmission in
countries where Verizon
Business facilities are
located.
The WHO declares an
epidemic or pandemic in
countries where Verizon
Business facilities are
located.
World Health Organization
Inter-pandemic Period
Phase 1
17
Phase 2
Pandemic Alert Period
Phase 3
Phase 4
Pandemic Period
Phase 5
Phase 6
Alert Notification Services
Traditional Approaches Not Adequate
• East Africa Embassy Bombings
– Manual Call Trees
– 25 people working full time
– 12 days to contact 480 people
• Beirut U.S. Evacuations
– Manual Calling Trees
– Limited to landlines and wireless
– No polling capability
• Fortune 100 NYC Financial Services Firm
–
–
–
–
September 11th couldn’t reach 6000 employees
Manual Call Trees and Broadcast Voicemails
8 days from alert to acknowledgement
No multi-modal capabilities
• Future situations could be worse
18
Alert Notification Solution Requirements
Create alert notifications dynamically for rapid, simultaneous delivery to employees,
first responders, and customers.
• Priority Delivery – Per customer needs.
• Phone, Web and BlackBerry – At any time.
®
• International Delivery – 125 countries, with time zone awareness.
• Interactivity – Respond or transfer to an operator or live conference call.
• Message Escalation – Ensure message reaches the intended recipient.
• Delivery Manager – Avoid overloading phone systems or switches.
• Message Center – Retrieve messages on answering machines or posted to employee
dial-in numbers.
• Tracking and Reporting – Instant view of message delivery and response. Generate scheduled and ad
hoc reports.
19
Verizon Business Continuity Plan in Action
Enabling the Resilient Enterprise
1
2
3
4
Hurricane approaches company executes its
Business Continuity
Plan.
Managed BC planning
application sends
documents to Incident
Response Teams.
Web-based Operations
Center tracks incident,
coordinates response.
Integrated alertnotification system
sends voice and text
messages to GPS
enabled wireless
devices being used to
track vehicles and
personnel.
Employees use wireless
broadband to access
private IP for email &
collaborative apps,
avoiding congested
public IP networks.
Verizon Business Continuity Plan in Action
20
Workforce Continuity Insights
GLOBAL CAPABILITY.
PERSONAL ACCOUNTABILITY.
© 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
Regional Workforce Continuity Challenges
Communications Networks
• Will telecommuting work during a crisis?
• Are commercial networks prepared?
What we know…
• Most enterprises have not planned for workforce continuity
• The commercial, best effort, residential Internet access structure
may not handle increased load
• Impossible to precisely predict what would happen
• Networks are largely automated and redundant
• 99.999% level of availability = minimal impact on critical operations
• Shift in traffic patterns or bandwidth demand will cause congestion
• Congestion during peak hours 3:00 – 8:00 PM
• Congestion at aggregation points near residential communities
22
Regional Workforce Continuity Challenges
Communications Networks
23
Source: Department of Homeland Security
Regional Workforce Continuity Challenges
Communications Networks
24
Source: Department of Homeland Security
Regional Workforce Continuity Challenges Remote
Working Requirements
• Remote Working Connectivity
– Up-to-date PC & software
– Broadband Internet access
– Security – tokens, PKI cards,
passwords
– Voice – land lines, cell, VOIP soft
phones
– Voice-Mail Forwarding & unified
messaging
– Updated contact information
• Remote Working Infrastructure
–
–
–
–
UPS & batteries for all devices
Dedicated workspace
Applications
Signed Telecommuting/Contingency
Work Agreement
– Electronic media storage
»CD and DVD media drives
»Zip disks, memory sticks, flash keys
– Shipping and receiving processes
25
• Alternate Worksite-Office Basics
– LAN – Internet access, routers, firewalls,
etc.
– Telecom
– Printers, copiers, faxes
– PDAs, cell phones, radios
– Monitors, PCs, laptops
– Task lighting
– Office supplies
– Office furniture
– 2 sq meter work area
– Secure storage and disposal of records
– PPE & hygiene supplies
Regional Workforce Continuity Challenges
Remote Working Strategies
26

During disasters, communications industry focuses on service
maintenance rather than order fulfillment
 Provisioning moratorium
 Provisioning intervals will increase
 Training time for employees will increase

Increase your business process intervals

Redundant systems, back up sites, employee communications
and alternate work sites
 Consider how employees can access alternate facilities
 Consider an alert notification system

Surge demand for network resources and call center services
 Remote access support
 Demand for HR information

Shift bandwidth intensive process to non-peak hours

Filter bandwidth intensive attachments on email
Service Restoration Priority
Plan, prioritize and pre-deploy
1. Verizon Communications needs to manage recovery
2. Telecommunications Service Priority (TSP) service
3. Essential Government Services
4. Public Safety Services
5. Network Infrastructure
6. Priorities of Federal, State, and Local governments
7. Other services
27
Remote Work
Practical Solutions
 Telecommunications Priorities
 Telecommunications Service Priority
 Wireless Priority Services
 Government Emergency Telecommunications Service Cards
 First Steps
 Mission Critical Processes and Applications
 SCM, CRM, ERP
 Systems and applications
 Bandwidth, access, hardware requirements etc.
 Skill-sets and personnel
 Secure Remote Access Considerations
 Private networks and Wireless broadband access
 Access Continuity – fiber to satellite broadband
 Quality of Service and SLAs
 Connect to private MPLS network to avoid local public congestion points
 Real-time automatic desktop backup – restoration & replication
 Enhance security and help desk services
28
Remote Work
Practical Solutions Wireless Broadband Access
• Purchase a PC Card, ExpressCard, or USB
Modem to connect your personal computer to
wireless broadband network
• Impact Assessment
• Procure and support
– Hardware
– Software
– Security
– Help desk
– Wireless services
Broadband service
Faster downloads — typical speeds of 600 Kbps to 1.4 Mbps 1
Faster uploads — typical speeds of 500-800 Kbps 1
Lower latency — improved performance for many applications bursting up to 144
Regular service - bursting up to 144 Kbps, with typical speeds of 60 to 80 Kbps
29
Remote Work
Practical Solutions Wireless Router
– Failover connection
Broadband Access Wireless Router offers a failover option
– Simplified deployment and may be more cost-effective than ISDN or other wireline
solutions.
– Primary connection
Broadband Access Wireless Router provides a cost-effective and reliable
alternative to terrestrial wireline connections
– Other remote applications
Broadband Access Wireless Router can be used for remote banking and ATM
machines, remote-location cash registers, or mobile applications or portable
communications kits for temporary sites.
30
Verizon Business
Mobile Office Solution
 Quickly deployable
 Highly mobile
 Access to Voice, Data,
and Network
(VPN/Internet)
31
Verizon Business Mobile Office
Deployment Options
Fixed:
 WAN Connection Can Be Terrestrial or
Satellite Based
 Radio Interface For Two-Way Mobile Radios
Fixed mobile:
 Ease Of Configuration And Setup
 In The Field Very Quickly
Swift deploy:
 Rapid Setup
 Rugged Cases
32
Verizon Business
Mobile Office Edge Detail
Remote
Satellite
Receiver
802.x
Wireless PC
PC
Wired
VoIP
Phone
Satellite Hub
LAN Switch
Or T1 Access
Verizon Business
VoIP Server
®
Verizon Internet
®
Edge Access
Dual Ethernet/T1
Access Device
Mix of 4-24
Wired Analog
Phones or
OPX Lines
Corporate
VPN
Edge = Customer Edge Router (edge access equipment depicted above)
33
802.x
Wireless IP
Phone
Conventional Configuration vs.
Verizon Business Mobile Office
PCs
PCs
PBX
Private
Line
Network
PBX
Video
Router
Router Video
VPN
Frame
Relay
Network
ATM
Network
Satellite
Gateway
Internet
Verizon
VOIP
®
Router
Terrestrial T1/E1/DSL
Router
Video
PBX
PCs
Video
PBX
Traditional Frame Relay,
ATM, or Private Line Networks
34
PCs
PBX
Edge Access
Router
®
Video
Verizon Business Mobile Office
Access Continuity
Satellite Services
• North America & EMEA coverage
• IP over satellite technology
• QoS for real time and critical applications
• Built-in router functionality
• VLAN & VRF compatible for private transmission
• Seamless integration with (MPLS) Private IP, true Private IP end-to-end
• Private IP Satellite Automatic Disaster Recovery
• Redundant Hub locations – large customers can be split among
two Hubs
• Geographic Hub diversity – Hubs located geographically apart
for disaster mitigation
• Terrestrial redundancy – two diverse Hubs with diverse
connectivity
• Auto DR – Remotes on one Hub can automatically fail over to
the other if Hub fails
• Site visits – dispatches not needed to re-point in case of Hub
failure
35
Access Continuity
Typical Satellite Applications
• Private IP (PIP) and Public Internet Access
• Disaster Recovery and Business Continuity
• Point-of-Sales (POS)
• Data, VoIP, & Video over IP
• FTP (File Transfer)
• Web access/browsing, HTTP
• Lotus Notes, Oracle
• SAP, Citrix version 1.8 and newer
• SSL (an application layer VPN)
• TCP and Web Acceleration over VSAT IP network
• Integration & Support Verizon Wireless Applications
36
Satellite Services
Disaster Recovery Overview
• Provides true Private IP (MPLS) disaster recovery
• VSAT acts as a backup to terrestrial Private IP (PIP)
• Upon failure, the VSAT automatically takes over and all
routing updates are sent via the VSAT link
• Upon restoral, terrestrial link is primary and VSAT again
becomes backup, also automatically
• Customers grouped and sized according to their backup
needs and number of sites
37
Satellite Services
Hurricanes Katrina/Rita Backup & Restoral
• Restored critical services for the FAA, Army,
Navy, insurers and other customers after the
hurricanes.
• Voice, video, insurance applications and
government services.
–Flyaway Trailers for:
»FEMA
»Army
»Navy
• Rapid deployment of fly-away VSAT systems
including trailers with voice, data and fax service
to a variety of gulf locations.
• Some locations only had our trailer for all means
of communication
38
Private IP Satellite and PIP
National Retailer Disaster Recovery Advantage
39
Disaster Recovery via Private IP Satellite & Verizon
Wireless Network
40
Business Continuity Example
Wireless IP Enables Mobility and Disaster Recovery
• Re-establish agency service in
areas struck by disasters
• Re-route to temporary location
or remote home workers
• Help reduce total downtime
• Rapidly set up permanent or temporary
claims centers
– Satellite and EVDO high-speed,
secure wireless
Source: Nemertes Research Group Inc., 2006
VoIP
41
Mobile
Phone
IM
Unified
Messaging
Remote
Home Workers
VoIP Network
Verizon Business
BCDR/COOP Challenges
Summary
GLOBAL CAPABILITY.
PERSONAL ACCOUNTABILITY.
© 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
911, Katrina and London Bombings Insights
• SCM and interdependent systems increase risk
• Integrate network and COOP plans
• Third-party supplier and partner relationships
• Lodging, food, supplies, etc.
• Plan for “worst case” and regular scenarios
– Takes time to successfully implement a resilient enterprise
– Deploy crisis communications system
• Factor in data recovery and storage into plans
– What is your recovery time objective
– What are the critical systems, processes and skill sets
• Purchase and deploy communications assets
• Be flexible – more process than product
43
24