Transcript Dealing With The Dark Side

Dealing With The Dark Side
Managing The Malware Menace
Jenifer Jarriel
Vice President of Information Technology
and Chief Information Officer
Baylor College of Medicine
Copyright Jenifer Jarriel, Baylor College of Medicine, 2004. This work is the intellectual property of
the author. Permission is granted for this material to be shared for non-commercial, educational
purposes, provided that this copyright statement appears on the reproduced materials and notice
is given that the copying is by permission of the author. To disseminate otherwise or to republish
requires written permission from the author.
Agenda
Review of 2003
The Costs of Viruses and Spam
Baylor Attacked! What It Has Cost
Baylor Case Studies
How Did This Happen
Defeating The Dark Side
– Goals of the initiative
What has been completed
What is being developed
– Approach of the initiative
Funding The Force
– Costs for implementation
– Costs savings examples
Summary
Questions & Answers
Review of 2003
2003 was the worst year in computer virus history
Jan 2003, Slammer worm contributed to the biggest
Internet attack ever!
– Scanned through all 4 billion public IP addresses in just 15
minutes!
BugBear.B, Welchi, & Sobig.F worms released between
June and August
– Sobig.F was the worst e-mail worm ever, sending over 300
million infected emails around the world
In August 2003, BCM experienced major attack, losing
connectivity to affiliates, affecting over 1/3 of Baylor
community, and costing over 100K in remediation efforts
Viruses
Images courtesy of Virtual Virus Library
This isn’t just a biologic problem……..
Viruses
Recent Research
– Costs (Computer Economics, 2002)
Year Code
2003 SoBig
2003 Blaster
2003 Slammer
2001 Nimda
2001 Code Red
2001 SirCam
2000 Love Bug
1999 Melissa
1999 Explorer
Worldwide Economic Impact
$ 13 Billion
$ 2.5 Billion
$ 635 Million
$ 2.62 Billion
$ 1.15 Billion
$ 8.75 Billion
$ 1.10 Billion
$ 1.02 Billion
– SoBig Was/Is Faster Spreading Worm EVER! It infected 1 out of every
17 messages (InformationWeek, August 2003)
– Blaster worm infected 30,000 PC’s an hour
– August 2003 saw 75 new Internet viruses categorized at least as a minor threat
(Computerworld, August 2003)
Viruses (continued)
Recent Research Continued
– Recent research suggests 113 virus encounters a
month (Computerworld, March 2003)
– On average 23 staff days were required for virus
disaster recovery (Computerworld, March 2003)
– Average cost to companies was $81,000 (ICSA, 2003)
Spam
It’s not just a curious food product…..
Spam
Spam
– In January 2003 approximately 42% of all email was spam, but
by December 2003 that number had risen to 58%
(Internetnews.com, January 2004)
– In 2004, spam is expected to comprise 65% of all email
– From May-Dec 2003 Baylor IT found 42%- 46% of all email was
spam
Hoaxes
– Jdbgmgr.exe was first reported in April 2002 (Symantec)
– Users at Baylor are still receiving this email message and still
responding to it despite multiple warnings
– “Hoax viruses cost companies and people as much in terms of
loss of productivity as the real virus.”
Baylor Attacked!
How often does this happen at Baylor
– 4,015 infected files caught by BCM IT between January and March 2003
What it cost in resources (Just during August 12-22 alone)
– Over 2100 PC’s were infected by the Blaster and SoBig worms and were
fixed by IT. More were infected in departments with SA’s.
– Estimates suggest over 1/3 of Baylor was infected
– Over 2000 IT man hours spent fixing these viruses
What it cost financially (Just during August 12-22 alone)
– Approximately $60,000 (IT only) spent on virus remediation during
August 12-22 and this is still growing (approx 100k+ now)!
Estimated productivity loss of 2.5 days for 2000 employees
Estimated financial loss due to decrease of productivity =$2,400,000
Helpdesk volume went from 100 calls per day to over 750 calls
Case Study
Worm Attack Began
Case Study (cont)
As can be seen from the illustration on the previous slide, the
recent worms utilize ICMP (Internet Control Message Protocol) to
actively scan the network looking for vulnerabilities.
Since the events in August, activity has dramatically increased
from just under 200 flows/sec to over 1000/flows sec.
As a result, any system that connects to the network and is not
appropriately patched, will likely become infected and will try and
infect other vulnerable systems.
How Did This Happen
Review of IT infrastructure
– 39% of all Baylor PC’s no longer have mainstream
support from Microsoft as of December 31, 2003
These include Windows 3.x, 95, 98, ME, NT
– Unknown (but significant) number of computing
systems using desktop software that are either no
longer supported, or will no longer have support by
Microsoft as of December 31, 2003
These include Office 95, 97, & Office 98 for Macintosh
– Every Wednesday Microsoft typically releases a new
critical update for their OS’s
How Did This Happen (cont)
Review of IT infrastructure (continued)
– Automatic updates are only available for Windows
2000 and XP. Typically this is disabled
Review of policies
– There is no formal policy mandating Norton Antivirus
be installed
– There is no standardization policy for desktops or
servers (OS, Hardware, common Applications)
– There is no patch management policy to update
Baylor computing systems (desktops and servers)
Goals
The goals of the IT Asset Management &
Planning Initiative are the following:
– To support a diverse environment in an
efficient and cost effective manner
– To provide a safe, secure, and reliable
network environment
– To be proactive in the management of
network intrusions, patch management, and
asset inventory
What’s Completed
Two commercial anti-spam products were
evaluated (IronMail, SpamAssassin)
A phased implementation of commercial antispam products was initiated
Increased collaboration with user community
Gained approval for Wireless Network policy
Gained approval for Virtual Domain policy
Gained approval for Web Server Management
policy
What’s Completed (cont)
Upgrade Norton Antivirus
– NAV for Mac OS X now available
– Currently testing NAV 8.1 for future rollout
Increased Coordination with Affiliated Institutions
Blocking of IP Addresses for Infected Systems
Proof of Remediation for Infected Systems
System Administrators Roundtable During Crises
What’s Being Developed
BCM IT developing proposal to implement and
receive funding for the following:
– Hardware, software, OS standards
– Disaster recovery/business continuity plans for
critical systems and applications
– Secure BCM network perimeters
– Implement secure messaging
– Automatic updates
– Firewall clustering
– Additional supporting policies and procedures
What’s Being Developed (cont)
BCM IT is developing the following policies
and procedures:
–
–
–
–
Corporate Anti-virus Software Policy
Network scanning Policy
Use of non-BCM equipment Policy
Lifecycle Program for Hardware, Software, and
Common Applications Policy
– Asset Management Policy
– Patch Management Policy
– Enterprise Directory Services Policy
Antivirus Requirement
Recommendations
– Require all IT systems capable of participating (UNIX
and Linux do not at this time) in the College IT
managed Norton Anti-Virus (NAV) system do so
– Replacement of current non-IT managed anti-virus
software with the Norton Anti-Virus
– Assurance the most current version of NAV running
and that users cannot disable
– Ability to schedule a mandatory scan of all
desktops/servers if required
Network Scanning
Recommendations
– IT authorized to conduct scans of computing
systems attached to the IT network as
necessary and appropriate
– Continue to collaborate with departments to
assure non-disruption of computing systems
Non-BCM Equipment
Recommendations
– Modify Acceptable Use Policy to prohibit use
of non-BCM computers
– Exceptions authorized by IT based on justified
business need
– If authorized, non-BCM computer’s require
IT managed anti-virus installed
Conform to set IT security requirements
Lifecycle Management
Recommendations
– Baylor College of Medicine requires that BCM computing assets,
with the exception of servers, be replaced on a four year cycle.
– Servers are to be replaced on a four year cycle with
initiation/planning beginning in year three.
– Operating systems and applications must be upgraded or
replaced to the latest version supported by the manufacturer.
– Replaced or upgraded computing assets, operating systems and
applications must be procured through BCM Purchasing.
– BCM computing assets lifecycle will be tracked by the
Information Technology Assets Inventory Management System.
Asset Management
Recommendations
– All BCM Computing Assets must be registered with the
Information Technology Enterprise Asset Inventory Management
System before they can be connected to the BCM Network.
– Computing systems on the Baylor network must be able to report
to the asset management system the following information:
Responsible organizational entity for the system.
Purpose of the system.
Name of the hardware supplier.
Warranty and maintenance information
Operating system installed
Applications installed.
Hardware configuration and peripherals installed.
Security sensitive information such as presence of confidential data.
Patch Management
Recommendations
– Before connecting a new or rebuilt system to the BCM network, any
initial critical patches must be installed on the system. A CD of the
current initial critical patches can be obtained from the departmental
system administrator or by contacting the Information Technology
HelpDesk at 713-798-8737 or [email protected].
– All desktop and laptop computing assets (home and network) must
enable automatic installation of critical patches on their operating
systems.
– Any non-BCM computing asset must have installed all critical patches to
their operating system before connecting to the BCM network.
– All servers should begin testing critical patches for either the operating
system or any applications on that server within 24 hours after being
released, and installed within 72 hours.
– The departmental system administrator or Baylor College of Medicine’s
Information Technology program may require that certain non-critical
patches should be applied. If the notification applies to a system, the
patches will need to be applied.
Enterprise Directory Services
Recommendations
– All organizational units of Baylor College of
Medicine that maintain Microsoft servers or
desktops will be required to be a part of Active
Directory.
Approach
Phased implementation
Implement tools to accurately define project
scope and pilot patch management
Requires upgrades on hardware and software
Initiates life cycle management
Requires funding commitments on an annual
basis
Phased Approach Implementation
Guidelines
A Three-Phased approach is recommended:
– Phase 1 Implement Asset & Patch Management
Solutions
– Phase 2 Central Departments
Finance, HR, Office of Development, Legal, IT, Investment,
Public Affairs, Facilities, Administration, Office of the
President, Office of the COO
– Phase 3 Clinical, Research, and Education
Departments
Phased Approach
Implementation
1.
2.
Phase 1 Recommendations – FY 2004 & 2005
Implement patch management solution on enterprise
servers.
Implement enterprise asset management solution on
enterprise servers.
Phase 1 estimated costs are $769,000
**At the same time as Phase 1, also develop and implement new
recommended policies
Phased Approach
Implementation
Phase 2 Recommendations – FY 2005 & 2006
1.
2.
3.
4.
5.
Upgrade hardware and software for all Central Department
computers to Windows XP for PC’s and OS X for Macintosh’s.
Upgrade all Central Department computers with Microsoft Office
XP or 2003 for PC’s and Microsoft Office for Mac OS X for
Mac’s.
Patch all Windows 2000 and XP systems in Central
Departments.
Visit all Central Department PC and Mac systems and ensure
Norton Antivirus Corporate Edition is installed, or install if
needed.
Migrate all Central Department computers to Active Directory.
Phase 2 estimated costs cannot be calculated until Phase 1,
asset inventory is complete
Phased Approach
Implementation
Phase 3 Recommendations – FY 2006 & 2007
The third phase of the project will concentrate on the clinical, research,
and education (CRE) departments at the College. The following are
the recommendations for phase 2 of this project.
1. Upgrade all CRE computers to Windows XP for PC’s and OS X for
Macintosh’s.
2. Upgrade all CRE computers with Microsoft Office XP or 2003 for PC’s
and Microsoft Office for Mac OS X for Mac’s.
3. Patch all Windows 2000 and XP systems in CRE.
4. Visit all CRE desktop PC and Mac systems and ensure Norton Antivirus
Corporate Edition is installed, or install if needed.
5. Migrate all CRE computers to Active Directory.
Phase 3 estimated costs cannot be calculated until Phase 1, asset inventory
is complete
Funding The Force
Summary of Costs
Implement Asset Inventory Solution
– $125,000 - $200,000
Implement Patch Management Solution
– $569,000
Upgrade Hardware & OS
– $2,988,600 - $4,072,451
Upgrade Microsoft Office
– $176,167
Install/Ensure All Computers Have NAV Corp
– $143,000
Total Minimum Estimated Costs = $4,001,767 - $5,160,618
Cost Savings With Implementation
Continued
– Reduced Downtime – The College can expect to save additional money by
implementation of standards as less time will be spent recovering systems that
experience failure. In addition, by the consolidation of hardware vendors, it will
become easier to monitor systems that may be prone to failure.
– Quicker Resolution Times – With standardization in place as well as the other
recommendations, problem resolution times will be decreased because less time
will be spent learning about new systems and software.
– Quicker Setup Times – With the standardization of computer software and
hardware, desktop images can be created which can be quickly installed for
systems that experience failure, as opposed to rebuilding a system from the
beginning. This alone could save from 20-60 minutes per event.
– Reduced Costs During Virus/Worm Outbreaks – By implementing the patch
management solution, the College can expect to have significantly reduced costs
related to, lost productivity, data loss, and connectivity to affiliates.
Costs Savings Examples
Based on example of 39% of systems needing replacement:
2457 PC setup with standardized configuration
–
–
–
–
2 hours to create image
30 minutes to transfer image to 2456 PC’s (1228 hrs total)
1230 hours total time to setup 2457 PC’s
Avg hourly rate of $20.50 = $25,215.00
2457 PC setup with non-standardized configuration
– 2 hours per PC
– 4914 hours total time to setup 2457 PC’s
– Avg hourly rate of $20.50 = $100,737.00
Represents a savings of over 75% using standardized configuration vs.
non-standardized configuration!
Cost Savings Examples
5000 PC’s need security patch
– With patch management solution installed no resources need to
be utilized as it will be automated after hours
– Without patch management solution, 5000 PC’s x 20 minutes =
1667 hours x $20.50 = $34,173.50
This example is only for 1 patch. Microsoft released a total of 76 critical patches
for Windows 2000 and 70 critical patches for Windows XP just in 2003
In the current environment many machines remain un-patched creating
significant vulnerability to Baylor network. In addition, because of extensive
outdated operating systems on the network, not all machines could be
patched even with automated solution
Summary of Costs
IT has annual budget of approx $15 million
Requesting funding of less than 1% of
total IT budget
With prevention of just 1 major virus
outbreak, costs of asset management and
patch management solutions will have
100% payback.
Summary
Baylor College of Medicine currently has a well diversified computing
environment, but this diversity also increases the College’s
vulnerability to attacks from viruses and worms.
In addition, because of the lack of a lifecycle management policy,
over 39% of PC’s and 35% of Macintosh’s will no longer have
support beyond 2003. This creates substantial vulnerability because
vendors will no longer release new security patches for many of the
systems currently deployed throughout the College. If a malicious
virus or worm is released, and these systems are unable to be
patched, the loss of data and productivity to the College would be
catastrophic.
Additionally, if the College maintains the current IT infrastructure,
then if the types of attacks occur once again, the IT program would
still not be able to prevent this from occurring in the future, it would
just be able to restore the network to its previous state. While the
one time costs may be high, they will be spread over a period of
months, AND the real potential loss to the College would be much
greater.
Questions & Answers
May The Force Be With You,
Always
Contact Information
Jenifer Jarriel
Vice President of Information Technology
and Chief Information Officer
Phone: 713-798-1103
Email: [email protected]