Transcript 2007-10-30-MSU-SecurityConsulting-Part1

Analysts International
Performing a Computer
Security Investigation
Introductions
•
•
•
•
•
•
•
•
•
•
Mark Lachniet from Analysts International
Member of the HTCIA
Not in law enforcement or a lawyer
Solutions Architect and Analysts International
Frequent presenter and trainer
Certified Information Systems Security Professional
(CISSP)
Certified Information Systems Auditor (CISA)
GIAC Certified Forensic Analyst (GCFA)
Microsoft MCSE, Novell Master CNE, Linux LPI
Certified LPIC-1, Check Point Certified CCSE,
TruSecure TICSA, etc.
Former MSU student (English major!)
2
Agenda
• Where a technical security engineer fits into
an investigation
• Frequent types of incidents
• Anonymous hacks vs. targeted
• How hacking happens
• Types of investigation
• Expanding the scope of investigation
• Hard drives and Metadata
• Documentation and procedures
• Real life examples
• Security services – detection and prevention
3
The Security Engineer
• My job is to understand the technical details of
computer security, and to know enough about
forensics and the legal system not to mess things up
• A security engineer is (usually) from the private
sector, or internal Information Security staff for
larger organizations
• There are a variety of security professionals who
work in the industry with different emphasis
–
–
–
–
Policies and procedures
Networking
Server / workstation
“White Hat” ethical hacking
4
The Security Engineer
• Recently, there has been a massive influx of people
with questionable credentials and skills
• Look for engineers that have industry-accepted
certifications from respected organizations:
– #1 The Certified Information Systems Security
Professional (CISSP) from isc2.org
– Other low-level technical certs exist (TICSA,
Security+) but are not appropriate for sensitive work
• Certifications also exist for forensic specialists, but
this is somewhat different from what I do
• Also look for specific product certifications on the
products used (Windows, Linux, etc.)
• Using an engineer with certifications may make for
an easier day in court because they have been
accredited by a recognized body
5
Where the Security Engineer Fits
•
Dedicated security consultants can help in both
prevention and response
In prevention – designing and maintaining secure
technological and organizational systems (not just
technology!)
In response – the topic at hand
•
•
–
–
–
–
For specific tools and technical expertise for a variety
of systems (servers, workstations, network devices)
To investigate an incident before deciding whether or
not to prosecute
To help weigh costs and benefits of various courses of
action – how to investigate, how to secure
To assist in prosecution by thoroughly researching
and documenting findings without the constrictions
that law enforcement would have
6
Frequent Security Incidents
• The vast majority of calls I get are in regard
to a “hacking incident”
• Almost of these incidents are on Internetconnected machines
• Most incidents are precipitated by:
– An external complaint (your mail server is sending me
a lot of spam e-mail)
– A change in the system (the hard drive is full, strange
new programs are running, tape backups are taking a
lot longer)
– The Internet is “slow” or we see strange activity
– A threat from an insider – usually a network
administrator making casual statements about how
they could “take them out” if they ever got fired
7
Frequent Security Incidents
• Many complaints focus on inappropriate use
of company technology:
– Employees looking at pornography at work
– A user is suspected of having “hacking” tools
– Suspected theft of trade secrets / proprietary info
• Another frequent event is an “employee
termination” scenario:
–
–
–
–
Employee is usually a computer administrator
Employee has extensive access to many systems
Employee is a “troublemaker”
Employer wishes help in terminating the employee, and
wants to remove their access FIRST before firing him
– Typically involves a lot of brainstorming to identify all
possible points of ingress to the computing
environment
8
An Impersonal World
• There are really two different types of computer
hacking incidents – personal and impersonal
• In my work, they are almost always impersonal
hacking attacks, not someone who intentionally
targeted the victim
• Most hackers could care less who you are, or what
sensitive information you have, they simply want to
control an Internet-connected server
• Usually this access is used in a few ways:
– To commit crimes, using you as the staging point
– To share questionable material, using your Internet
connection and server space (the “warez”server)
– To access questionable material, using you as a relay to
hide their origin (frequently porn)
– To use you as a SPAM relay to send junk e-mail to
thousands of people
9
How Hacking Happens
• Hacking is generally possible due to a vulnerability or
a mis-configuration in some server or device
• Vulnerabilities exist, and are constantly discovered, in
all types of systems by hackers and “white hats”
• Patches are released, but rarely applied due to lack of
resources, awareness, or just plain apathy
• Case in point – an Internet worm called “slammer”
took advantage of a hole that has had a software fix
for over a year!
• Hacking also occurs due to a variety of misconfiguration issues such as:
–
–
–
–
Not using a firewall to restrict access from the Internet
Running programs that are not necessary
Poor passwords, default passwords
Default configurations
10
Understanding Networks
11
Understanding Networks
• The example given previously is an example of “best
practices” in network design, and provides some defense
against Internet attacks
• Many (most?) organizations do not have an adequate
network design, and have significant risk from the Net
• Even the BEST network design can’t protect a machine
that is insecure!
• Each machine that can talk to the Internet has a unique
identifier called an “IP Address”
• IP addresses are sometimes static, and sometimes change
frequently (especially for dial-up users)
• Regardless, tracking IP addresses is frequently our only
recourse to track network attacks
• For example, if the IP address of a hacker can be tracked
to AOL, it is then possible to obtain further info from AOL
through legal action
12
Types of Investigation
• Once a call comes in requesting help in investigation,
the engineer is dispatched on-site
• The first (and perhaps most important) step is discuss
the situation with the victim before doing any work
• There are basically three ways to approach an
investigation:
– “Pull the Plug” – don’t touch the machine
– “Limited Investigation” – tread lightly
– “Extensive Investigation” – heavy footprint
• Each of these approaches have advantages and
disadvantages, depending on your goals
• The most important question to ask is how strongly
the customer feels about trying to prosecute
• The second most important question to ask is how
much $$ they have to spend
13
“Pull the Plug”
• Used when a company is VERY intent on prosecution
and does not want to risk any tampering w/ evidence
• As the title implies, the only investigation physically
performed on the target system would be to pull the
power and network cords
• This is highly disruptive and expensive, as the server
is no longer available
• There are also potential immediate results (you might
miss evidence that would lead you to investigate other
systems, for example)
• There is also no opportunity to examine the “state” of
the machine that will be lost when turned off:
– Which programs are running
– Current network connections
• Investigation of other data sources should still be
performed (for all types)
14
“Limited Investigation”
• Used when the company hasn’t decided if they want
to prosecute, and are willing to obtain more
information at the risk of having evidence modified
• Is less disruptive and less expensive – the server
doesn’t need to be taken down to do the work
• Must analyze the system with tools that leave a very
light “footprint” and will not modify much system
information:
– File (M)odify, (A)ccess, (C)reate date flags on files
– System registry settings (for Windows machines)
• The goal is to determine what happened without
modifying the system in a way that we lose evidence
that a forensic investigator could use in court
• Doing this is technically difficult
• Some information cannot be easily found without
leaving a footprint
15
“Extensive Investigation”
• Most extensive data-gathering, thus slightly more
expensive due to labor
• Still non-disruptive, the server is up and running,
although it may need to be restarted occasionally
• Includes all of the work of the previous
• After all “light footprint” methods have been tried, a
decision should be made whether to continue with
more invasive techniques
• More invasive tools can be used – these will leave a
trail, but will provide the maximum of information
• For example, it may be possible to do things such as:
–
–
–
–
–
Monitor all file accesses on the system in real-time
Monitor and record network traffic
Improve the logging data collected (usually none by default)
Read logs, files, view disk contents
Plant honeypots (password.xls, etc.)
16
Analyze Other Log Sources
• In the networked world, no machine is an island
• If systems have been appropriately designed and
implemented, which isn’t that often, there will be
useful information in a variety of places
• The investigator must expand the scope from the
“victim system” and look elsewhere
• Additional evidence can be found in many places:
–
–
–
–
Network and security devices on location
Internet Service Providers (AOL, DSL providers, etc)
Other servers on the network
Client workstations (especially if an insider is
suspected)
– Authentication systems
– The attacker’s workstation
17
Expanding the Scope of Investigation
18
Analyzing Router/Firewall logs
• Some of the best information for figuring out how an
attack occurred and subsequent activity is by
examining the logs of network devices such as routers
and firewalls
• Unfortunately, many people don’t collect this data
and store it, or even know that its possible
• Network device logs can provide a detail of what type
of information traveled between network systems:
– Determine how the system was profiled (reconnaissance)
– Determine how the system was attacked (vulnerability)
– Determine what happened after the attack – did the hacker
use your system to store files? Attack other systems?
– Determine if multiple parties were involved (hackers tend to
run in packs in different parts of the world)
19
Analyzing User Workstations
• In the event that some internal involvement is
suspected, or even just to be thorough, other servers
and workstations should be examined
• Computers that are in regular use store a lot of
interesting information such as:
–
–
–
–
Internet history (Internet Explorer, Netscape)
E-mail (settings that lead to servers, old mail)
Content (naughty pictures, confidential info)
Hacking tools and software
• Once an attack has been tracked to a particular
computer (perhaps through IP address) a forensic
analyst can pick apart the workstation to find
evidence
• Organizations with strong security policies will
enforce mandatory vacations and analyze the user’s
workstation as a part of standard practice
20
Forensic Imaging
• Analyzing a hard drive for forensic information
is a discipline all its own
• There is a lot of information that is left behind
• One must assume that any case taken will
eventually end up in the legal system
• As such, one must be very careful with potential
evidence
• In general one should use a read-only means of
copying the original hard drive (such as
hardware write blocker or a Helix boot CD) to an
image file and work ONLY from the image file
• Never work on the original system!
• Keep chain of custody documentation
21
Data and Metadata
• One thing to be aware of is that hard drives
(and other storage devices) use different types
of filesystems
• Filesystems behave in different ways, but
generally they are intended to make disk
access organized and efficient (not secure)
• To do this, they use “metadata” to keep track
of where data on a disk is – for example the
“fat file” that you may have heard about
• Metadata is like a card catalog – it is data
about data
22
Filesystem Metadata
• Because filesystems aim to be efficient, they
also leave behind remnants of data
• For example, if you delete a file, the computer
generally does not go back and wipe out the
data with 0’s or 1’s
• Instead, it simply marks the space the data
used to use as unallocated
• This means that you might be able to find the
metadata and simply “undelete” the file
• Or, you might be able to “carve” the data
from the filesystem directly (bypassing the
metadata)
23
Data Carving
• Data carving, for example the open source
“foremost” program is used to recover files
from the hard drive, even if they have been
deleted and their metadata has been removed
• It does this by identifying file headers (the
first few bytes that are associated with a file
type) and footers
• When it sees a header that matches, for
example, a JPG file, it cuts it out and saves it
to a new directory
• In this way, I can take an image of a hard
drive and carve out every identifiable JPG
and GIF image on it for review
24
Thumbnails
• Another little tidbit to know about is the Windows
thumbnails feature
• When you have your windows explorer window in
thumbnail view, it creates a small index file called
thumbs.db (slightly different for Vista)
• Inside of this is a very small thumbnail of every image
that is in the directory
• However, when you delete the image, you do NOT
delete the thumbnail inside of thumbs.db
• Hence you have a running record of all the files that
once existed, no matter how long ago they were
deleted, even if they were over-written)
• This can be a surprise to people who thought they
were being sneaky
25
E-Mail Programs
• Another area where a lot of time is spent is on e-mail
programs
• Many files such as Thunderbird and Outlook leave a
cached copy of all of your email on the local
workstation
• This can sometimes be recovered if it is deleted
• It is possible to convert these into different formats
(for example from an outlook PST file to a MBOX
format UNIX filesystem) and then do all manner of
analysis on
• The main thing that comes up is dealing with
attachments
26
E-Mail Programs
• Attachments are typically “mime encoded”
text that (when viewed with a text editor) look
like a block of garbage ASCII
• These can be converted from a textual MIME
format into a binary such as JPG file using
open source tools
• Hence it is sometimes possible to prove, for
example that alice sent bob a naughty joke by
recovering the email file from the deleted
space of the hard drive and then converting
the attachments from ASCII to binary
27
Usenet
• Another area that I have been working on is
analyzing Usenet abuse
• Usenet is a world-wide distribution system,
similar to e-mail
• It is heavily used by file traders and it is possible
to get all of the latest movies, software, etc. from
it
• Some people use it for Child pornography
• It is typically better for criminals because it is
much harder to track down
• Unfortunately for some the RIAA and MPAA
have also figured this out and started cracking
down
28
Record Keeping and
Static Procedures
• When doing this work, the security engineer should
take detailed written (physical) notes Actions taken
should be detailed along with the time it was done
• Note: Time is a big issue! The time of each device is
probably a little bit different – what is the time of the
victim system vs. local time? Other devices?
• It is good if more than one person is involved, with
the second person signing off on it
• Static procedures should be used to eliminate the risk
of error and to have a standardized methodology
• Electronic record keeping must also be secured to
minimize the risk of modification – one way is
through digital signatures (cryptographic hashes that
prove the integrity of data)
29
Create a Deliverable Document
• Once you have as much information as possible, you
need to document all of the data you have collected
and provide an analysis of the raw data
• This document should attempt to summarize:
–
–
–
–
What happened (chronological sequence of events)
How it happened (what vulnerability was used)
Problem areas (what couldn’t be done / analyzed)
Next steps (both short term recovery and long term security
steps that should be taken)
– Full appendix of collected data
• All of this information needs to be thoroughly
explained so that non-technical people can
understand the scope and impact of the incident and
make decisions
• This document can be given to law enforcement to
save time – a nice tidy package
30
Next Steps
• The decision to prosecute is not an easy one to make
because there are many implications:
– What will be the cost of prosecuting, in terms of legal
expenses, time spent, interruption to operations, etc.
– What is the likelihood of success?
– What is to be gained by prosecuting?
– What are the implications to public image? Nobody
wants to be in the newspaper, nobody wants to be
exposed as having poor security
– There is no guarantee that you will even be able to
prosecute if you want to. What if the perpetrator lives
in a developing country with now computer laws?
• Unless it was an insider job, or a specifically targeted
attack, most people consider it a “learning
experience” and hopefully secure their systems
31
Examples: The Warez Server
• Recently, I did a little experiment, and set up a
“honeypot” server on the Internet
• This server was a standard Windows 2000 server,
and was fully up to date (no vulnerabilities)
• The only change made from the default
configuration was a single (confusing) checkbox
that said to allow write access on the File Transfer
Protocol (FTP) server – an easy mistake to make
• I put the machine on the Internet to see how long it
would take for hackers to find it and abuse it
• The answer is: 3 days. Within 3 days, hackers had
found the server, and discovered that it was
possible to store files there anonymously
32
Examples: The Warez Server
• Within a week, a “tag” had been placed
(hacker lingo for claiming the server – there is
honor among thieves)
• A few days later, a huge number of “hidden”
directories were created on the server, and
software was uploaded to it.
• A few days after that, people from the
Internet were downloading the illicit content,
and I pulled the plug
• I’m still not sure what they uploaded, but
most of the time its porn
• The lesson here is that they WILL find you,
and quickly at that
33
Examples: Manufacturing
• A manufacturing company was getting
complaints from people claiming that spam was
coming from their mail server
• Their ISP shut them down due to abuse calls
• They had investigated internally and couldn’t
figure out what was happening
• Analysis of the server found that they were
directly connected to the Internet without a
firewall or other protection
• Further analysis found several problems:
– An open mail relay (allows spam)
– An open proxy server (allows anonymous web access)
– An open socks server (allows full Internet access)
34
Examples: Manufacturing
• Analysis of log files showed that people from all
over the world had been relaying connections
through their server
• Abuse included people looking at pornographic
web sites, sending spam
• A search of the Internet found that the company
server had been listed on multiple hacker sites as
being an “open” relay
• Thus, not only are the hackers who find you
going to abuse you, but they are going to share
their good fortune with others
• What are the legal liabilities of being a third
party to this type of activity?
35
Examples: Marketing
• A marketing firm calls with concerns because
the network administrator found a remotecontrol program on the server (very bad)
• The server was connected to the Internet
without a firewall
• Additional user ID’s had been created and
granted administrative access
• Client suspected internal involvement
• Logging on the server was turned off, so no
good data was collected
• Logging on the network devices was also
turned off, so there was no data there either
36
Examples: Marketing
• Examination of the server turned up some
evidence, such as the time and date that the
remote control software was installed, and
evidence that there was a hack but not much!
• However, because there was no logging, there
was no sure way to know if the attack was
internal or external
• Also because there was no logging, there was
no way to track to an offending workstation
by IP address
• The only real option was to clean up the
damage, and start recommending some
security services to stop it from happening
again
37
Examples: K12 District
• School district in Michigan with a fast
connection to the Internet
• No problems were known
• The district contracted with us to have a
managed firewall installed
• As soon as we turned it on and started
analyzing traffic, it was obvious that they
were currently being abused
• Investigation by a district employee showed
that they were unknowingly hosting child
pornography – not a good thing for a school
• Many other people have found existing
problems just by logging
38
Examples: Disciplinary Action
• Often I will be brought in to confirm or deny an
allegation of misuse of computer resources
• In a few cases I have had to prove that an employee
did something they shouldn’t have with company
material
• In one instance I recovered a bunch of dirty pictures
that were circulating in e-mail. The individual had
sued my client for wrongful termination (until they
saw my report)
• In another instance I had to prove and an employee
was working two jobs simultaneously. I was able to
find e-mails from their new employer’s HR people
about orientation procedures
• This particular individual was also writing a pseudo
pornographic humorous screenplay at the same time
39
Prevention and Response
• None of the previous incidents made it to the legal
system, it just wasn’t worth it for them
• None the less, it was an expensive, emotional and
painful experience for them
• Much of that pain could have been minimized
through prevention instead of response
• Unfortunately, computer security is a somewhat
like the wild west – its somewhat lawless,
although serious crimes can be pursued its
usually not worth it
• We use the metaphor of the neighborhood when
describing computer security – the best approach
is to make your own home hard enough to break
into that they go to your neighbor instead
40
Security Services to Know
• There are some security services that are simply
mandatory for anyone who has important data
• Failure in security due diligence can, in itself,
lead to prosecution of corporate officers
• Privacy laws, especially the Health Insurance
Portability and Accountability act of 1996
(HIPAA) mandate security best practices
• In my opinion, this will be a huge area of
emphasis in the next two decades, both for
criminal and civil action
• Security breaches are becoming commonplace in
the media, 6 million credit card numbers
compromised, etc.
• Thus, people need prevention!
41
Security Services to Know
• The following list doesn’t do justice to the field, but
here are a few things that every company needs to do:
– Design secure solutions - networks, systems and
software with security in mind. At least a firewall
– Have vulnerability assessments performed (ethical
hacking, or security needs analysis)
– Ensure that all servers that are Internet connected
or store important data are properly “hardened”
– Use some kind of auditing and logging system to
maintain an audit trail
– Maintain appropriate computer use policies
– Retain security staff to regularly evaluate log data,
perform analysis, etc.
42
Thank You!
Mark Lachniet, Sr. Security Engineer
CISSP, CISA, GCFA, MCNE, MCSE, CCSE, LPIC-1, TICSA
Analysts International
3101 Technology Blvd. Suite A
Lansing, MI 48910
phone: 517.336.1004
fax: 517.336.1004
43