Transcript The Need for Intrusion Detection

Network Intrusion Detection
System
Network Intrusion Detection
Basics
 Network intrusion detection systems are designed to sniff network traffic
and analyze it to identify threats in the forms of reconnaissance activities
and attacks.
 By detecting malicious activity, network intrusion detection enables you
to identify and react to threats against your environment, as well as
threats that your hosts might be directing at hosts on other networks.
 There is also a related IDS technology known as host-based intrusion
detection.
 Host-based IDS software focuses on detecting attacks against a particular
host, such as a workstation or server, and is run from the host itself.
Network Intrusion Detection
Basics
 There are several types of host-based IDS software products, including log
analyzers and file integrity checkers.
 Log analyzers monitor operating system and application logs, looking for
entries that might be related to attacks or security violations.
 File integrity checkers alert you if particular files are altered, which might
indicate a successful attack.
 Without intrusion detection, you may be unaware of many attacks that
occur.
The Need for Intrusion Detection
 Most ominously, you may never know about an attack that doesn't
damage your host, but simply extracts information, such as a password
file.
 Many attacks involve multiple steps or phases. For example, an attacker
might start by launching a scan that sends a DNS query containing a
version.bind request to each IP address in a range.
 Some DNS servers respond to such a query with their BIND version
number.
The Need for Intrusion Detection
 The goal of this scan is to identify which hosts are DNS servers and what
versions of BIND they are using.
 Based on the results of the first scan, the attacker then sends a specially
crafted DNS query to some of the DNS servers to exploit a buffer overflow
vulnerability in a particular version of BIND.
 This second set of queries could occur hours or days after the first setor it
could occur within seconds. It depends on the tools being used and the
attacker's methodology.
The Need for Intrusion Detection
 If any of the buffer overflow exploits are successful, the attacker might be
able to perform unauthorized actions on one or more of the servers,
potentially gaining administrator-level privileges.
 If you are not using intrusion detection, how will you know when a
successful attack has occurred? Some attacks are immediately visible, such
as a website defacement; others are not.
 A properly configured, robust IDS can play more than one role in
identifying typical attacks. An IDS can detect reconnaissance activity that
may indicate future targets of particular interest.
The Need for Intrusion Detection
 It also generates alerts for the subsequent attempts to breach host
security.
 Alerts are usually generated through one of two methods. The first
method, anomaly detection, relies on statistical analysis to identify traffic
that falls outside the range normally seen in this environment, or it relies
on protocol analysis to identify traffic that violates protocol standards or
typical behavior.
 The second method, signature detection, identifies known attack
signatures observed in traffic.
Anomaly Detection
• IDS detect anomalies in network traffic by observing network activity over
a period of time, then detecting significant deviations from the baseline.
• For example, a product could monitor a network for two weeks to identify
which network services are provided by each host, which hosts use each
service, and what volume of activity occurs during different times of the
day and days of the week.
• A month later, if the IDS sensor sees a high volume of traffic involving a
previously unused service on a host, this could indicate a distributed
denial of service (DDoS) attack against the host or a compromised host
providing a new service.
Anomaly Detection
 This class of product is sometimes known as a DDoS attack mitigation
system, because it not only detects the DDoS attacks, but also acts to stop
them through intrusion prevention techniques.
 Examples of DDoS attack mitigation systems include Captus IPS, Mazu
Enforcer, and Top Layer Attack Mitigator.
 One obvious drawback of this type of anomaly detection is that the
baseline needs to be updated constantly to reflect authorized changes to
the environment, such as the deployment of a new server or the addition
of another service to an existing server
Anomaly Detection
 Some products permit the baseline to be updated manually, at least in
terms of identifying which hosts are authorized to provide and use certain
services.
 If the baseline can be kept current, or if the environment is quite static
and the baseline changes rarely, anomaly detection can be extremely
effective at identifying certain types of attacks and reconnaissance
activities, such as port and host scans, network-based denial of service
attacks, and malicious code (particularly worms).
 Unfortunately, this type of anomaly detection cannot identify most other
types of attacks, so it is best used to complement other IDS technologies.
Anomaly Detection
 Many signature-based IDS products perform some type of protocol
anomaly detection. This means that the IDS compares traffic to the
expected characteristics for widely used protocols, such as HTTP, DNS, and
SMTP.
 When a serious discrepancy is discovered for example, a header field that
contains hundreds of binary values instead of the expected small number
of text characters the IDS generates an alert that an anomaly has been
discovered.
 This can be very effective at identifying certain previously unknown
instances of attacks that a purely signature-based IDS could not identify.
Anomaly Detection
 One significant limitation of anomaly-based IDS is that generally it cannot
determine the intent of an attack just that something anomalous is
occurring.
 Analysts need to study the data captured by the IDS to determine what
has happened, validate the alerts, and react appropriately. However,
because network and protocol anomaly detection based methods can
detect attacks that signature-based IDS cannot, a robust IDS solution for
an enterprise should incorporate both anomaly-based and signaturebased methods if resources permit.
Signature Detection
 A network IDS signature is a pattern you are looking for in traffic.
 When a signature for an attack matches observed traffic, an alert is
generated, or the event is otherwise recorded.
 Some older operating systems could not properly handle such packets,
which violate standards, so attackers would send crafted packets with the
same source and destination addresses.
 Many signatures are protocol or application specific; for example, many
signatures pertain only to DNS traffic
Signature Detection
 Because DNS zone transfers traditionally use TCP port 53, a signature
pertaining to a zone transfer vulnerability includes the TCP protocol and
also needs to look for destination port 53.
 Each signature specifies other characteristics that help to identify the
vulnerability that the attacker is trying to exploit.
 Some IDSs focus on long sequences of code from published exploits,
whereas other IDSs actually perform full protocol analysis, examining and
validating the header and payload values of each packet.
 Although full analysis is more resource intensive, it tends to produce a
much more robust signature solution.
Signature Detection
 Most intrusion detection vendors release new sets of signatures regularly.
If a major new vulnerability is discovered or an exploit is widely seen,
intrusion detection vendors quickly research the exploit or vulnerability,
create new signatures for it, and release the signatures to their users as
soon as possible.
 This process is similar to the process that antivirus software vendors
follow when addressing threats from new viruses, worms, Trojans, and
other types of malicious code.
 When a major new threat emerges suddenly, such as the Slammer and
Netsky worm, IDS vendors typically have a signature available for
How Signatures Work
 Let's look at an example of a signature for the Nimda worm. Because the
worm uses a common text-based protocol (HTTP) and uses a relatively
simple exploitation technique.
 When the Nimda worm tries to infect a web server, it sends a set of HTTP
requests, including this one:
GET /scripts/..%c0%af../winnt/system32/cmd.exe?/c+dir
 The purpose of this particular request is to exploit a Unicode-related
vulnerability in certain unpatched Microsoft IIS servers to gain
unauthorized privileges.
How Signatures Work
 %c0%af is the Unicode equivalent of a slash, so this command is actually
trying to traverse the root directory to exploit the vulnerability.
 You want your network intrusion detection sensors to notify you when
they see this traffic. Many different possible signatures exist.
 A simple text-matching signature would look for /scripts/..%c0%af../ in a
URL (more precisely, in the payload of a TCP packet that a client sends to a
server's port 80).
 Because the Nimda worm issues several GET requests with slightly
different values, you would need a separate signature for each one of
them if you wanted to identify each attack attempt.
How Signatures Work
 Some IDS sensors have a much more robust way of identifying these types
of requests as attacks.
 Rather than doing simple text matching, they actually decode the request,
substituting a slash for the %c0%af sequence, and then analyze the
validity of the decoded URL.
 As a result of the analysis, the IDS sensor determines that the attacker is
trying to go past the root directory, and it generates an alert.
 Although this method is more resource intensive, it provides a much
better detection capability than the simple text-matching method because
it actually examines traffic for the attack technique, not for specific known
instances of that technique being used.
False Positives and False Negatives
 Signature development is always a balancing act. A specific signature
might be extremely accurate in identifying a particular attack, yet it might
take many resources to do so.
 If an attacker slightly modifies the attack, the signature might not be able
to identify it at all.
 On the other hand, a general signature might be much faster and require
far fewer resources, and it might be better at finding new attacks and
variants on existing ones.
False Positives and False Negatives
 The downside of a general signature is that it also might cause many false
positives when a sensor classifies benign activity as an attack.
 Another factor is the original release date of the signature; over time, false
positives are identified by the IDS vendor and corrected, leading to the
release of higher-quality signatures. Brand-new signatures often generate
high numbers of false positives.
 Every intrusion detection system generates false positives. When you first
look at an IDS console and see dozens or hundreds of alerts, you might
think that your systems are under massive attack. But those alerts are
much more likely to be false positives.
False Positives and False Negatives
 By selecting a solid IDS product and tuning your sensors, you can reduce
false positives, but you can't completely eliminate them.
 No matter how precisely a particular signature is written, there's still a
chance that some benign traffic will accidentally match that signature.
 False positives are a headache for an intrusion analyst because you have
to spend time and resources determining that they are, in fact, false.
False Positives and False Negatives
 A false negative occurs when a signature fails to generate an alert when its
associated attack occurs.
 People tend to focus on false positives much more than false negatives,
but each false negative is a legitimate attack or incident that IDS failed to
notice.
 False negatives get less attention because you usually have no way of
knowing that they have occurred.
 It's important to understand what causes false positives and negatives so
that you can choose an IDS that minimizes them.
Developing Signatures That Minimize False
Positives and Negatives
 To better understand false positives and negatives, let's look at a simple
example.
 Suppose that in your intrusion detection system, you write a signature
that looks for cmd.exe anywhere in a URL. (cmd.exe is often used in root
traversal exploits against IIS servers, such as the Nimda worm.)
 This is a general signature that matches many different attack attempts.
 Unfortunately, it also matches cmd.exe-analysis.html, a web page that
contains an analysis of cmd.exe attacks.
Developing Signatures That Minimize False
Positives and Negatives
 Because of the false positives, you decide to rewrite your signature to use
some contextual information as well. Now your signature is a URL that
contains
/winnt/system32/cmd.exe
 Because this is a more specific signature, it triggers fewer false positives
than the more general signature.
 Unfortunately, reducing false positives often comes at the expense of
increasing false negatives.
Developing Signatures That Minimize False
Positives and Negatives
 If
an
attacker
uses
a
URL
that
includes
/winnt/system32/../system32/cmd.exe, the more specific signature misses
it, causing a false negative to occur.
 However, the more general first signature would have sent an alert. And
the more robust method described earlier, which decodes the entire URL
and evaluates it for root traversals, would have sent an alert for both
versions.
 More complex signatures are usually better than simple text-matching
signatures, which are unable to identify most variations on attacks. In fact,
some attackers purposely craft their attacks to avoid detection by simple
signatures; this is known as IDS evasion.
Detecting IDS Evasion Techniques
 As if developing an IDS signature that minimizes false positives and false
negatives isn't difficult enough, you must also consider the effects of IDS
evasion techniques on signature development.
 Attackers have many methods of altering their code to avoid IDS
detection.
 For example technique used in URLs is replacing a character with its hex or
Unicode equivalent. Therefore, cmd.exe could become cmd.%65xe
because %65 is the hex representation of e.
Detecting IDS Evasion Techniques
 Only IDS products that perform hex decoding before performing signature
comparisons would determine that this string matches cmd.exe.
 Some evasion techniques actually relate to the IDS sensor's core
architecture, not specific signatures.
 One such technique is to fragment a packet so that it has a small initial
fragment; the attacker hopes that the IDS examines only the first
fragment, which looks benign, and ignores the remaining fragments,
which contain malicious content.
 An IDS that does full-packet reassembly is not fooled by this evasion
method.
Avoiding Unwanted Alerts
 Besides false positives, you might also have alerts that address legitimate
problems that you simply don't care about.
 Unwanted alerts seem to occur in just about every environment.
 For example, if your environment is an ISP or a wireless company, you are
providing network services to customers. You probably have a lot of traffic
passing on your networks between your customers and other outside
hosts.
 If you are monitoring these networks with IDS sensors and you tune them
to identify all possible scans and attacks, you are likely to be overwhelmed
with a huge number of alerts that you honestly don't care about.
Alerting, Logging, and Reporting
 When an intrusion detection system sees traffic that matches one of its
signatures, it logs the pertinent aspects of the traffic or generates an alert,
depending on the severity of the activity and the configuration of the IDS
sensor.
 Alerts can be delivered to intrusion analysts in a variety of ways, including
messages on the analyst console, emails, and SNMP traps, depending on
the product being used.
Alerting, Logging, and Reporting
 Reporting is another key element of intrusion detection.
 If you have only one sensor, reporting should be simple.
 If you have dozens of sensors, you probably don't want separate reports
from each of them. Instead, you want all data to be collected in one place
because it's more convenient to view and easier to back up. It's also far
easier to correlate events that all sensors see.
 Reporting formats vary widely among IDS products; most also permit
sensor alerts and logs to be exported to other databases for further
queries and reporting.
Intrusion Detection Software
 Many different network IDSs are available, each with its own distinct
features. Some of the best-known and most popular products include
Cisco Secure IDS, Enterasys Dragon Network Sensor, ISS RealSecure
Network Sensor, NFR Sentivist IDS, Snort, and Sourcefire Intrusion
Management System.
 They all provide alerting, logging, and reporting capabilities. Signature sets
are available through a variety of means, depending on the product.
 Users of some products have written their own signatures and made them
publicly available. Other products must be purchased to get signatures;
some vendors hide the signature details, whereas others allow users to
view and even modify their signatures.
Intrusion Detection Software
 Network IDS sensors have other helpful features. For example, some
systems can perform network monitoring and traffic analysis, collecting
statistics on connections, protocols in use, and the like.
 This capability can be invaluable in identifying policy violations, such as
unauthorized services in use, and traffic oddities in general.
 Some IDS products provide tiered solutions, which typically involve
deploying multiple IDS sensors on various network segments. Each sensor
transmits its IDS data to a centralized server that stores all the data. The
individual sensors or the centralized box, which is able to correlate events
from all the sensors' data, can make alert decisions.
Intrusion Detection Software
 Many IDS solutions also have separate analyst consoles that connect to
the centralized system or the individual sensors; these enable analysts to
view recorded data and alerts, as well as reconfigure the sensors and
update signatures.
 Some software vendors, such as ArcSight, e-Security, GuardedNet,
Intellitactics, and netForensics, offer console products that can process
and correlate data from various brands of IDS sensors, firewalls, and other
hosts.
Intrusion-Related Services
 Besides intrusion detection products, some intrusion detection related
services might be able to assist you with the analysis of your intrusion
detection data.
 Some organizations choose to outsource the analysis of their IDS sensor
data to specialized commercial monitoring services.
 Other organizations elect to do their own analysis, but also submit their
data to free distributed IDS services for additional help. Following are
main services:
1.
Distributed IDS Services
2.
Outsourced Intrusion Detection System Monitoring
Distributed IDS Services
 As IDS products have become more popular, the use of free distributed IDS
services has also grown significantly.
 The basic concept is that administrators from organizations all over the
world submit logs to a distributed IDS service from their own IDS sensors,
firewalls, and other devices.
 These services analyze the data from all the sites and perform
correlations to identify likely attacks.
 Two of the best-known distributed IDS services are Symantec's DeepSight
Analyzer, located at http://analyzer.symantec.com,
 DShield, located at http://www.dshield.org.
Distributed IDS Services
 Using a distributed IDS service has a few benefits. Because the service receives
logs from many environments, it has a wealth of data to use for its intrusion
analysis.
 A distributed IDS service finds patterns among all the attacks that individual sites
cannot see.
 Another great feature of some distributed IDS services is that if they conclude that
a host at a particular IP address attacked a substantial number of sites, they will
email the technical contact for that domain so that the activity can be investigated.
 Distributed IDS services can reduce some of the burden placed on intrusion
analysts by analyzing logs, correlating events, and following up on highly
suspicious activity essentially providing another tool that can help analysts.
Outsourced Intrusion Detection
System Monitoring
 Many companies, such as Counterpane, ISS, and Symantec, offer managed
security services that include IDS monitoring capabilities.
 This generally entails 24-hour remote monitoring of your IDS sensors,
analysis of IDS data, and rapid notification of your staff in case of serious
attack or IDS failure.
 One advantage of using these services is staffing related; they can provide
trained intrusion analysts to monitor your sensors at all times.
 Another advantage is that because these services collect data from so
many networks, they can correlate attacks among them.
Outsourced Intrusion Detection
System Monitoring
 The disadvantage is their lack of knowledge about your environment and
their inability to correlate events unless they can access firewall and host
log files in near real time.
Outsourced Intrusion Detection
System Monitoring
 At best, outsourced IDS monitoring can identify serious attacks and help
your organization respond to them quickly and appropriately.
 If your organization doesn't have the time or expertise to analyze its own
IDS sensor data properly and promptly, outsourcing that work is important
for maintaining a strong perimeter defense.
 At worst, such services add little or nothing to the existing IDS solution,
while requiring substantial financial commitments. Because outsourced
IDS monitoring is very resource-intensive, it can easily cost millions of
dollars a year for larger organizations.