Transcript Title of presentation: 32 pt Arial bold

Assessing Large Networks
George G. McBride, CISSP
RSA Conference 2004 San Francisco, CA
The Goals This Morning:
To share with you some ideas and techniques to efficiently
assess large data networks for security vulnerabilities.
These ideas may help for smaller networks, but are optimized
for the larger ones.
These are just guidelines. You will have to do what is
comfortable and best for you. There is a going to be a learning
curve and will take some time to “get it right”.
Presentation Outline
Introduction
Before You Begin
Interviewing
Scanning
Correlating the results
Getting the results back to the System Administrators
Post-Mortem
Getting ready for the next assessment
Notes:
Be advised that all tools can potentially disrupt network
operations. Run all tools at your own risk. Myself, Lucent, and
RSA are not liable!
US Export and your country’s import laws may restrict the use
of certain tools.
Double check any IP address that you scan to ensure that you
are authorized to scan the addresses and that the addresses
are accurate.
All IP addresses captured in this report were captured on the
author’s home network or are screen shots from the product’s
home page.
What is a large network?
More than one subnet?
More than the corporate office?
Too many machines that you can’t scan in one day?
More than one country?
More than one continent
All of the continents?
Does it Matter? Let’s assume that a “large” network is pretty big.
Everything is relative…
Vint Cerf’s
presentation
to RSA
Gives new
meaning to
“large”
Getting it right the first time:
Do as much as possible up front before you start to gather and
collect data.
Set up questionnaires to ensure you get the required data from
the appropriate people.
Scanning templates? What are we scanning for?
Repetitive tasks should be streamlined.
Plans, procedures, tasks, and databases should be reviewed
and optimized if necessary.
Spend the time the first time to do things right.
Where Do You Start?
The hardest part of large assessments is agreeing on a
valuable and useful scope. Will you assess by:
— Business Unit (Widget Design)
— Location or Region (Palo Alto Manufacturing)
— System Administrator (George’s systems)
— Platform or Operating System (all Linux boxes)
— IP Address or Subnet (10.5.4.x)
— Vulnerabilities (Only RPC or blank Administrator passwords)
Every location, every company, and every situation will be
significantly different from each other. Be flexible. What works
today may not work tomorrow.
Get Ready, Get Set, And Hang On!
Start all long term processes early. War Dialing and Wireless sweeps
can take a tremendous amount of time.
— Don’t underestimate the contributing factors that can increase the time to
complete some of these tasks.
I like to work directly with each location and be able to distribute the
collected data at each location.
For example, if I was reviewing the Tokyo office which may have three
different business units, I may do one large effort and then divide the
assessment results into three completely separate reports.
However, if all business units are managed by the same officers or the
same IT group supports the businesses, one report may be better!
Things to Consider During Scope Layout:
Your assessment may involve the same personnel multiple
times if working more than one business unit. Likewise, you
may not meet everybody.
— Ensure you’ve got all questions ready the first time.
With large networks and small IT shops, it may be difficult to
obtain specific IP Addresses.
— Scan subnets to find active machines and services and then work
with the customer to determine which machines should be
scanned.
Nobody wants to take responsibility for a shared machine.
— But you can still scan it with proper notification if your policy
permits it!
The Interview
What about questionnaires?
— Electronic vs Paper-based
— Preliminary vs Complete
— Ensure that you are meeting with the proper people
I’d still recommend meeting with the personnel, not only for
some face time, but to review responses and answer any new
questions that come up from the questionnaire.
Make sure that you take the time to acknowledge and thank
those that provide information to you.
— Recognition and thanks go a long way!
It’s Good To Meet You!
Meeting with the system administrators can give you MORE
information than you want to know.
You know you’re asking the proper questions when the
responses set your expectations of the scan results.
— I.E., you’ll know the scan results before you start scanning.
I like to have a living list of questions already prepared for each
group (DBA, help desk, system administrator, etc.).
Make sure you schedule some time to collate and review the
information.
Due to political issues, you may need to interview some nonessential persons!
Interviewing (Cont’d)
If you send off the interview questions to be completed prior to
arrival, MAKE SURE you review them prior to meeting with the
people.
— Don’t ask questions for which you’ve already got answers.
— Questionnaires make follow-ups and more in depth questions
significantly easier to follow
— Questionnaires help identify vulnerabilities where you might not
normally look.
Let your questions be your guide, not a verbatim reading.
I like to schedule interviews on a one to one basis, for about
one hour.
Understanding the Network
If the client doesn’t have
network diagrams, generate
your own.
— It’s an awesome deliverable
A picture is worth a thousand
words
Don’t get too caught up in the
generation of the network
diagrams
A great time-saver to
understand the network
topology
Network Mapping: The Big Picture
The previous page illustrated a
Cheops-NG map, available at:
http://cheops-ng.sourceforge.net/ is
a great and free tool.
LuMeta, at http://www.lumeta.com
offers some great mapping
services as well as helping you
find your perimeter.
HP’s OpenView and other
network management tools may
be useful to understand the
architecture and topology.
Scanning
Before you start scanning, plan on how you will mine the data
that you will collect with the scanners.
Getting the data out of the scanner continues to be the
“hardest” part, but is a one-time effort.
— Database Format
— Spreadsheet
Research and understand the reporting or database structure
of your scanning tools to understand how the data will be
collected.
Sometimes a simple “Microsoft Excel import” works wonders.
Only Scan For Data You Will Use
It sounds obvious, but don’t scan for data that will be discarded.
Watch DOS attacks.
Nessus, Newt, ISS,
eEye, and all scanners
allow you to select and
de-select vulnerability
checks as required.
Review every option.
Only Scan For Data You Will Use
Just for comparison, a
Nessus screen.
Note the DOS option!
All scanners have the
potential to have
undocumented DOS
tests!
Data-Output
ISS’ Internet Scanner has
several different output options,
some of which may be easy to
import into a database.
A-Ha! The data is stored locally
on the operators PC in a
Microsoft Data Engine
database format.
Nessus Output
Adobe PDF, Microsoft .DOC,
and .TXT file format outputs.
Nessus Output: .NBE Format
Nessus .NBE file
format makes it easy
to convert into a
database format.
Got your baseline?
If you are only interested in
checking for changes since the
last scan, try the delta or
“differential” scan.
This will highlight changes
since the last scan including
new systems.
Is there ever a time you aren’t
really concerned with the “old
stuff”?
— Only if the old stuff is noise
Extra Sensitive Systems?
Are you or your client concerned with generating traffic which
may “negatively impact service” (take the network down)?
Consider exploring the use of Tenable Security’s Nevo tool
which is a passive scanner.
— Generates no traffic
— Fills the gap between active scans since it immediately detects
any new systems once they generate traffic that pass through one
of its sensors
By definition, it generates no traffic. If a vulnerable system
does not generate enough traffic, all vulnerabilities may not be
identified.
Nevo: Passive Vulnerability Detection
Screen shot of Nevo detecting
traffic as it runs.
You can see DHCP server
identified, WWW server (And
version) and SSH running.
Sometimes it gets specific
service versions, sometimes it
doesn’t.
Leaves some more ambiguity
than a passive scan which
could conduct additional
probes.
Nevo: Passive Vulnerability Detection
Nevo Output = Nessus Input
Based on identified systems
and services as well as their
version numbers; vulnerabilities
can be identified.
Again, if the system wasn’t
accessed or didn’t generate any
traffic, Nevo won’t find it.
What Systems Should Be Scanned?
What’s on a typical network:
— User’s desktops and workstations
— Servers such as file, print, WWW, database, and major
applications
— Network equipment such as routers, firewalls, wireless access
points, network management equipment
Workstation Risks (Windows, UNIX, Linux, Mac, etc.)
— A lot of local data (mail, personal files, local working documents)
— Possibly exploit trust to access other machines
— Often managed by users who can change anything on the system
— A LOT of workstations to scan and A LOT of risks
What Systems Should Be Scanned?
Servers:
— Lots of user’s data
— Sensitive data including source code libraries, print queues, restricted
web documents
— Malicious users could change, add, delete, and data on the server
— In general, managed by IT Organization, should follow some security
standards
— Fewer servers than workstations, but generally more sensitive
Network Nodes:
— Continue to find default passwords installed
— Can be used to sniff traffic
— Can be used to disable network segments
Divide and Conquer
Avoid getting lost in the sea of numbers. A scan of a hundred
machines properly analyzed is probably better than a scan of
several thousand machines.
Reduce the number of machines to scan:
— Multi-phased approach where only servers or critical / sensitive
machines
— Perhaps a scan of a cross-section of systems by:
• Operating system
• Administrator
• Purpose – Function
• Configuration
Saturating Your Network And Hosts
Won’t happen with a passive assessment
Hosts that are close to network and processor overload can be
pushed to the “edge” and impact performance.
Most scanners err on the side of caution with the number of
parallel system scans and service scans. But that is user
changeable! 
The only time that I consistently see any type of true problem
introduced during a network scan is when the scan is across
some low-speed WAN connection.
— Consider distributed scanning!
Network Overload
Watch Your Network Boundaries
It’s not just your network anymore!
— You’ve got connections to customer networks
— And connections to vendor networks
— And connection to business partners
— And joint ventures
— And dial-up users and remote administration / maintenance
If you are assessing ISP provide equipment, ensure that you
have the required approval and notify their administration
Watch all local laws, procedures, regulations, etc. What you
can do where you start your scan may not be the same laws
where you scan terminates.
Speaking of Bad Things
Check and then double-check that you DO NOT HAVE denial of
service checks turned on.
If you are doing a large scan, it may be wise to do a subnet or
two first. If the machines are vulnerable to some particular test,
it’s better to find it out now.
And always, make sure that you notify the system and network
administrators when and what you are scanning.
— Give everybody your pager or mobile number
Unfortunately, if you do enough scans, “stuff will happen”.
Looking for the needle in the haystack
When you can’t scan every machine for every
vulnerability,
consider scanning for the top
ten threats.
Check out the SANS Top 20 resources at
(http://www.sans.org/top20/)
This document is updated somewhat regularly and lists the top
10 UNIX/Linux vulnerabilities and the top 10 Microsoft Windows
vulnerabilities
Another SANS Site,
http://www.incidents.org,
has a real-time “Top
10” list
Top 10 Methodologies…Why Scan?
Several Lucent and Bell Labs researchers performed a study in late
1999 to identify and understand the trends of network and host
security vulnerabilities on the Lucent network.
That study is available at:
http://www.lucent.com/minds/techjournal/common/arc_issues.html.
The study shows that the “top nine vulnerabilities account for 89
percent of all high risk vulnerabilities”.
The study also indicates that the high and medium risk vulnerabilities
account for at 80 percent of all of the vulnerabilities.
When you don’t have time to scan or perhaps prior to your next
“enterprise” scan, consider pushing out the fixes for the top ten or
twenty vulnerabilities to the systems on your network.
Tackling the False Positive Problems
Given:
— 100 Hosts
— 10 Vulnerabilities Per Host
— 95% Confidence Level
1000 Vulnerabilities means that fifty may be suspect. But which
fifty and how do you find them?
What about the vulnerabilities that you didn’t detect?
Check all of the results? Don’t check at all?
Random spot check? Run the tool again? Run a different tool?
Verification of Data
Each false positive or undetected vulnerability counts against you in
the “credibility” category!
I recommend a sanity check approach which requires a manual
review of vulnerability findings with interview responses and
configuration information.
Some vulnerabilities are prone to false positives
— These should all be checked prior to report distribution
Review the results to make sure that the vulnerabilities match the
machine
— You can’t have a BIND vulnerability on an HP Printer.
Watch items in the reports that say “may be vulnerable if a file is
present”. It should be up to you to clarify those findings.
Data Presentation
In a report where you’ve scanned a large number of systems
transcending multiple locations, business units, or support staff,
you should consider multiple reports, specific to each recipient.
Summarize the findings into higher levels to present trends and
summaries.
In general, only the system administrators need to get a
detailed report of vulnerabilities by IP address (with the required
fix information).
Include the good things that you found.
The report should be distributed in a draft format immediately
after completion.
Getting Ready for the Next One!
Consider a post-mortem:
— At least after your first few assessments with all team members
that were involved.
— Even if things went “well”, I suspect that there was room for
improvement or positive criticism.
— If you are comfortable, talk to the key contacts at the customer
site and solicit their feedback.
Review the processes and steps that took the most time or
those that are the most labor intensive. Can anything be done
with these?
Continually keep your tools up to date and complete.
Before the next assessment…
Subscribe to (and read!) relevant mailing lists on Security
Focus!
NT Bugtraq is an excellent resource
INCIDENTS.ORG and SANS.ORG are equally valuable
FRESHMEAT.NET, ISECOM.ORG, INSECURE.ORG,
NESSUS.ORG, and even SNORT.ORG and their mailing lists
are excellent resources!
— Most have archives to search past messages and lists
Establish a stand-alone network to install and test new tools.
As always, your customers network is not a test network.
Questions?
Contact me at [email protected] with any questions that
you may have or any thoughts or comments on this talk.
Lucent Technologies
Bell Labs Innovations
George McBride
Senior Manager
IT Risk Management
Lucent Technologies Inc.
Room 2N-611G
101 Crawfords Corner Road
Holmdel, NJ 07733
Phone: +1.732.949.3408
E-mail: [email protected]