Transcript Closing the Gap: Analyzing the Limitations of Web

Closing the Gap: Analyzing the Limitations of
Web Application Vulnerability Scanners
David Shelly
Randy Marchany
Joseph Tront
Virginia Polytechnic Institute
and State University
Purpose
• To analyze the limitations of web application
scanners by using both a secure and
insecure version of a custom-built web
application
• Identify scanner weaknesses
• Improve scanner detection techniques
• Reduce false-negatives and false-positives
2
Problem
• > 60% of attacks seen on Internet aimed at
web applications [SANS 2009]
• Web application vulnerability scanners are not
capable of detecting all of the vulnerabilities
and attack vectors that exist
• Numerous false-negatives and falsepositives
• False sense of web application security
• Web application flaws remain unpatched
Relevant
Vulnerabilities
• SQL Injection
• Cross-Site Scripting (XSS)
• Session Management Flaws
Web Application
Scanners
Related Work
• Evaluation Applications
• WebMaven’s BuggyBank
• Foundstone’s Hacme Bank, Hacme
Shipping, Hacme Travel
• Evaluation Techniques
• Scan publicly available web
applications
• Benchmark system
Related Work
• Evaluation Applications
• WebMaven’s BuggyBank
• Foundstone’s Hacme Bank, Hacme
Shipping, Hacme Travel
• Evaluation Techniques
• Scan publicly available web
applications
• Benchmark system
Test Bed Design
•
•
Client-Side Features
Regular users
•
•
•
•
Submit, assess, purchase, and categorize products
Password reminder
Register new user
Edit account settings
Administrator users
•
•
•
Add other administrators
Approve/deny uploaded templates and item categorizations
View validated templates
•
Server-Side Features
Database
•
•
•
•
•
Users table (10 regular, 1 administrator)
Categories table (12 static, 1 custom)
Categorize table (15 uncategorized, 5 categorized)
Items table (10 available)
Templates
•
•
5 validated item templates
5 awaiting approval
Vulnerabilities
Implemented
•
•
SQL Injection
•
•
Form input (12)
Cookie Variable (8)
Session Management
•
•
Predictable session ID (1)
Unprotected variables (9)
•
Cross-Site Scripting
•
•
•
Reflected (10)
Stored (6)
DOM-based (1)
SQL Injection
Example
XSS Injection
Example
Session Management
Example
Session Management
Example
Session Management
Example
Methodology
Most popular web server technologies:
• CentOS 4.4
• Apache 2.0
• MySQL 4.1
• PHP 4.3.9
Methodology
• Controlled benchmark environment:
•
•
•
Controlled variables
•
Web server technologies
Independent variables
•
Number of deliberate vulnerabilities
Dependent variables
•
Observed false positives and false negatives
Methodology
• Black-box and White-box Analysis:
•
•
Black-box Analysis
•
•
Perspective of client-side user
Manipulates user supplied input
White-box Analysis
•
•
Source code is available
Verify actual number of vulnerabilities
Testing Approach
•Phases:
•1. Initialization
•2. Execution
3. Classification
4. Analysis
Initialization
•1.
Restore the MySQL database to its
original state
•2.
Delete all client side and server side
cookies
•3.
Restore the entire web server
directory with a clean backup
•4.
Restart the web server
•1.
Execution
Configure the web application
vulnerability scanner (primarily default
settings)
•2.
•3.
Start Wireshark packet capturing
Execute the web application
vulnerability scan
•4.
Stop Wireshark packet capturing and
save the trace
•5.
Save the contents of the database,
cookie files, and scanner results
Classification
•False Positives:
•1. The scan result is due to an application robustness
problem (error page, format exception, etc.) and not a
vulnerability
•2.
Normal operation of the web application results in
the same error/problem
•3.
Source code analysis determines scan result is
incorrect
•4.
The results duplicate a vulnerability that has
already been accounted for
•Note: Repeated input forms that are displayed recursively on a web page
are not considered duplicate vulnerabilities if they are detected multiple
times by a web application scanner (e.g. user comment boxes)
Analysis
• HTTP requests and responses
• Client and server cookies
• Database copies
• Scanner result reports
SQL Injection Results –
Form Inputs
SQL Injection Results –
Cookie Variables
XSS Injection Results
- Reflected
XSS Injection Results
- Stored
XSS Injection Results
– DOM-based
Session Management Results
– Predictable SID
Session Management Results
– Insecure Cookie Variables
False Positives –
SQL Injection
False Positives –
XSS Injection
•
Analysis – SQL
Injection
False Negatives
•
•
•
•
Unable to recognize reflected SQL error in response page
Required fields not completed (ex: register and log-in forms need
more than one input)
Cookie variables not tested for injection
Improvements
•
•
•
Flag any error or warning response as suspected vulnerability
(information disclosure)
All possible combinations of form inputs should be tested
Add cookie variables to list of parameters to be checked
•
Analysis – SQL
Injection
False Positives
•
Duplicate entries reported
•
•
•
•
Tested both GET and POST methods
Multiple exploit strings for same vulnerability instance
Blind SQL injection returned the same HTML page
Improvements
•
•
•
Only report the implemented HTTP method (unless both methods
are supported)
Verify vulnerability using multiple exploit scenarios, but only report
SQL injection vulnerability once
Blind SQL injection tests should use time delay statements
•
Analysis – XSS
Injection
False Negatives
•
•
•
•
Required fields not completed
Did not test all possible user supplied input (DOM variable,
cookie variables, and uploadable files)
Stored XSS detected through reflected techniques (relies on
SQL error pages)
Improvements
•
•
•
Attempt every combination of form inputs
Test all parameters (even search embedded scripts)
Use unique injection strings and then crawl web application
multiple times searching for string
•
Analysis – XSS
Injection
False Positives
•
Duplicate entries reported
•
•
Same exploit string with different set of parameters (causes
request to appear different)
Improvements
•
Same as SQL injection
•
Verify vulnerability using multiple exploit scenarios, but only
report one XSS vulnerability per input form
•
Analysis – Session
Management
False Negatives
•
Did not attempt to manipulate cookie to perform session
hijacking (only tested for information disclosure)
•
•
•
Most determined cookie not set as HttpOnly
Only tested for session management vulnerabilities using
information from initial connection
Improvements
•
•
Tamper with all unprotected session variables throughout
the application (check if response page different)
Analyze SIDs for strong algorithm
Analysis – Other Findings
•
Possible explanations
•
Overload of requests
•
•
•
Scanners not handling all of server responses
Failures in spidering techniques
•
•
•
Server not producing proper response pages
Scanners missing parameters and links while crawling
Did not test initial log-in page
Improvements
•
•
Monitor current connection speed and throttle number of
simultaneous connections
Test all pages and variables within scope of web application
Conclusions
•
Scanners detect simple forms of reflected XSS and SQL
injection relatively well
•
•
Most false-positives observed from insecure version of web
application
•
•
•
Work still needs to be done to detect non-traditional
instances of these vulnerabilities
Expected secure version to produce false-positives
Did not test enough of the input parameters
Better to use multiple web application scanners
•
Combined results better than any one individually
Future Work
• Expand to include analysis of other web
application technologies and server
configurations
• Web 2.0 technologies (Ajax, Flash,
Ruby, Python)
• Develop black-box web application
scanner
• Using proposed techniques
Questions
• security.vt.edu