Transcript Web/Google Data Mining - Colorado Higher Ed Computing

Web/Google Data Mining
Testing Your Web Security and
Privacy
Jim Dillon, IT Audit Manager
University of Colorado
[email protected]
Teoma
Yahoo
AltaVista
Google – The Page
Sample
Search
Terms: SSN:
Filetype: XLS
DOMAIN: UMICH
Terms: SSN:
Filetype: XLS
DOMAIN: UMICH
http://www.google.com
Results
LSA Voucher
Advanced Searches
• The key to a successful search is
–
–
–
–
Art
Knowing your environment
Understanding Web applications
Utilizing someone else’s smarts to do the above
• Example: Social Security Number Searches
– SSN: in Excel Files
– Search for “521” thru “524” in Excel or .htm files
– Combo of words like “registration” and “SID”
Google Hacking
• Is It Hacking?
– Never have to enter the domain
– Can just look into cache files (sometimes only
the cache image is left)
– Information that has not been protected by the
information owner
http://johnny.ihackstuff.com/
Johnny
http://johnny.ihackstuff.com/
GHDB
The Tools
• SiteDigger, SiteDigger2 (Foundstone)
– http://www.foundstone.com/ (Resources/Free Tools)
• Athena, Athena 2
– http://www.snakeoillabs.com/
• Wikto (Sensepost)
– http://www.sensepost.com/research/wikto/WiktoDoc151.htm
The Database
- <signature>
<signatureReferenceNumber>23</signatureReferenceNumber>
<categoryref>T2</categoryref>
<category>TECHNOLOGY PROFILE</category>
<querytype>DON</querytype>
<querystring>intitle:index.of master.passwd</querystring>
<shortDescription>HTTP Access Password File</shortDescription>
<textualDescription>This query looked for a directory listing that might contain a password
file.</textualDescription>
<cveNumber>1000</cveNumber>
<cveLocation>http://www.1000.com</cveLocation>
</signature>- <signature>
<signatureReferenceNumber>24</signatureReferenceNumber>
<categoryref>T3</categoryref>
<category>TECHNOLOGY PROFILE</category>
<querytype>DONT</querytype>
<querystring>intitle:"Index of" ".htpasswd" htpasswd.bak</querystring>
<shortDescription>HTTP Access Password File</shortDescription>
<textualDescription>This query looked for a directory listing that contain a password
file.</textualDescription>
<cveNumber>1002</cveNumber>
<cveLocation>http://www.1000.com</cveLocation>
</signature>-

Google API PAGE
API – License (+MS .net)
Mon 3/14/2005 4:01 PM
Thank you for signing up for the Google Web APIs service!
Please note that your use of Google Web APIs is subject to the terms
and conditions listed below.
Your Google Web APIs license key is 6+6ykixQFHJqpoBdVdCu6Vm8JEjUUZyU
You must include this license key with every call you make to the
Google Web APIs service. This license key entitles you to 1000 queries
per day.
If you have questions, you can join the discussion at the
google.public.web-apis Google Group or send email to
<[email protected]>.
SiteDigger
Mask
SiteDigger
Signatures
SiteDigger
Scan
Results
SiteDigger
Report
Athena
Wikto
Config
Wikto
GHDB
Wikto:
Load
Nikto
DB for
CGI
Vuln.
Scan
Back
End
Googler
SiteDigger2 and Athena2
• Haven’t been able to install .msi file errors
•
•
•
•
Sitedigger2 allows up to 10 hits per signature
Fixes error conditions, false returns
Updated database
Ability to raw search
• Athena2 ???
Likely Findings
• Sensitive Data
–
–
–
–
–
–
Grades, IDs, Rosters
SSN, IDs
Email content, List archives
Credit Card Number (CC#) Repositories
Health Related Information (Dept. Newsletters!)
Source Code to Enterprise Systems, Reporting Systems
• Server Weaknesses
–
–
–
–
–
SQL, Injection, Scripting
Configuration, Backup and Development Code/Scripts
Passwords, UserIDs, Pathspecs, Potential Trusts
Weak Web Practices, Unprotected Data Collection (CC#s!)
Vendor Weaknesses in All the Above
• Old Data, Inefficiency
Conclusions
• Tools are free – barriers to entry few
• Search engines do the work
• XML files can be modified for relative searches in
your domain
• Old data cleanup is essential
• Training on secure development and good Web
practice is weak, particularly in the wild edges
• Consequences for private data leaks can be in
the $Millions!!!