Transcript Using Traffic Shaping to Combat Spam

Using Traffic Shaping to Combat Spam
David Cawley, Senior Engineer
December 12th, 2007
Overview
1. Evolution of E-mail & Spam
2. Spamonomics
3. SMTP Multiplexing
4. Traffic Shaping
5. Asynchronous IO
6. Passive OS Fingerprinting
The Dawn of E-mail
• 1965 MIT shared mainframe
• 1971 The @ symbol
• 1976 Queen of England sends an e-mail
• 1982 IETF RFC821/822
• 1989 Lotus Notes released (35k copies sold)
• 1996 Microsoft Internet Mail 1.0
• 2001 IETF RFC2821/2822
Attempts to secure...
• SMTP is inherently insecure
• SMTP-Auth/TLS
• SPF
• Sender-ID
• Why it didn't stop spam
The Evolution of Spam
• 1978 The first spam
• 1988 Usenet cross-posting
• 1993 “spam” coined as a name
• 1997 Open Relays abused
• 2000 Birth of Nigerian spam
• 2001 Formail exploit
• 2003 Sobig virus sends spam
The Evolution of Spam
• 2003
CAN-SPAM act
• 2004
Bill gates prediction & botnets
• 2005
Image spam, Ascii art
• 2006
Animated images, flash, pdf
• 2007
mp3, excel, p2p botnets
The escalating spam problem
The good old days.
Source: spamnation.info/stats
Spammer Economics
• 0.02% people click and buy [source: NY Times]
• Average filter effectiveness is 90%
– 1/10 of spam messages get through
• Improve effectiveness to 95%
– 1/20 of spam messages get through
• Spammer Solution?
– Double spam volume
– Same profit
Traditional Filtering
• MD5's, Fuzzy Signatures, Bayesian
• Header Regex, RBL's, URL Lists, Grey Listing
• Problems
– Obfuscation Techniques
– Formats – html, image, pdf, doc, xls, ole, mp3..
– Zombies, Botnets
How often do we see a unique Botnet IP?
The Number of Unique IP's versus the number of times reported
800000
700000
600000
# Unique Botnet IP's
500000
400000
300000
200000
100000
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Times Reported
17
18
19
20
21
22
23
24
25
26
27
28
29
30
SMTP Multiplexing
• Transparent SMTP Proxy
• Connection Pooling
• Insulates the MTA
• Avoids delay of legitimate mail
• High Concurrency
– Up to 10,000 simultaneous connections
12
Traffic Shaping
• What can we do?
• Provide a Quality of Service
• Reputation Network
• Throttle unknown senders
• Fast track legitimate senders
Percentage of Connections Still Connected
Spammers are Less Patient than Legitimate Senders
100%
90%
Spammers
80%
Legitimate Senders
70%
60%
50%
40%
30%
20%
10%
0%
0
50
100
150
200
250
300
350
400
450
Time (Seconds)
16
Does Sendmail Throttle?

ratecontrol

ConnectionRateThrottle

conncontrol
Asynchronous IO
• Non-Blocking front end
• Blocking Back-end
• Event driven
• Finite State Machine
• Management of Resources
Passive OS Fingerprinting
1.Look at IP packet data
2.Determine the Operating System
3.Decision to Throttle
OS Comparison
Conclusions
1.Spamming is driven by economics
2.Botnet operators need to make money
3.Slowing down spam makes it go away
Nick Shelness, Former CTO, Lotus:
“I am able to report that I have been running an instance of
TrafficControl in my own network for four months, and that it has
reduced the volume of spam hitting my boundary MTAs on most days
by approximately 95%.”
[email protected]
+1-778-785-6143
www.mailchannels.com