Transcript Fighting Spam at AOL: Lessons Learned and Issues

Fighting Spam at AOL: Lessons
Learned and Issues Raised
Carl Hutzler
Director of Anti-Spam Operations
America Online, Inc.
12/9/2005
1
Agenda
• Email Identity Technologies
• Email Forwarding
• Email Service Provider Best Practices
2
What do Email Identity Technologies Do?
• They provide some assurances that a domain is being used with
permission
– Citibank can control the use of their domain, but cit1bank.com will
still be abused
– Bounces can be analyzed to see if they are legitimate
– Information can be analyzed on the responsible domain owners
and their reputation/accreditation
• But remember, email identity technologies do not stop
spammers!
– They only force spammers into other behaviors, many of which are
better for enforcement and controls.
– But without message providers doing their part to use these
technologies wisely, we will be no better off.
3
AOL is a Crystal Ball
Report from 9/14/2004
188841 hotmail.com
64543 x-mailer.co.uk
62757 shawcable.com
46312 concentric.net
32259 cnchost.com
32022 zero.ou.edu
23557 mail.atl.earthlink.net
22837 grp.scd.yahoo.com
21005 ucla.edu
17676 oemgrp.com
16849 mail.cornell.edu
16260 dejazzd.com
15764 mta01.tie.cl
15659 mrf.mail.rcn.net
14343 urbanhomesecurity.com
14280 mail.pas.earthlink.net
14246 smtp.nextra.cz
13646 mail.yahoo.com
Note1: Greyed domains have very low spam penetration
due to very large number of emails sent which counters the
total complaint statistic.
Note2: Italic domains were whitelisted and subsequently
blocked for spamming.
•
•
•
•
Bulk Mailers on AOL’s whitelist comprise 3050% of our daily email volume but only 5-10% of
complaints.
>80% of AOL’s spam problem comes from other
provider’s main outbound MTAs and
compromised web servers (CGI scripts)
AOL began seeing this shift in Sept 2003
The rest of the internet is beginning to see this
now…
–
“We're the biggest spammer on the Internet," network
engineer Sean Lutner, Comcast - source CNET.com, May
24, 2004
4
All spam will eventually come from Email
Message Provider Networks
For example: AOL, BlackLists, and
other organizations are getting really
fast at blocking zombie machines
MyDoom’d ZOMBIE PC on DSL.NET
BLOCK
mx.aol.com
Hacker/Spammer
outbound1.dsl.net
BUT…
The machines do not get un-infected
No SMTP AUTH
• Most ISPs “trust” internal
networks
No Outbound Spam controls
No Rate controls
Results?
ISP mail servers act as forwarding
MTAs for a network of open relay
Zombie machines
5
Will SenderID, SPF, DomainKeys, etc
stop spam?
•
•
Simple answer, NO. Complex answer, NO.
Why?
– Most AOL spam obeys sender identity technologies TODAY!
– Spammers send through the local MTA and use the local ISP’s domain as
the FROM/Sender
•
Identity Technologies can allow blacklists/whitelists to work from
DOMAINs instead of IP addresses
– Good from a not blocking innocents by IP address standpoint
– Reputation/Accreditation systems will be key to success of Email Identity
technologies
– Without SMTP Authentication, we are only validating the DOMAIN and not
the USER portion of the address ([email protected])
Bottom Line: If ISPs don’t get smart soon and control the sources of spam on their
networks, the reputation for their domain (e.g., comcast.net) will be so poor that they
will not have connectivity to other ISPs
6
Email Forwarding
7
Forwarding Spam to AOL Customers
• AOL can only trust the IP address of the client MTA
that connects to an AOL server
– No other headers can be trusted as they are all forgeable
– This is why internet whitelist/blacklists are all done by IP
address.
• AOL has no way to no that a message is simply a
forwarded email
– Does this even matter?
8
So what happens when a University
FORWARDS Spam?
• Generally, if AOL gets enough complaints from our
members, we block or temp fail the IP address
• Is this the members fault?
– No, as there is nothing in the email that shows it is from their
forwarded account
– AOL members do not read headers, nor should they be
expected to.
9
Possible Solutions?
• Dedicate an IP address to handle forwarded mail and tell AOL
about it.
• Do better spam filtering inbound to your network.
• Spam filter the outbound traffic and insert a spamassassin xheader that identifies a message as spam. AOL will spam folder
it.
• Change the headers of forwarded mail to identify the situation to
final recipient.
– From: [email protected]
– Subject: [FORWARD] Original Subject
– ReplyTo: [email protected]
Bottom Line: Forwarding spam to someone’s inbox innocently or
intentionally still creates a bad experience for the final recipient. Port25 is
your responsibility.
10
Mail Service Provider Best
Practices
11
Message Provider Code of Conduct:
Take Responsibility for outbound Port 25
• ISPs must take full responsibility for all
traffic/messages emanating from their network on
port25.
– Port25 traffic is always Unauthenticated traffic and as such
must be accepted by server MTAs.
– Abuse issues are always the responsibility of the
sending/client MTA
12
How does a Message Provider like AOL
control outbound port25 traffic?
• Hijack all direct port25 connections from dynamic IP space to
other ISP mail servers and process it for viruses/spam.
– Other providers block port 25
– Still others use a mail proxy to detect SMTP authentication
credentials and only allow authenticated SMTP traffic on port25
– Some simply rate limit how much a single IP can send if their IP
space is rather static or they can tie an IP to a customer account
• Rate limit all customers through outbound, authenticated MTAs.
Rate limits per hour and per day work well.
• Monitor complaints about customers via the SCOMP Feedback
Loop system
• URL blocking for known spammer URLs
• Secure accounts that are spamming - thousands daily
13
Summary: What technologies will stop spam?
•
ISPs and Network Providers “waking up” and working together to cut off
the spammer’s oxygen supply:
– Spammers need connectivity
– Spammers need large numbers of high throughput IP addresses
•
So what is the formula for success?
– ISPs should monitor their networks for sources of spam LEAVING their
network
• Port25 is always the responsibility of the originating ISP
• Shift some of the resources from inbound filtering to OUTBOUND Controls
– Enforce strong authentication to authorize use of an ISPs MTAs
– Monitor customer sending patterns like a credit company monitors
“fraudulent charges”
– Monitor/Sign-up to receive complaints from AOL and other sources
(spamcop, abuse@, etc)
– Remove sources of spam within minutes (Zombie machines, insecure CGI
scripts, bad customers, etc)
14
Thank you!
• For more information, contact Carl Hutzler:
– [email protected]
• Delivery issues to AOL?
– See if your network is a source of spam
• http://postmaster.aol.com/
• Click on the “Feedback Loop” Button
– Contact the AOL Postmaster 24x7
• 1.888.212.5537
15