An Email Expert's Perspective On Spam
27 November 2003
The following thoughtful and well presented piece is a resonse to
a column I wrote in which I
suggested that improved email protocols might be a solution to the
spam problem.
David Harris, developer of the excellent
Pegaus Mail program
(which I've used for years and years) wrote this:
The first thing you have to do is ask: "what changes could be made to the
Internet's mail system that would defeat spam?". In fact, this is not
nearly as simple a question as it seems; the most common suggestions I
have seen are the following:
- Introduce a small charge for every mail message
- Require everyone who sends mail to use a valid public-key certificate
that clearly and unequivocally identifies them.
- Only allow servers to use authenticated connections based on valid
public-key certificates (there is actually a proposal called AMTP in front
of the Internet Engineering Task Force [IETF] that describes this
approach).
- Only allow mail from pre-approved addresses to be accepted (also
called "whitelisting").
- Only allow mail to be accepted if the authoritative DNS entries for the
domain match the machine making the connection (in other words, only
accept mail from machines formally designated mail exchangers in the DNS
records for the domain).
I'll deal with these ideas in three groups - the first group contains
items 5 and 4, the second group contains items 3 and 2, while I'll reserve
a special group for item 1; the reason for the groupings will become clear
as I go along, I hope.
The first group
The first group, consisting of items 5 and 4, is currently feasible -
these things can be done without completely tearing up the basis of
Internet mail and starting again. Item 5, in fact, is already reasonably
widely-done to varying degrees. Both approaches have some basic problems,
though:
- Approach 5 relies on all domains wanting to send mail having
properly-configured DNS entries that are 100% available. This results
in a vast increase in DNS traffic, increased costs and management for
smaller organizations, and some serious complications for sites who
are running a mail server across a link with a dynamically- allocated
IP address (such as ADSL). My own experience when I tried adding this
to Mercury (my mail server) was that about 5% of all legitimate mail
was turned away because of network outages, transient DNS failures or
DNS configuration errors - and that was two or three years ago, before
the rise of ADSL.
- Approach 4 actually works and is a good solution for individuals who
only ever exchange mail with a static community of colleagues. In
essence, when a mail message is sent from an address that the intended
recipient has not previously encountered, the sender receives an
automated response requesting that they go to a web page and confirm
their identity; once the sender has done this, the message they sent
is delivered, and all subsequent messages from them are automatically
approved. On the surface, this sounds like a perfect solution, but as
is often the case, practice reveals a different outcome: whitelist
schemes of this kind almost invariably result in people losing mail or
antagonizing those who legitimately want to communicate with them. It
doesn't matter how simple and painless you make the verification
process, a significant proportion of senders simply won't do it, and
the mail will be discarded. There are also some really serious
problems with this type of approach if the user wants to belong to
mailing lists, or to use e-commerce facilities (many e-commerce
facilities send an e-mail message in confirmation to an order and do
not ship the order until they get a response). Still, if the problem
of spam continues to escalate, this may be the only genuinely workable
technical way of getting around it - we may simply have to make a
social change and accept that verification prior to delivery is a fact
of life.
The second group
Items 2 and 3 are similar, in that both require public key infrastructure
to make them work: these two approaches are very commonly proposed,
usually by a class of people who see encryption as a solution to
everything or who have a vested interest in the technology. There are some
very real problems associated with technical approaches based on
authentication, though:
- Both approaches depend on the existence of a viable, globally-
accessible public-key infrastructure. They need authenticable
certificates issued by responsible CAs (certification authorities,
such as Thawte or Verisign). Such certificates are very costly (US$250
per annum is not unusual) and are not at all easy to obtain. There are
also very few CAs out there, most of them being based in or controlled
by the USA (for many people this would be a problem in itself). Given
that there are tens of millions of mail servers and hundreds of
millions of users (even billions) it's not at all clear that the
existing CAs could cope with new systems requiring that level of
support and maintenance. A vast new global high-security industry
would be required to accommodate the necessary infrastructure.
- Both approaches depend on some fairly heavy-duty encryption -
typically SSL/TLS: this introduces a significant level of complexity
into the protocols and potentially creates export and certification
headaches. There's also the problem that SSL has some fairly severe
interoperability problems at the moment (although these will be ironed
out over time).
- Because a side-effect of these approaches is typically that all mail
transactions will be encrypted, there is likely to be significant
opposition from shady groups with an interest in intercepting mail,
such as the FBI, the NSA and the Department of Homeland Security.
- Approaches like this would essentially require the complete
replacement of the entire e-mail infrastructure of the Internet, and
the redevelopment of every client and server. Is this easy to imagine?
But the biggest problem with these approaches is that they actually do
nothing to reduce spam - all they do is give you certainty about who sent
the message. In theory, they could be used as as basis for setting up a
register of known abusers, but that's also a major infrastructure addition
with its own web-of-trust and technical problems, and nothing prevents
spammers from either stealing certificates or virtual-relocating to TLDs
that are happy to issue disposable domains (I can easily imagine a
flourishing market in this type of thing springing up almost overnight if
this scenario ever arose).
Furthermore, there are some major privacy implications with this -
anonymous mail becomes impossible in an environment of this kind, and many
services such as mailing lists could easily be compromised by
authentication requirements.
I tend to regard the items in this group as solutions looking for a
problem: they are really political agenda items thrown up by people with
vested interests or barrows to push, rather than any kind of serious
attempt to deal with the issue of spam. I find it particularly interesting
that "Uncle Bill from Redmond" seems to favour approaches like this - it
would be a perfect way for him to lock the world even more tightly into
mail that depended exclusively on his products, while appearing to be
doing the world a service: perfect PR. Am I a consipiracy theorist? I
think not.
The last group
My greatest scorn, though, is reserved for those who suggest collecting a
small fee on every mail message sent... This is naïve in the same way that
it would be to say "Let's solve all the poverty in the world by giving
everyone $100,000": it's an easy formula to spout, but impossible to
implement. Consider the following -
- Who gets the money? Governments? No thanks! ISPs? Who
regulates them, audits them, makes sure that they're honest?
- How is the money collected? People proposing this scheme usually
talk about numbers like a cent a message, but how do you collect a
cent a message? How can you charge it? Micro-billing is an issue the
credit card companies have been struggling with for years without
working out an effective method. Who provides the infrastructure
necessary for tracking and collecting the money? We're talking about
billions of tiny transactions every day across the entire globe!
- What happens to the money? We're talking about millions and
millions of dollars here: what does it get used for? Who decides that?
How do they get appointed, audited, verified? Which countries get what
share of the proceeds?
- What happens to people who have legitimate, opt-in mass mailing
needs? Why do these people get penalized?
- What happens to mailing lists? Who carries the cost of a message
sent to a mailing list?
...and so the list goes on. This approach is simply unworkable, even in
an ideal world. Anyone who seriously suggests this as a "solution" to spam
simply hasn't thought through the issues well enough.
--
On the surface, it's easy to say "overhaul the Internet's e-mail - that
will fix spam", but it's not at all evident that such an overhaul WOULD
fix the spam problem, even if it were practical to consider doing it.
Furthermore, as long as there is money in spam, people will find ways of
propagating it: anything science can devise, science can also circumvent.
The real solutions to spam aren't really technical at all... Technical
solutions are at best a stop-gap that might allow us to scrape through
until the real solutions exist - and those real solutions are two-fold:
- Make spam illegal, and enforce the law. I'm not saying that this is
easy, but it's the essential first step. The law is how society defines
what is wrong and what is right, and at the moment, spam is legal in
almost every country in the world. We need a global push for anti-spam
legislation and a few high-profile cases that result in imprisonment or
vast fines - once that has happened, a disincentive will exist for
spamming, and remedies will be possible for abuse. Without legal penalties
and the threat of formal enforcement, we have no remedies more severe than
harsh language, and suspect that most spammers are far too thick-skinned
to be bothered by that.
- Undertake a huge program of public education promoting the three
Golden Rules of spam:
- Never purchase anything advertised in a spam
- Never reply to spam
- Never use the "Remove" link in spam
The first point is crucial - if there is no money to be made from spam,
then there is no incentive to send it. As long as there are gullible twits
out there who buy the type of dross peddled by spammers, there will always
be spammers to sell to them.
[END]
Okay, so what do you think?
Have your say.
