‘Introduce sanity by making people pay for what they use’
By Barry Shein
August 6, 2003

Vipul Prakash makes many good points. I do, however, take exception to his point eschewing payment systems as one possible response to the problem of spam.

For starters, nothing is free. So, to say that e-mail should remain free is already starting out on the wrong foot. The cost may be built into your monthly service fees, but it’s not free. What spammers are doing is breaking the model that led to those all-you-can-eat pricing models.

Spammers are increasing costs to service providers and end-users, particularly companies who manage their own e-mail. These are costs which, for example, generate Cloudmark’s anti-spam product revenue stream.

It’s wonderful to think that somehow a slice of the pie will be allocated invisibly to spam-fighting while nothing else changes, particularly the underlying e-mail pricing model. But the problem is growing too quickly to rely on such a happy outcome.

Spam is ultimately what is called a “tragedy of the commons”: it is in everyone’s individual interest to take what value they can from the medium, even though if everyone does so the result will be the destruction of the medium itself. Spammers “graze” the uncontrolled resource of electronic mail, and unchecked that’s almost certainly a harbinger of the future for more conventional e-mail advertising. One tried and true way to introduce sanity into such a situation is to introduce the reality of market economics: to make people pay for what they use.

This is not to say that it wouldn’t be reasonable to allocate some free use quota so individuals wouldn’t have to worry about being “on the meter” every time they send an e-mail. But beyond some generous allowance, charges should be incurred to help pay for the resources being used and to inject some reality into decision-making about that usage.

I realize this raises new questions regarding accurate and precise payment systems. This can be a complex subject, and schemes needn’t be entirely automated or simplistic. There exist business models outside of the realm of network resource accounting that solve hard usage problems by a judicious mixture of statistical sampling and reasonable business relationships. Legitimate businesspeople understand what spam is doing to e-mail and, in my experience, are receptive to the prospect of paying something to help civilize the situation, so long as it’s fair.

Dave Crocker’s response

‘Usage-based charging would add up to a massive effort and distraction’
By Dave Crocker
August 6, 2003

Vipul Prakash shows excellent insight into the topic of spam control. In particular, his post stresses three concerns that are not given enough attention in the dialogue about spam.

The first is that each spam control approach has significant, inherent limitations and will not “solve” the problem on its own. The second is that combining techniques is likely to make them far more effective, notably by applying relatively marginal techniques in constrained situations. This lets us obtain their benefits without seriously increasing false positives or false negatives.

Prakash’s third notable insight is that the 30-year history of Internet mail service entails some key features we want to avoid losing. His list of three rules for anti-spam software should be a cornerstone to serious discussions about spam control mechanisms. It does not matter whether one agrees with his specific list. What matters is that serious discussion about mechanisms must seriously consider both the impact on the nature of e-mail and the limitations of the mechanisms. At the least, his list should prompt us all to approach changes tentatively.

We need to be particularly careful that changes implemented to control spam do not have catastrophic side-effects on the beneficial nature of e-mail. Those proposing changes often do so cavalierly, either dismissing the likelihood of the change, or the seriousness of the damage from it.

Barry Shein provides us with an excellent example of this error. In his latest post, he gives us the usual lecture about e-mail not being free. Prakash meant that there currently is no incremental charge for e-mail, and that that affords some very important benefits to the interpersonal and intercompany use of e-mail. With the movement of other communication services toward flat-fee (rather than usage sensitive) charging, it is strange to see anyone blithely call for moving in the opposite direction.

The underlying costs of doing usage-based charging for frequent communications, such as telephone calls or e-mail, are astronomical. The negotiation, infrastructure, data transfer, computation, and administration all add up to a massive effort and distraction.

“Negotiation” refers to the small matter of deciding how much to charge and how to distribute the proceeds. Besides the very legitimate question of what is a fair distribution, there will be an absolute feeding frenzy as folks jockey for position at the money trough.  We saw it when charges were first imposed for DNS registrations; the potential revenue for usage-sensitive e-mail charging is vastly larger.

Previous entries:
Prakash (July 23)
Crocker (June 20)
Shein (June 20)
Shein (June 16)
Crocker (June 16)

Don’t Break E-Mail To Save It
By Vipul Ved Prakash
July 23, 2003

I couldn’t agree more with David Crocker’s post “Controlling Spam: Ready, Fire, Aim.” A lot of people seem to be in a rush to blame the spam deluge on the lack of authentication provided by the venerable Simple Mail Transfer Protocol (SMTP) standard. The truth is, existence of SMTP authentication would, by itself, do almost nothing to alleviate the spam problem. It’s important to understand what SMTP authentication really adds to the equation, how it might be useful in the fight against spam and what are its limitations.

There are two kinds of authentication. One is domain based authentication-a mechanism that connects each sender of an e-mail to his or her domain name. Several such schemes have been proposed recently, most notably Reverse MX and SPF, which provide the recipient with an assurance that the e-mail actually originated from the sender’s domain name. For example, if you receive mail from alice@wonderland.com, and wonderland.com was participating in one of the domain authentication schemes, you could be sure that her e-mails actually originated from wonderland.com.

The second kind of authentication is identity authentication with digital signatures, in which the sender digitally signs the e-mail such that it can be verified by the recipient with help of sender’s public key. PGP has been used for identity authentication for the last 10 years by the cryptography and security community, but has not been widely integrated into e-mail applications.

Both kind of authentications have one thing in common: they allow us to trust the “From” field. But what would we do next? Should we “whitelist” known correspondents and quarantine the mail from unknown people in a folder labeled something like, “potential spam”? Clearly, that just shifts the problem from Inbox to the “potential spam” folder; we’d still have to sift through the spam to get at legit messages. We could issue challenge/responses to unknown but authenticated senders. Several challenge/response tools exist today (Spam Arrest, Matador, etc.) that respond to unknown senders with a graphical or analytical question to establish they are indeed human (as opposed to spam-spewing automata). Such systems are flawed without authentication, however, since they end up sending challenges to whatever fake address the spammer puts in the From field. Authentication would benefit these systems but challenge/response interactions alter the e-mail experience (and don’t work for mailing lists), and is perhaps the least desired “solution” to spam. We could use network origination information coupled with sender’s e-mail addresses as a vector in a statistical spam filtration system. Cloudmark does this to some extent already and SPF based systems intend to do this as well; the technique turns out to be a decent metric to prioritize and filter e-mail. However, it’s does not “solve” the spam problem better than filtration technologies available today.

A major change to a well-established protocol will invariably have an extended adoption time-line. Spam filtration systems would have to choose a cut-off point after which they start using the authentication for filtration purposes. If this adoption cut-off is 95 percent (for legitimate mail) than the false positive rate of the system would be 5 percent. I doubt we are going to see 100 percent adoption very quickly. For this reason, I believe that e-mail authentication would be useful only in a statistical context, and not as a deterministic function to drop e-mail on the floor.

My perspective on design of spam filtration solutions is centered around exploitation of the various constraints of the spammer. One thing we don’t talk about enough is the fact that spammers have rather serious constraints. They have to send out a marketing message (containing the same meme) to millions of people from (at most) a few thousand different IP addresses. They have to do this in a relatively short period of time. They have to differentiate their content from other spam. They have to defeat existing spam filtration systems.

A successful anti-spam solution will be able to leverage one or more of these constraints to differentiate spam from anti-spam. Cloudmark’s product, SpamNet, for example, leverages the “same meme to millions of people” constraint by allowing the first few recipients to identify the meme and share the knowledge with other intended recipients before they receive the message. Bayesian classifiers, on the other hand, leverage the fact that marketing messages are not statistically representative of the mail a person receives. Blacklists, like RBL and SpamCop, leverage the fact that spammers have a limited number of IP addresses from which to send their spam. Cloudmark’s Authority product looks for “mutations” in a message crafted to defeat anti-spam systems.

Some of these constraints make for good differentiators and others don’t.  IP addresses, for example, are a bad differentiator. A spammer can use a major Internet service provider to send out spam, and blocking the ISP’s IP would block all the legit mail originating from the ISP. However, the knowledge of IP combined with another constraint (e.g., meme X originating from IP Y) can be a good differentiator. SMTP authentication would provide us another constraint to exploit. Like IP address, SMTP makes for a poor differentiator by itself (specially domain based authentication), but could be useful when combined with other constraints.

But do we really need to introduce another constraint to defeat spam? I don’t believe that we do. If we develop clever systems that can effectively leverage existing constraints we can solve the problem without requiring authentication. Still, if I had to choose, I would be more supportive of identity-based authentication techniques that employ digital signatures, since such schemes will be based on identifying individual users (instead of domains). As far as I know, no such scheme has yet been proposed for fighting spam, but they can’t be far from coming.

Barry Shein talks about augmenting e-mail to use payment systems. Assuming such a system ever gets widely deployed, it will change e-mail a lot more radically than authentication. The tradeoff is more apparent in this case; is addition of another constraint that helps us solve the spam problem worth radically changing the way e-mail functions? I think not.

I am a fan of the current Internet e-mail architecture. If e-mail had strong authentication or payment systems built into it, I doubt it would have been as wildly successful as it is today. David Crocker’s warning is well grounded: Let’s not go off and change e-mail, specially not without an understanding of what can be done with the resources we have today.

Here’s a list of three rules (created after the most important features of e-mail) that anti-spam software should strive to follow:

    1) Ability to send and receive e-mail from a stranger. (Whitelisting, payment systems, and challenge/response break this rule.)
    2) Ability to send and receive pseudo-anonymous e-mail. (Domain-based authentication breaks this rule.)
    3) E-mail should be free. (Payment systems break this rule.)

If we can solve the spam problem while maintaining the three features of e-mail, it would be a much sweeter victory.

Vipul Ved Prakash is founder and chief scientist for Cloudmark.

Previous entries:
Crocker (June 20)
Shein (June 20)
Shein (June 16)
Crocker (June 16)

Controlling Spam:  Ready, Fire, Aim
By Dave Crocker
June 20, 2003

I am starting to suspect that our impatience to take some action against spam will turn out to be the most serious barrier to taking useful action against it. Rather than trying to gain control of spam by attacking it at its social and technical core, we seem to want to let the spammers define our response and, thereby, let them change the entire nature of e-mail.

Since some spammers send mail with a fraudulent “From:” field, there are anti-spam warriors who say we must prevent anyone from sending anonymous mail, no matter how much that is a critical part of the repertoire of human interaction. If spammers show up uninvited, then let’s eliminate all mail that does not have prior permission, no matter how much such mail is a critical part of commerce and serendipity. If spammers use HTML content, then we must ban all HTML in e-mail, no matter how much the content is enhanced over the Draconian (and ethnocentric) brutality of 7-bit ASCII. If spammers send mail about Viagra or pornography, then we must ban all mail that mentions these things, no matter the impact on free speech.

In the process of making email safe from spammers, we are in danger of making email content-free.

Perhaps we should slow down a bit, and try to understand the problem, before we act so hastily.

No one requires that postal mail be signed or have a return address. No one requires that telephone calls identify the caller by name-and there is nothing that guarantees that a disclosed telephone number tells you anything about the identity of a caller.

If we make sure that an e-mail sending host is properly identified, what do we actually know about the sender of the content?  Not much. Yet host identification is at the core of a number of popular proposals.

If we make sure that the author of each e-mail is properly identified, what do we actually know about the propriety of that content?  Not much. Yet this, too, is a commonly suggested solution.

In order to attack social misbehavior, we need to be clear about the things that make the behavior unacceptable. Just because that behavior is accompanied by some obvious traits does not make those traits relevant to controlling the problem. We could make every e-mail host identify itself,  and we would still have masses of spam. We could require that message content be signed, and we would still have masses of spam.

These rash steps will not stop spam, but they will reduce or eliminate e-mail’s usefulness.

If you live in a house in a small, friendly town, you probably do not lock your doors. As the town grows and becomes more diverse, your model of home security changes quite a bit. Eventually you need state-of-the-art locks on the doors and grates on the windows, with an alarm system that is set off if anyone thinks too loudly.

This is not a pleasant reality, but it is one we understand. However, we should note that no one says that the only way to live safely is to tear the house down and replace it with a stone fort. We adapt the house to suit the real security needs of the changed environment. And we try very, very hard to make as few changes as possible. We take this minimalist approach because we understand just how onerous grates on windows, alarm systems, and the like, are on the quality of our daily life.

Let’s try to be equally judicious when attacking the problem of spam.

Remember that firing without aiming is a good way to shoot yourself in the foot…  if you are lucky.

Barry Shein’s response

‘Weapon of Mass Distraction’
By Barry Shein
June 20, 2003

Although I agree with Dave Crocker that dumb solutions to spam are dumb and therefore shouldn’t be adopted, we also have to be careful that we’re comparing apples to apples.

For example, postal mail is, as Crocker describes, unidentified. You can write any origin address you like in the upper-left corner of a postal envelope and it will get delivered if only the recipient’s address and postage are correct. However, given the per-piece price of paper mail it’s unlikely anyone is going to send millions of people enough pieces to make it a true problem.

Postal mail just doesn’t have the potential as a weapon of mass distraction like e-mail. The same goes for telephones. The telemarketer has to pay per call in most venues. There are also severe limits on the kind of equipment that can be attached to telephone lines for mass telemarketing and the number of simultaneous calls is limited by the number of outgoing lines the telemarketer has-another expense that scales with the problem. Finally, there are all those do-not-call-lists, which are actually enforced. Massachusetts, for example, offers the protection of such a list to all its residents at: https://www.madonotcall.govconnect.com

But telephony is somewhat more evolved in this way and still does not have the power to harass millions daily for (generic) beer money as spam does.

Spam  is more akin to the marketeer who would mount loudspeakers on a vehicle and drive through your neighborhood at 3:00 A.M. blasting an advertisement. Or someone who would break into a TV or radio signal with his own broadcast or cable equipment in order to insert his advertisements.

Those analogies emphasize the base illegality of the spammer’s methodologies and demonstrate that we have managed to deal with such excesses without creating new threats to our constitutional rights.

As to Crocker’s analogy of grates and alarms protecting your home as a town becomes a city: I’ve often said that a measure of civilization is that I can have glass in my windows and still consider my vehicle or home reasonably well “locked up.” When someone forces me to board up my windows out of fear, the idea occurs to me that the hammer in my hand needs to be applied to the perpetrators rather than my window sills.

That’s about where we are with spam; desperate enough to begin considering draconian responses analogous to boarding up all the windows in our home just to feel safe.

What we need to do is go after the spammers themselves and stop cowering in the dark behind boarded-up windows.

Previous dialog entries:
Shein (June 16)
Crocker (June 16)

‘E-Mail As We Know It Is Dying’
By Barry Shein
June 16, 2003

As someone who manages an Internet service provider, my perspective on spam is a little different than many you’ve seen in the press. For example, in the Technology Review article, Evan Schwartz writes about dictionary attacks, wherein a spammer guesses mailbox names by sending to john1, john2, john3, and so on. Probably 99 percent or more of those millions of guesses are going to just bounce back with a user unknown error.

Unfortunately, the return addresses to send those errors to are invariably phony so the mail cannot be returned. Who do you think looks up all those address guesses? Who tries to return those millions of bounces for days? Servers such as ours. Our systems get choked with this non-stop effluvia.

It is this sort of problem which makes spam like the proverbial iceberg. The junk in your mailbox is just the tip of that iceberg. Lying beneath the surface are the 99,999 guesses the spammer made to discover your mailbox. As Paul Judge points out in the article, this is likely to grow worse as spammers work harder to get around anti-spam schemes.

From this perspective I’ll make two observations about the spam situation:

The first is that fraud and criminality are inherent in spam. Spam is organized crime come to the Internet. To you, it may be a come-on for a mortgage or body enhancement product you don’t want. To me, it’s 200 or more computers simultaneously spewing that same message at our servers. The spammer doesn’t own 200 computers. These computers are often infected with viruses turning them into unwitting spam slaves. This, and exploiting misconfigured server software, is not the behavior of a new breed of businessperson just trying to make a living on the cyber-frontier. These are the machinations of sociopathic criminals.

My second observation is that, from this front-row seat on the problem, it looks like e-mail as we know it is dying. Ultimately it won’t just be these criminals who killed e-mail. They’re just the first wave. Next will come the real, professional, marketeers. The ones with the millions, and in some cases billions, of dollars in their ad budgets.

Even so-called “opt-in mail,” where the recipient explicitly asks for the material, is doomed.

Consider the analogy of the U.S. Postal Service. Will they deliver a magazine or catalog to you for free just because you subscribed or had a bona-fide business relationship with the sender? Then how well is this going to work with e-mail? Your web of e-relationships grows as time goes on, it doesn’t diminish. Marketeers never forget a contact, particularly when sending out ads is virtually free. The volume is only going to increase. Add in multi-media mail and the future doesn’t look good for this free-for-all.

The current model of e-mail is doomed. It was a nice experiment but it didn’t work out. What we need to do now is move to a sender-pays model, perhaps with some allowance for bona-fide personal usage.

Barry Shein is founder, president, and CEO of the Software Tool & Die, which  launched the first commercial dial-up Internet service.

Dave Crocker’s response

Like Fighting Cockroaches
By Dave Crocker
June 16, 2003

The Technology Review article on spam shows the wide range of approaches that are trying to deal with the problem, and the implication that effective control is going to need the combined effect of more than one. The piece suggests that people promoting a particular approach tend to think that it, alone, is necessary and sufficient. This makes discussion about incremental (i.e., partial or combined) effect very difficult. So the article does a good job communicating just how messy the topic is.

My own combination of social science training, commercial network operations, and participation in the evolution of e-mail technology tempers my emotions and expectations about spam. It is a serious problem and needs serious attention, but that attention needs to be realistic rather than simplistic. I suggest that we view spam the way we view cockroaches. We are not going to eliminate roaches, but we can control them down to an acceptable level. However it takes a range of techniques-what the article nicely calls an “arsenal.” Worse, just like these nasty critters, spammers adapt over time, and as with roach control, spam control techniques must adapt over time.

As we consider the ways to stock the arsenal, there are some key points we need to keep top-most in our minds: The article notes that spam has no core, technical differences from legitimate mail. John Mozena correctly observes, “It’s not as if the Internet is broken. You can’t address social problems solely with technical means.” Alas we do not even have broad agreement about an operational definition of spam. Folks range from saying it is “whatever I don’t want” to “unsolicited bulk e-mail” (UBE). If we build controls based on the first definition, we will never have any spontaneous contacts through e-mail. So I prefer the latter term.  Most folks agree that UBE is a core problem, even if some insist that the total problem is larger. If we do something useful about UBE, we will have a meaningful impact on spam.

Let’s consider some of the items to place in the arsenal. I think that the article’s discussion of legal actions misleads the reader. It is popular to cite the dominance of U.S.-based spam and that its financial basis makes it possible to “follow the money.” It also suggests that we might be able to throw out the existing e-mail service and replace it with something newer and better. This creates a strong sense of being able to hold spammers accountable and it forgets the observation that spammers adapt.

At the Federal Trade Commission’s April Spam Forum that the article mentions, some presenters discussed their attempts to enforce existing laws and had painful stories about the difficulty in tracking down spammers. Better laws will not change this. Note that there is no “international” law and we are never going to get all countries in the world to pass, and vigorously enforce, strong anti-spam laws. Spammers will mount their global attacks from whatever haven is available. Better laws will provide a clear, common, operational definition of spam, and better laws will provide meaningful guidelines for acceptable behavior. This will be useful for controlling “responsible” spammers-those nice people who run legitimate, accountable businesses but are just too aggressive with their e-mail marketing campaigns.  The laws will have no effect, however, on other, “rogue” spammers.

I disagree with my correspondent in this dialogue, Barry Shein, on one big point. His claim that spam is inherently fraud-what Jon Praed is quoted in the article as calling a violation of Common Law prohibition of unauthorized use of someone else’s property-is just plain wrong. It is entirely acceptable for me, personally, to send one message to any random person and it always has been. Law likes precedent, and Internet mail has always operated with implied permission for such unsolicited contact. So, yes, I want to fight spam vigorously, but let’s not distort the legal issues.

With respect to simply replacing existing e-mail with “something better” the question is what needs to be in that something better and why can’t it be supported as an increment to current e-mail? E-mail has gone through 25 years of constant change, always building incrementally. For example, there is already a mechanism for restricting access to SMTP relaying. And there are already two techniques for signing messages digitally. So, perhaps we will have to replace SMTP-based e-mail, but we need the technical and operational reasons to be clear and compelling. So far, those reasons are missing in action.

Like everyone else who is engaged in discussing this topic, I could ramble on for quite a few more screenfuls. I’ll stop here, to let folks start shooting back.

most recent entries

Dave Crocker is a principal with Brandenburg InternetWorking, a consultancy focused on the creation of distributed applications. He authored or contributed to most of the technical standards that make Internet mail possible.