Spam belt tightening done badly rejects legitimate mail

I'll plug my own project, CAKE. Seriously, I think there are two major thrusts to the answer. The first is a strong system of public-key backed pseudonyms, which is what CAKE's main purpose is. The second is a built in challenge response protocol that requires the person the challenge is issued to perform some sort of task or give you money.

The task could be some sort of CAPTCHA thing designed to identify humans. Or it could be some sort of expensive computation that is feasible to do to communicate with one person, but isn't feasible to do for each person when you want to send mail to millions.

Maybe we should reedirect all spams to the moronic postmasters.

the issue as i see it is that retro-fixes to an existing system just ain't gonna cut it.

it's more a social engineering thing than anything else.

you have to convince people to implement such a scheme on windows, linux, solaris, mac, everything, and in postfix, exim, sendmail, qmail - the works.

maybe that's possible, maybe it isn't.

personally i think that an approach that stands a much better chance of success would be to design an _alternate_ service to SMTP, backed by reputable ISPs and the IETF, designed from the ground up with security, authentication, identification and filtering in mind.

at least then you would stand a chance of being able to make a global announcement, "better SMTP than SMTP! no more spam!"

hey, i'd be delighted if it were possible to retro-fit MTA filters into SMTP that were easy enough for any programmer to write a plug-in module on any system, and delighted if it were taken up and actually installed - and used - correctly.

...maybe we're all looking at this the wrong way: maybe it's possible to mandate the use of IPsec or IPv6 and to use that as a chance to cut unauthorised spammers off at the knees...

As pointed out before, "If foolish coders ignore the old standards, why would they magically pay attention to the new ones?"

... for the reference. ... much as i am confused to say it, and mad as it is, the present situation is about it!

other than... how about creating VPNs (tinc, openvpn, ipsec).

e.g. how about this:

ISPs create, amongst themselves, VPNs that they pass mail between themselves over that VPN (store and forward?).

ISPs only allow email into their network from trusted - registered - users. it should be easy for users to register with another ISP's SMTP service if an ISP misbehaves.

so, a bit like DNS where there are TLDs and it goes down from there, mail could be routed "up" the tree to the SMTP-TLDs and then "down".

the rules are that you do NOT allow SPAM from one level to the next, and if you do, you can be cut off at the knees, by your peers, not by some arbitrarily managed blacklist.

the smaller and more involved the group of peers is, the more likely they are to help each other out, and the more likely they are to take swift and _correct_ action.

ultimately, users (us) would not "see" any of this: you would have mail delivered a la SMTP.

the significant difference is that, rather than at the moment where email can get trashed by some company running brain-dead SMTP server that doesn't check the RBLs properly, they can trust the top-level SMTP servers delivering to them _and_ they can block the firewall rules in the surefire confident knowledge that no legitimate SMTP traffic will ever come to them from anywhere but their upstream SMTP provider.

their store-and-forwarder.

i'm writing this as i think it, out loud, i'll try to clarify it a bit, later, as it pulls in several ideas and existing practices that i have heard of, but the hierarchy bit and VPN bit is a new idea (hey to me, at least).

l.

The poster seemed to indicate that the SMTP servers need MX records. I don't think that's the case, and could be the cause of your difficulties. MX records are only needed for receiving mail, not sending it.

thank you for correcting my misconceptions. i check the exim logs and see a lot of messages rejected by the exim4 + sa-exim combination, because there is no MX record possible to be retrieved for the name: turning this around, i misconcluded that a sender would have to have an MX record.

this report in the times newspaper, today, quotes 70% of all email traffic, 3 billion per day handled by the lovely aol.com, as being spam, and likely to be 90% by the end of the year.

indication from spamhaus interview on how serious this is getting

if spamhaus' assessement is correct, in that 90% of all email by the end of the year is going to be spam, and that there are 400,000 "granny-boxes" out there with windows-virus-installed SMTP servers, we could indeed be looking soon at the total breakdown of email communication.

based on this assessement, it is almost worthwhile having in place a strategy to replace email.

or windows.

hasn't it occurred to anyone yet that microsoft's product is causing such a serious problem?

I don't like the VPN idea at all. It smacks of a centralization that will be even more harmful than the spam problem. I prefer adding things to email clients, and changing the nature of the messages that are transmitted so that it's much harder for spammers to send you messages that you'll actually read.

hierarchy and peer cooperation. in the UK there are 25 major ISPs: i would expect those 25 ISPs to be responsible for... say... store-and-forwarding all *.uk domain name based email. for example.

not everyone can have stuff added to email clients, that's the whole point.

for example, if using GPRS, you _certainly_ don't want your 14,400 (allegedly 33,600 baud) bandwidth - for which you pay £1.50 per megabyte - loaded 90% with total shit: that makes it ten times more expensive - approximately £0.20 - per actual real email received (est.)

the VPN part isn't strictly necessary: it kinda "codifies" however the relationship between ISP peers and their communication paths up and down the domain hierarchy.

store-and-forwarding is becoming common: firewalling to only accept incoming SMTP from the forwarding SMTP server is perhaps not so common but is something i would like to see people take up as "the norm".

at least it will cut out the 400,000 or so compromised NT systems.

in the end it should even be possible for ISPs to simply block SMTP port 25 even between their dial-up clients!

If people stop responding to spam, it'll stop being sent. And if that's not what happens, the Internet infrastructure is in big trouble no matter what. At the current exponential growth rate, it won't be long before 90% of the backbone bandwidth is used by spam.

Any kind of exclusive arrangement at all seems the wrong thing to do to me.

There's new stuff that gets added to email clients all the time. 10 years ago, no email client had MIME. Now, only Unix email clients widely support PGP/MIME. New stuff can get added to email clients. If it solves a pressing problem that a lot of people have, it will be added to email clients extremely quickly.

There are people able to sent SPAM.
There are people paying for SPAM to be sent.
There are people reading and using SPAM for their benefit.

So ... Up until all of them exist SPAM will be. No blocking will help. :)

Is there any solution? Yes - if there are people who need it, then we should do it! But in the manner which is not abusing ISP or users.

1. We should create Spam Delivery Protocol (ICD - Internet Comercials Delivery).
2. We should convert ICD to e-mails for the end-users, but with addition of "unsubscribe" mechanism.
3. We should popularize the idea among business, ISPs, users, admins and SPAM-senders

If you could do system, which will cost less and be close-targeted then SPAM, then there will not be any reason to pay SPAMers. You just pay ICD representative for delivering your info! :)

It is essential that first ICD message is delivered and only then you could "unsubscribe" from all like this (so it must be split in categories).

It is essential to implement special protocol so you could deliver only one copy of message to ISP and then it should be sent to every interested customer. So the "unsubscribe" messages whould be handled be the local ISP!

Why ISP will ever receive those messages?! Of course because there are 1) interest from end users in receiving those messages, 2) ICD-sender company would set contract with that ISP. And it will save them money to recive only one copy and not to hire special admins for SPAM-fighting

Why company pay for it? Because it now are able to select region to deliver message to and it will not be punished for doing so. And they know how to pay for delivery!

Why end users will be happy with that? Because he will recive information he really needs! It could receive list of today TV-casts or plan of holidays or shoping discounts or (for geeks) new projects relesed.

And the last - how to beat your own complex against SPAM? :)
Just think of ICD as new system like NNTP(global delivery, regional settings, clear names) + IRC(tree delivery mechanizm, one message to every, ban/ignore server side) + multicast MBONE(subscribe/unsubscribe) + RSS feeds(news from companies) + E-mail(familiar user interface).

Are you voting for ICD? ;)
Let's convert SPAM! :)

a contract with the providers: no spam, no money.

it would be quite easy to set up companies based on guaranteed or minimised spam.

marketing strategy: save time and money not having any spam delivered to your company.

straightforward... etc.

hmm :)

Paying providers to send you extra junk? This is a**-backwards. The trend these days is that web companies are asking users to pay in order not to receive advertisements. It's obvious which way the preferences are going.

Show me any free web hosting service / free e-mail service / free whatever which asks people to pay for more advertisements.

When the ad becomes the thing you really need now, it is no more spam, it is "usefull info".

Example - you need a computer and recive an "ad" about computers from 10 companies. Then you are able to select best for you.
Or you are reciving notifications about new posts of Advogato - again it is the same "ad" but YOU need this information and you CAN unsubscribe it any time. Could you list all notification maillist you subscribed to now? (security, annonce etc) They are the same! :)
It is the best thing when people whould receive information they need.

I do not know about other countries, but in Russian (and russian speaking region) there is http://subscribe.ru/. There are 16809 mailing lists (that means one way list - you can't post there, only the owner of list could post messages to it). And 2148549 people are subscribers.

Even 10 years ago there were commercial newsgroups delivered via UUCP. You could subscribe them to recive advertizements! And they were popular, because it was really usefull info. It is way better then to search through bunch of web-sites. :)

There were people who need nice colorfull WEB - and it is created now.
There are people who need SPAM - and we should provide nice way of delivering it. Arn't you agree? :)

The people who want to send these ads will already pay the web hosts.

Ha!

But the E-mail is easie to use. And it consumes less time to find anything you need :)

You have not answerned about number of mail-lists you are on ;)

Some thoughts that chip at the realities in spam

• ICD is a wonderful idea, but why would any spam-advertiser want to use anything but spam? It's cheap, and it's blazingly effective. There were, mid-90's or so, opt-in mailing lists, they failed miserably.

• if spam was about advertising as information, they why do I never receive any spam these days other than for Viagra, Nigerian officials or some other obvious scam? I don't remember the last time I saw UCE where the C meant "commercial" ... most commercial companies instead send me Dear Info emails addressed to my website, but addressed and with a valid reply-to, and those messages are in the 1% of the load.

• filtering with fancy AI filters that log the false-positives works to a point, but it's getting so that spam attachments now seriously eat into the stupid bit-caps that the telco/cableco ISPs are inflicting. There may be places in the world where bandwidth is unlimited, but not in Ontario.

Now let's ponder some further realities and ask ...

Daddy, where does Spam come from?

Way back in the old days, spam came from improperly configured Sendmail servers; although this was fixed nearly 10 years ago (and as much as 5 years ago in the third world) there is a popular urban myth that persists on the compromised university-lab mailhost.

Spammers don't use compromised servers anymore, they use hundreds of thousands of compromised Windows boxes. Long live supercomputing! And why Windows? Because

• Windows users tend to have DSL or better connections but lack any sort of monitor ability on their bandwidth use
• Windows itself has only one security level, root
• Windows has notoriously bad track record for thinking in a security conscious way about rolling out new features. they deploy first, ask questions later

What can Microsoft do?

Microsoft doesn't want to fix spam, spam is Microsoft's best friend. They could fix spam in the next release with two reasonably doable but non-trivial changes to their O/S strategy:

1. implement a multi-tier permissions system so virii cannot infect the O/S.
2. double check the permission system so they know they actually did (1) right.

That's it, that's all. Spam would be stopped dead because it would now find the desktop as impervious to relaying as the current mail hosts. They'd then make max-protection the default, and maybe throw in their own hack-act version of iptables in each box while there's mucking about in the code ...

But Microsoft won't do that. Not a chance. They've known of this solution since at least 1995 and they haven't made a single step in that direction. It could be because organized crime buys an awful lot of computers, but I don't think so. I think the real reason may be far more sinister:

• Email continues to be the killer-app online with far more people using Email than any other single protocol. Unfortunately, Email is an open standard, meaning any vendor can implement SMTP and create an email client --- Email cannot be owned.
• Spam attacks email, threatening the one network application that 90% of net users cannot live without. It's not that they'd go back to telephone and snail-mail, they must have electronic messaging, and spam is killing email...
• Microsoft has never made any move to fix the permission system. NT has permissions, but not the desktop, leaving the average cable/dsl connected desktop as wide open as a 1992-vintage university server ... only there are millions of them. What, instead, is Microsoft's spam strategy? Glad you asked ...
• Black Penny, which Gates has announced will roll out "within two years" is proprietary electronic messaging that will be the defact standard messaging system on 90+% of the world's desktops by 2006. Just as today you can't exchange docs with a lot of MsOffice shops (OOo works but not that well and not bilaterally, and it will be worse when MS patents DOC's XML), in that near future you will not be able to exchange messages with any of your corporate clients ... not unless you have Black Penny.

QED: If Spam did not exist, it would have been necessary for Microsoft to invent it. Spam is their universe unfolding exactly as they want it.

I personally like the Remington 870 approach to fixing spam -- find the sending unit(s) and fill them full of 1 oz. 12 guage 00 shot at 1400 FPS.

Don't most modern nations have anti-spam laws on the books? Can't we, as taxpayers, get some of our legislative bodies to arise from their phat asses and kindle some enforcement of these laws? Why keep pushing technical solutions to what are basically social problems?

Does anybody here remember the mid-'80s copy protection craze? It was a daily race between the creators of copy protection (for diskettes, no less!) and the crackers of copy protection. I had one gig where there was a club of guys whose sole enjoyment in computers was breaking copy protection. Once guy was telling me he had over a thousand titles for three different platforms (MS-DOS, Apple II, Mac) but had never really used any of this stuff. He didn't want to do productive work on his box, just break copy protection schemes.

Let's examine this problem from a larger perspective. Compare junk email to junk snail mail, telemarketers, and other forms of advertising. What can be done to limit the intrusion of spammers into our daily communications? How about following up on the trail of spammers as is done on the senders of faxes? Remember, it costs money to receive a fax (paper, toner, blah blah blah). In the US it is not legal to send a fax without a return phone number enclosed. (Somebody back me up on this -- is that just commercial faxes?)

What happens if the whole civilized world starts insisting that all emails have a valid, living and breathing, non-machine-generated, return address before it is conveyed? What happens if the civilized world stops forwarding email without such return addresses and at the same time starts prosecuting the people who send such crapola? D'ya think all this might make a difference?

We need to fix spam at its source, which is the profit motive that makes scummy people willing to polute an entire communication medium just to make a couple of sales. If it costs to spam then the spam will stop. Nothing short of this will have any impact that I can see.