Realtime Worm Filtering System

Posted 13 Feb 2001 at 20:36 UTC by Waldo Share This

Microsoft Outlook worms have been appearing with alarming regularity, most notably Melissa in March of 2000, and Love Letter in May of the same year. Now comes the AnnaKournikova worm, though it's yet to be seen how much trouble that will cause. Educating users ("Please, by all that's holy, don't open VBS attachments!") hasn't made an appreciable difference. Perhaps it's time for something a little beefier -- a realtime worm filtering system.

Groups like ORBS and MAPS are quite contentious, and certainly politically-charged. But they've both proved to be solid methods of limiting spam, and are popular as a result. Perhaps these services could be extended, or a new service could be created, that would exist solely to put a stop to worms.

Thus far, worm outbreaks have been few and far between, but the damage has been significant. According to one CNN story, the damage from Love Letter may have reached US$1,000,000,000. User education hasn't been very successful, because these worms inevitably spread from friend-to-friend, via address books. So people open the attachments, as they're from somebody that they trust. Virus scanners can help immensely, but updating dozens, hundreds, or even thousands of systems in order to innoculate a single business or organization is hardly practical. Server-based virus scanners exist, but they're generally expensive, non-Unix-friendly and blackbox.

Those of us that run Sendmail are familiar with the rulesets that can be used to filter out particular subject lines. When some new worm rears its ugly little head, the first thing that I do is update the .mc files on all of my servers. And when a new strain appears a few days later, with a new subject line, I update them again. This method is somewhat time-intensive, so many postmasters don't bother to go through with it. So the worm continues to spread, the number of infections increases geometrically, and the costs creep up.

But what if we had a realtime method of blocking these worms? A lookup system, like MAPS or ORBS, with which a mail server could compare a subject line, attachment name and size, and other distinguishing characteristics of a message. With a few basic metrics, a worm could be identified and the message could be rejected. It's an extremely simple system; that's what makes it feasible. A system of this nature could drastically reduce the impact of worms, and, perhaps more important, reduce the financial losses that inevitably result.

The Achilles heel of this plan is that it could easily become a victim of its own success. Much like real-life virus innoculations, widespread use of this system may result in more polymorphic worms, such that they become considerably harder to identify and thusly compounding the existing problem.

However, the same was said of anti-virus software in the early 90s, and the same has been said of spam filters. And, to some extent, that has proved to be true. Viruses have gotten trickier and spammers have gotten nastier. But does this mean that anti-virus software and spam filters were a bad idea? Probably not.

My question to the community is this: Is this a Good Thing(tm)? Could it work?

Doesn't this already exist?, posted 13 Feb 2001 at 20:55 UTC by logic » (Journeyer)

Doesn't this already exist in the form of anti-virus companies and automatic filter updates? Several firewall vendors support pre-screening of email through a vendor-delivered filtering backend, and most mail server backends support some form of bolting up to an existing virus-checking system. So, how does this proposal differ from the current scheme, other than reduce the cost to the administrator?

A Few Benefits, posted 13 Feb 2001 at 22:26 UTC by Waldo » (Journeyer)

I think one of the biggest advantages of a system of this nature is that it would save a lot of system overhead. I remember back when I ran a BBS in the early 90s, it was a little overwhelming for my system (an 8086) to unzip and virus-scan every upload. Though we've obviously seen a considerably advance in processor power, I still shudder at the thought of unzipping, untarring, ungzipping, un-Stuffing, uudecoding, etc., attachments to all incoming mail to run a full virus scan on every one. Some of my mail servers, I imagine, would gurgle, choke, and then die a horrible death at the hands of such abuse. (The POP and IMAP users, on the other hand, would be positively sterile, which I love the idea of.)

There's also -- as you pointed out, logic -- the issue of cost. Free beer is good. Then there's free speech -- I also prefer not to rely on a corporation for black-box virus/worm updates. Given Symantec's patent on virus updates, I'm a little skiddish about the whole anti-virus industry. A community effort is, to me, quite preferable.

It probably makes sense to start with static filtering, posted 14 Feb 2001 at 12:42 UTC by ztf » (Apprentice)

I did some quick research on this yesterday, since I had some users who are [too trusting | too desperate for pictures of tennis stars ]. Warning: I am not a sendmail guru, and procmail is on my list of "ought to learn this someday."

My first thought was that this sounds like something sendmail ought to handle, right? There ought to be some line-noise-like command in that will filter on MIME type. Unfortunately, this turns out not to be true -- sendmail's classic behavior is to concentrate on delivery, not content. A content-filtering API (libmilter) was added in sendmail 8.10, and the API was changed in 8.11. While this capability now exists, many mail systems are still on 8.9 and earlier and will not be able to take advantage of sendmail filtering. Also, this is simply an API, and so relies on other offerings like vbsfilter to actually perform the filtering.

The alternative solution is to set up procmail as the local mailer, and do this filtering via procmail rules. The most mature-seeming procmail recipe for this that I was able to find is the Procmail E-Mail Sanitizer, which "defangs" files based on a list of executable types [*.vbs, *.exe, *.dll, etc.], plus some well-know .zip worms. It also claims to have some rudimentary support for scanning Microsoft Office files for macro virii. I hope to give this a try Real Soon Now(tm).

As for Waldo's concern about mailserver load, the beauty of the E-Mail Sanitizer approach is that it offloads virus scanning onto the PC client, and if no one sends files in Microsoft Office formats, the macro virus scanning will never be invoked either. :^)

Does anyone have experience (good or bad) with the sendmail/procmail combinaition, or the E-Mail Sanitizer in particular?

Rope, nooses, and autohanging., posted 14 Feb 2001 at 18:25 UTC by duff » (Journeyer)

We already have a real-time worm filtering system: people. The problem is one of education. People should know better than to have any sort of automatic action triggered by reading their email. Mail client writers should know better than to have default actions for certain types of email. Mail client writers should also provide safety mechanisms that shield users from potential danger.

Also, there's good 'old signing. If that were to become more widespread then people wouldn't be fooled into trusting an email just because it appears to come from someone they know.

All I'm saying is solve the problem, not the symptom.

Behavioral Analysis, posted 14 Feb 2001 at 19:01 UTC by happybob » (Journeyer)

At my job, we do behavioral analysis measurement of running software.

One of the things that I've learned from the measurement tools we've built, and from measuring software's behavior as it runs, is that the problem of email worms can be avoided.

When a person is using Outlook (or whatever email client), the program doesn't normally start shipping messages out to everybody in the addressbook.

Recognizing and stopping this type of anomaly is really pretty easy. But, people don't build software with the sensors necessary to observe the actual internal behavior of the system. Physical systems are built this way (strain gauges, pressure sensors, temp sensors, etc.), and the important ones (airplanes, satelites, HVAC, etc.) are able to respond somewhat gracefully to abnormal behavior. Software on the other hand, just tends to make a mess of its house and home.

How would digital signatures help?, posted 14 Feb 2001 at 21:35 UTC by ztf » (Apprentice)

duff: How would digital signatures help? After all, a worm of this type is sending mail as the user, so there's no reason the generated spam wouldn't have a perfectly valid digital signature attached.

Couple Quick Thoughts, posted 15 Feb 2001 at 05:23 UTC by Johnath » (Journeyer)

ztf: I think duff was implying that people getting into the habit of digitally signing - with a password-protected signing key - would dampen these worms since the mail a worm would generate (unless it felt like asking for your password) would not be signed.

Waldo - I think your point about antivirus software - how the threat of polymorphic viruses as a result of a/v software was vastly overestimated - is a very good one, but I don't know how well it applies to email worms. Look at Anna, it appears to have been generated by a kiddie who couldn't even code VBS and auto-generated the script. To have such an auto-generator alter topics, content, and whitespace in the script attachment would be trivial, and would defeat most simple attribute-matching heuristics, in addition to filling any such databases with a lot of noise. Seems we are better, with md5 sums and such, at detecting change than at detecting consistency.

Instead, I would agree with happybob's assessment - this oughta be handled by the client software, because it should perceive there to be a problem when it starts spontaneously spewing dozens of messages. Even simple heuristics at the client level might solve a lot of problems by simply limiting the massive email floods that cripple everyone when one of these comes along.

Anyhow, just some random Dr. Pepper inspired thoughts. Good article.

Education is the key, posted 17 Feb 2001 at 03:59 UTC by jmg » (Master)

duff is correct, we need to nip this is the bud with education, but there are parts of MS that could improve in this department.

One would be making sure that the exensiont may not be what they seem. When it's a vbs script file say, this script could be mallicious, it could format your hard drive, are you sure you want to run it?" And make users take a more active roll. Part of this whole problem was because microsoft didn't quite decided how to parse the filement for mime-type autolaunch things.

Though, I guess I'm special in that I don't interact with many people who are on windows boxes, and those that are, use Netscape. I have yet to recieve a single one sent directly to me! I wonder if someone's going to send me one just to be funny. :)

It really does come to more strick warmnings and user education.

*shivers and thinks about when you can have vbs auto sign your emails for you because you didn't set a passphrase on it*

Why do anything about it?, posted 21 Feb 2001 at 15:58 UTC by Fefe » (Master)

If there is a lesson that can be learned from computer security, it is that you can't solve social problems technically. If people chose to use broken software with inherent virus and worm problems, you can't to anything about it.

Trying is a waste of time. The problem will solve itself in time.

Outlook already costs the world more money than web defacements, break-ins to credit card databases and other hazards. Companies that are too dense to see the real problem (Outlook) will go bankrupt. The market will handle it. This is not an education issue. There is no reason why people should not be allowed to clock on attachments. Allowing that is the function of email clients, after all!

May Outlook rest in peaces.

Outlook worms *are* a technical problem., posted 27 Feb 2001 at 14:25 UTC by argent » (Master)

Outlook has so many mechanisms for running untrusted code because it's displaying messages using the same engine that display folders on the desktop. Not just the interpreter, the whole thing, active content handlers and all. Then to "protect" you it filters out certain types of content... now then, does anyone run a "default allow" firewall on their network? No?

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page