lotus



previous page: How SPAM Finds you: Directory Harvest Attacks (DHA)
  
page up: Anti-SPAM, Anti-Phishing and Anti-Viruses Techniques
  
next page: Anti-SPAM Techniques: Bayesian Content Filtering

Anti-SPAM Techniques: Heuristic/Signature-based Content Filtering




Description

This article is a part of the series on undesired email (spam, phishing, viruses, etc.). The material covers the Poisons and the Remedies.

By Stas Bekman.

Published: May 15th 2006

Anti-SPAM Techniques: Heuristic/Signature-based Content Filtering

The majority of anti-spam email companies do Heuristic (also known as signature-based) content filtering. A typical product receives the message from the client, saves it to a disk, then it tries to apply a variety of checks trying to match certain patterns. A score is assigned based on how well the pattern was matched, or how many patterns were matched. The higher the score the higher is the possibility that the email is a undesired email. It's up to a user to decide at what scores a given email should be dumped, saved to a spam-maybe folder (quarantine) or delivered to an INBOX.

One of the techniques used to assign a score is a pattern signature. This approach is that it requires a special lab with spamtraps, so that they can attract as much undesired email as possible. Next there is a need for humans armed with some analysing tools, who go over the undesired email and extract signatures which are then distributed to the customers (usually via very frequent product domains).

The main disadvantage of this approach is the huge delay between the spam or virus or phishing outbreak and the time the customers receive a signature update, which will allow to filter out those unwanted messages. And of course it's a disadvantage to the anti-spam company, since they have to employ lots of people to take care of this semi-manual labour-intense task.

Another disadvantage is those signatures aren't perfect, i.e. they may assign a high SPAM-email score to a totally legitimate email, what's known as a false-positive.

Vendors

Here are some vendors supporting this technique (including open-source solutions):

Kaspersky Internet Security (http://www.kaspersky.com)
(Commercial) and its other products use bayesian-based filtering.

trimMail Inbox (http://www.trimmail.com/)
(Commercial) an easy, powerful, affordable way to protect your mail servers from SPAM, viruses, dictionary attacks, and other hazards of the internet

Sophos PureMessage (http://www.sophos.com/products/)
(Commercial) performs statistical analysis too.

SpamAssassin (http://spamassassin.apache.org/)
(OSS) is a mail filter which attempts to identify spam using a variety of mechanisms including text analysis, Bayesian filtering, DNS blocklists, and collaborative filtering databases.

Brightmail (http://www.brightmail.com/)
(Commercial) It was acquired by Symantec

Postini (http://www.postini.com/)
(Commercial) provides a powerful combination of patented content management and sophisticated content analysis to block viruses, spam, phishing, IM worms, directory harvest attacks and other email and IM threats.

 

MailFrontier (http://www.mailfrontier.com/)
(Commercial) anti-SPAM, anti-phishing, anti-virus

McAfee SpamKiller (http://www.mcafee.com/)
(Commercial) provides Bayesian filtering technology

CipherTrust (http://www.ciphertrust.com/)
(Commercial) inbound/outbound protection


 

 

Please notify me if you know of others.

Related Links

And here are some pointers for additional information on the subject:

 

Death to SPAM (http://www.mindworkshop.com/alchemy/nospam.html)
A guide to dealing with unwanted e-mail

Heuristic Filtering
at Wikipedia

Stopping Spam and Malware with Open Source (http://www.brettglass.com/spam/paper.html)
Presented at the O'Reilly Open Source Convention on July 27, 2001

Heuristic Spam Filters (http://www.emailcash.com/heuristic-spam-filters.html)

Five Companies to Squash Your Spam (http://www.darwinmag.com/read/020104/spam.html)
The Radicati Group took a long hard look at spam-slayers and came up with this quintet of conquerors (Feb 2004)


 

 

Continue reading about other Remedies or jump to the email-related Poisons section.















TOP
previous page: How SPAM Finds you: Directory Harvest Attacks (DHA)
  
page up: Anti-SPAM, Anti-Phishing and Anti-Viruses Techniques
  
next page: Anti-SPAM Techniques: Bayesian Content Filtering