Spam Tagging of E-mail for DPMMS

Email reaches DPMMS via the university mail system. As described in Computing Service Newsletter 216 and the Central Email Scanner page, the university is now putting a "tag" on each mail message, indicating how likely it is to be spam (that is, unsolicited and unwanted mail). The spam is not blocked by the university, but instead users can themselves easily arrange for appropriate action to be taken automatically, depending on the tag.

The tag is a real number (typically in the range -5 to 15). Values greater than zero indicate probable spam, the size of the number reflecting the strength of belief. Certainty is impossible, of course; for all the university knows, you might actually have a rich friend in Nigeria wanting to send you millions of dollars. A score of 10 or more is regarded by the university as essentially certain, but messages with much smaller scores are still likely to be spam.

The tag is added to the message by the insertion of extra header lines. Most mail programs have an option to allow you to see the header lines in a message; for example, in pine, pressing the "h" key makes them visible. Here is an example of lines that you might see:

X-Cam-ScannerAdmin: mail-scanner-support@ucs.cam.ac.uk
X-Cam-AntiVirus: Not scanned
X-Cam-SpamDetails: not spam, SpamAssassin (score=5.1, required 10,
        GAPPY_SUBJECT, HTML_60_70, HTML_IMAGE_ONLY_02, HTML_MESSAGE,
        MIME_HTML_ONLY, PLING_PLING, REMOVE_PAGE, SUBJ_FREE_CAP)
X-Cam-SpamScore: sssss

The crucial lines are the third and fourth. The third says the message scored 5.1, giving reasons why this score was reached (eg the message was written in HTML, a typical spam feature). The phrase "not spam" means only that the score was less than 10; this message is still very likely to be spam. The fourth line is the important one for filtering purposes: it gives a string of letter "s"s whose length is the score. This line is easily checkable by filtering programs, which can then take appropriate action. to record what it found.

Making a spam filter

If you have a file called .forward in your home directory with the lines

    # Exim filter
    if $h_X-Cam-SpamScore contains ssssss then
    	save Mail/spam
    	seen finish
    endif

then all mail with a score of 6 or more will be placed in a folder called "spam". The easiest way to make such a file is to copy the DPMMS default .forward file to your home directory:

         cp /usr/dpmms/adm/init/forward/Handout ~/.forward
         chmod 600 ~/.forward

This will mean that messages tagged as spam will be moved to your mail folder "spam". The chmod 600 ensures that you can write to the file and that no one else can read it.

Scoring - when to reject ?

As mentioned above the number of esses in the spam rule determines what is filtered into the same directory; more esses in the rule means that fewer messages will be considered to be spam. The rule given above and in the default ~/.forward file has six esses. At this level I haven't seen any real mail end up in the spam folder (this is not a guarantee) but quite a lot of spam is missed. Reduce the number of esses if you want to catch more spam; this will increase the risk of real mail going into the spam folder. Three esses is fairly safe and kills most spam. Note, though, that genuine email is likely to score at least one if it is in HTML format (as is typical with, say, mail sent from a hotmail account). Likewise, mail from a mailing list can score positively under some circumstances.

Please check through your spam folder regularly

Please check through your spam folder regularly, both to stop it from filling up our disks with spam, and because there will typically be one genuine message in this folder. I've seen quotes that this sort of spam-tagging is more than 99% reliable; that of course means that if you use this method, for every hundred emails in the spam folder, there will be one genuine email that you wished to receive.

What if you want to forward your mail elsewhere?

If you already have a file ~/.forward, and it does not start with # Exim then you will have to translate this into the exim filter language if you wish to filter spam. You can probable do this by replacing M.Name@remote.site with your forwarding address in one of the examples in the default .forward file and removing the ##s to enable the filter rule. Use the version with seen unless you want to keep a copy of the mail here as well as sending the mail on to your new address. You can find out how to write more general filters from the Exim filter manual (direct link to the current version).

Accessing DPMMS email from elsewhere

If you read your DPMMS email on a windows machine, or from outside the department, you may find that your email is collected by IMAP. IMAP will/should allow you to see the spam folder, and you can access it like any other DPMMS mail folder.
If anyone can suggest instructions on getting Eudora or Outlook/Outlook Express to do this, please let me know, and I will add them here.

What marks a message as spam ?

The computing service use Spam Assassin (master site, UK mirror) to add header tags to each email. The CS customize their tests very slightly, but they are almost the same as the ones listed at http://www.spamassassin.org/tests.html. If you set your mail viewer (pine, Eudora, etc.) to show you all the email headers, you will see the names of the tests that an email triggered in the X-Cam-SpamDetails line.