General-purpose rule-based mail filtering
6: Mail and Spam Filtering
Pegasus Mail can perform comprehensive automated evaluation and processing of your mail as it arrives in your mailbox. Using automated evaluation allows you to pre-sort messages into folders for later examination, detect and eliminate unwanted commercial e-mail, high-light certain messages that are important and so on. There are three separate processes that can perform automated handling of e-mail in Pegasus Mail:
1: General-purpose rule-based mail filtering Rule-based Mail Filtering is a general-pur-pose system that can be used to select messages in any folder and perform arbitrary actions on them. You can create any number of rules performing a wide range of actions, and can apply them at any time. You can have any number of general-purpose filtering rule sets; Pegasus Mail also supports special sets that are automatically applied to your new mail (either when the new mail folder is opened, or when it is closed), to your cop-ies-to-self, and another type of set that can be used to filter mail in your POP3 account before it is downloaded. General purpose filtering is not intended as a spam filter - it has a much wider range of uses and is intended for day-to-day use in sorting and arranging your mail. Pegasus Mail was the first mail program to have general-purpose mail filter-ing, way back in 1991, and its implementation is still the most powerful and useful in the industry.
2: Content control Content control is a specialized type of filtering that provides powerful content analysis processing. While originally intended as a rule-based spam filter, Con-tent control can be used anywhere you want to evaluate a message based on a detailed analysis of its content. Content control is a powerful complement to the Spamhalter Bayesian Spam filter (see below) – a combination of the two can catch very high levels of spam.
3: Spamhalter Spamhalter is a Bayesian Spam Filter developed by Lukas Gebauer of Ara-rat s.r.o in the Czech Replublic. Bayesian spam filters learn about your mail over time and can classify messages as spam or not-spam based on the mail you receive. Aimed purely at detecting spam, Spamhalter is a very easy and powerful tool that can achieve very high spam detection rates with very little effort or maintenance. When combined with the rule-based classifications provided by Content control, Spamhalter is an excep-tionally poweful solution to the problem of spam.
General-purpose rule-based mail filtering
Pegasus Mail can process mail automatically according to sets of instructions (or rules) that you provide. This feature can be very powerful – for example, it is possible to define a set of ruleswhich will pick out mail from a specific person, print it, then delete it, completely with-out your intervention. There are two ways you can create mail filtering rules in Pegasus Mail – you can use the Filtering Rule Wizard to create simple rules, or you can use the Full Filter-ing Rule Editor to harness the full power of the feature.
Using the Filtering Rule Wizard
If you want to perform simple tasks in your new mail folder, such as automating the filing of particular messages, then the Filtering Rule Wizard is an easy and intuitive way to do so.
Simply select a message you want to use as a model for creating the rule, then click the Fil-ter... button on the Folder window’s toolbar (the button is available in both Preview and
“Classic” folder display modes). Pegasus Mail will analyze the message and will then open the Filtering Rule Wizard: this Wizard has three pages, and requires you to go through four easy steps to create the new rule.
73
6: Mail and Spam Filtering General-purpose rule-based mail filtering
Step 1 – Choose the test to apply Pegasus Mail can create rules for you based on either the addresses in the message (the “From”, “To”, “Reply-to”, “Sender”, and “Return-path” headers), or on the Subject of the message. Click the radio button next to the type of test you would like Pegasus Mail to perform.
Step 2 – Choose when the rule should be applied Pegasus Mail supports two separate sets of rules for your new mail folder: the first set is applied when the folder is opened, while the second is applied when the new mail folder is closed. Normally, you will use the Wizard to create rules that are applied when the new mail folder is opened, but if you particularly want the rule applied to the folder-close set, check this control.
Step 3 – Defining the trigger condition for the rule The form this step takes depends on whether you chose to create a rule based on the addresses in the message or the subject; if you chose a rule based on the addresses, Pegasus Mail will show you all the addresses contained in the message and will allow you to select the ones you want to use to trigger the rule. You can select as many addresses as you wish, although you will typically only select one. The addresses you select will be the ones Pegasus Mail checks for when it applies the new rule to messages arriving in your new mail folder. If you chose a rule based on the subject, Pegasus Mail will present you with an editing field containing the subject line from the model mes-sage. You can either accept it as it is, or, more usually, you can edit it down to just the signif-icant part for which you wish to test.
Step 4 – Define the actions to take when the rule triggers In this step, you will choose one or more actions Pegasus Mail should take when it encounters a message matching the criteria you have chosen. The six options you can choose from allow you to:
• Move the message to a specific folder
• Set a display colour for the message
• Forward the message to another e-mail address
• Send a text file as a reply to the sender of the message
• Delete the message
• Exempt the message from further rule processing
Most of these actions are quite self-explanatory, but some combinations are not possible – for instance, you cannot both Move and Delete a message (the Wizard will not allow you to select both actions). Pegasus Mail is careful to ensure that the actions are applied in a sensible order when it creates the rule – so, if you choose to set a display colour for the message and then move it to a folder, the colour will be assigned before the message is moved. The Exempt the message from further rule processing option is a handy way of “whitelisting” particular mes-sages – it prevents any other rule in your new mail filtering rule set from being applied to the message.
When you have decided on the actions the rule should take, click the Finish button, and Pegasus Mail will create one or more rules that implement the tasks you have requested. The Filtering Rule Wizard always creates new rules at the top of the filtering rule file, so they will be applied before any other rules you may have previously created. The Rule Editor button in the final page of the Wizard’s dialog allows you to open the Full Filtering Rule Editor so you can see or adjust the rules it has created.
Using the Full Filtering Rule Editor
Using Pegasus Mail’s Full Filtering Rule Editor is a little more complex than using the Fil-tering Rule Wizard, but it gives you the greatest possible control over a very powerful facil-ity. It is also the only way you can create copy-to-self rule sets, POP3 filtering rule sets and
74
6: Mail and Spam FilteringGeneral-purpose rule-based mail filtering
rule sets that can be attached to folders other than the new mail folder. You can create and edit four quite distinct types of rule sets using the Full Editor:
• New mail filtering rules, entry and exit These are two separate sets of rules that are applied automatically to your unread mail when you open and close your new mail folder respectively. If the new mail folder is open, any new mail filtering rules you have defined for application when the folder is opened will also be applied to new incoming mail as it is picked up and added to the folder. Using new mail filtering rules approxi-mately doubles the time it will take Pegasus Mail to open or close the new mail folder.
New mail filtering rules can be accessed by clicking the Filtering rules button on the button panel, or by selecting an option from the Mail filtering rules submenu of the Tools menu. You might create entry rules to process mail you don’t usually need to see, such as confirmations of reading, or subscription requests, while you would use exit rules to clean up any mail that might be left in your new mail folder when you close it.
• Copy-self filtering rules These rules are applied automatically to any copies you make of your outgoing mail. This allows you to have your copies to self automatically filed based on the recipient, the subject, or any other criteria you care to define.
• General rule sets General rule sets can be applied to any folder or to a selection of mes-sages in a folder at any time. They are never applied automatically – only in response to your specific command. To apply a general rule set you have created, choose Apply gen-eral rule set to folder from the Mail filtering rules option on the Tools menu. You can also use general rule sets as quick actions in your folders – this especially powerful fea-ture allows you to perform complex sets of commands on a message with a single key-stroke.
• POP3 download filtering sets These rules are applied to messages in a POP3 mailbox before they are downloaded. You can, for instance, create rules that only dowload certain messages based on their address or size (very useful if you have a POP3 mailbox shared by several addresses). POP3 rules are generally limited to tests that could be applied to message headers or attributes – you typically cannot perform extensive tests on the body of the message in a POP3 rule.
To define rules, select the appropriate option from the Mail filtering rules submenu of the Tools menu, and the rule editor will open. You can also access the new mail filtering rule set applied when your new mail folder is opened directly by clicking the Filtering rules button on the program’s main toolbar.
A rule is activated when a particular condition, or trigger is met within a message. When a rule is “triggered”, the action defined in the rule is applied to the message. This process re-peats until either there are no more rules or messages, or the message is moved to a folder or deleted. You can make multiple rules apply to the same message by having them all match the same string; the rules will then be applied in the order they appear in the rule list, reading from the top down. The trigger condition can be any of the following:
• A simple textual phrase is encountered in the message headers or body
• A particular regular expression is encountered in the message headers or body
• The message date falls within (or outside) a particular range of dates
• The message is older or newer than a particular date or number of days
• The sender of the message is a member of one of your distribution lists
• The message has certain attributes (for instance, it is urgent, or has attachments)
• No condition – the rule always triggers when it is encountered.
75
6: Mail and Spam Filtering General-purpose rule-based mail filtering
For text matching within a message, you can perform two types of matching to detect the trig-ger text – simple header matching, where Pegasus Mail looks for the trigtrig-ger text in selected common headers in the message; and regular expression matching, which allows you to set up complex pattern-matching criteria in the message headers only, or in the message headers and message body, or only in the message body.
To create a rule, highlight the item in the rule list before which you want the new rule to ap-pear, then click on the Add rule button. The rule type selector window will open, in which you tell Pegasus Mail what type of rule you want to create. The following types of rule can be created; note that all rule types have an option to trigger if the condition they define is not met - this can be useful way of handling exceptional conditions. Similarly, all rules can be disabled, which is a handy way of preventing them from taking effect temporarily if you don’t want to delete them.
Standard header match (the “Headers” button) creates a rule which simply matches text in any of a set of predefined message headers. Simply click the controls representing the fields you want Pegasus Mail to check for the trigger text. You can check more than one control if you want Pegasus Mail to examine multiple fields. So, if you want Pegasus Mail to check in both the From and To fields for a string, you should check both controls. Pegasus Mail nor-mally searches for the text you enter anywhere in the header, so if you enter “bed”, the rule would trigger on “bed”, “tabbed”, “albedo”, or any word containing “bed”. If you want the rule to trigger only when the field matches the trigger text exactly, check the control labelled Message field must match this text exactly. So, if this control were checked and the trigger text were “Subscribe”, then the text “Please subscribe me” would not cause the rule to trigger.
The trigger text is not case-sensitive, so “SUBSCRIBE” and “Subscribe” would be regarded as a match.
Regular expression match (the “Expression” button) creates a rule that uses an special pat-tern called a regular expression to match lines in the messages. The scope controls specify in which parts of the message Pegasus Mail should try to match your expression: you can have your expression checked against only the message headers, only the message body, or against the entire message. Matching against the entire message or against the message body can slow down the process of opening the new mail folder dramatically – performance is affected by having a single rule which does a message body check, although subsequent rules will not slow the process down further. The trigger text for a regular expression rule contains the text or expression Pegasus Mail should attempt to find in your message. The text is always case-insensitive – so to Pegasus Mail, NOVELL@suvm is the same as Novell@SUVM. The text must be an expression beginning at the start of a line. Your regular expression can contain the following special characters for pattern matching:
Sets of characters can be entered literally – for example [abcd1234], or you can specify ranges of characters using a hyphen, like this: [a-d1-4]. If you need to search for a literal occurrence of a special character, you must enter it as a set expression – so, to search for an asterisk, you would enter [*]. Remember that regular expressions begin at the start of a line, so if you want to match text anywhere in a line, the first character in the expression must be a *.
* Matches any number of characters
? Matches any single character (but not zero characters) [ ] Defines a set of characters to match (see below).
+ Matches any number of recurrences of the last character or pattern that was matched.
76
6: Mail and Spam FilteringGeneral-purpose rule-based mail filtering
Message size creates a rule that triggers if the message is smaller or larger than a size you specify.
Message date creates a rule that compares the date of the message with a pair of dates or ages you supply. You can specify that the rule should trigger if the message falls within a particular range of dates, or has an age within a certain number of days. Dates are entered in the form YY (year) MM (month, January being 1), DD (day, 1-31), hh (hour, 0-23), mm (minute, 0-59) ss (second, 0-59). You can enter the dates in any order you wish - Pegasus Mail arranges them in proper chronological order internally. Conditions involving “Year 2000” comparisons are handled correctly. Message date rules are commonly used in general rule sets as a way of se-lecting messages that fall within a particular period using the Select message action. Message date rules always use the Date header from the message with any timezone correction ap-plied.
Message age creates a rule that triggers if the Date field from the message is older than a particular date or a certain number of days. Dates are entered using the same format as in Message date rules. This type of rule is commonly used in general rule sets as a way of purg-ing messages older than a certain age from mail folders.
Message colour Allows you to filter based on any colour that has been applied to a message using the folder browser or message reader Set colour command. This type of rule is gener-ally only useful in general rule sets that are applied to folders other than the new mail folder.
Attachment filtering This button creates a special type of rule that checks the filename and/
or extension of attachments to messages. The rule is special because it will always trigger if the attachment matches the conditions you specify. There are two specialized actions that are only available in attachment filtering rules – one that deletes the attachment from the mes-sage, and another that saves the attachment to a file.
Attributes Creates a rule that tests certain physical characteristics of a message, such as whether it has attachments, whether it has been marked urgent and so on.
List membership scan (the “Scan list” button) creates a rule that triggers if the sender of the message is a member of a specified distribution list (see the section later in this manual for more information on distribution lists). You can use this type of rule to control access to your mailing lists (for instance, by allowing only list members to post mail to a list); you can also use it to control spam, or unsolicited commercial e-mail: when you receive an unwanted mes-sage, add the sender to a “killfile” distribution list using the right-click options in the message reader or folder browser, then use this rule to delete or move all future mail from that address.
Rule always triggers creates a rule that has no conditions and always triggers. You will most commonly use this type of rule in conjunction with flow control actions (see later in this sec-tion).
Comment simply adds a textual comment to the rule set. Use this to remind yourself of why
Comment simply adds a textual comment to the rule set. Use this to remind yourself of why