Possible Filtering Features

This is an exhaustive list of filter features that an email program could have. The OSAF team will eventually prioritize this list to determine which features should be in which releases (if any). Please also note that in Chandler, it might make sense to do these things with agents instead of the traditional filtering framework.

Note that this page covers very "traditional" filters, which mostly have to do with explaining to the email client what your rules are for how to classify messages. There is a new and exciting area of automatic filtering using Bayesian probabilistic filtering algorithms which may prove superior to these procedural algorithms.

There also need to be "hooks" for anti-virus plug-ins.

Filter Management

Possible things a client might let you do with filters:

• Share them
  • Note: sharing is easier with a few macros (e.g. "me" and "myDomain")
• Version them
• Name them
• Name sets of them (for easy block import/export and/or enabling/disabling)
• see how often/frequently/recently each of them were triggered

A client might come with some filters pre-loaded (although perhaps not pre-enabled) including:

• junk mail filters
• auto-highlighting
• "vacation" messages implemented w/filters (that are smart enough to not send vacation messages to mailing lists!)
• "attach" in outbound message with no attachment gives warning

A user might also want to see

• which messages trigger a specific filter
• which filters are triggered by a specific message

There are several points at which filters are evaluated. It will be necessary either to place filters in different sets or to allow filters to determine which case is in effect. This list is not yet exhaustive:

• When first receiving an incoming message
• When message(s) are manually filtered
• When replying/forwarding message(s)
• When a draft message is created
• When a message is sent
• When a message attribute is changed, e.g.,
  • Marked read
  • Marked done ("dismissed")
  • Marked as spam

Filter Conditions and/or Exceptions

• Boolean combinations of filters
• if header header {contains, is, starts with, ends with} regexp
  • regexp in any receiver
  • regexp in sender's "friendly name" (e.g. "Mabel Garcia")
  • regexp in sender's email address (e.g. "mabel@flossrecycling.com")
• if body {contains, is, starts with, ends with} regexp
  • regexp in plain-text body
  • regexp in raw (HTML, RTF) body
• if body contains script (usually Javascript)
• if body contains image
  • can be mostly implemented as "if HTML body contains IMG"
• if any attachment name {contains, is, starts with, ends with} regexp
  • e.g. "if any attachment name ends with .exe"
• if attachment is executable
• if sender is in address book addressBook
  • e.g. "if sender is in 'Friends' address book"
• if attribute score {=, =, <, > } number
  • e.g. "if spamScore > 300"
• if language {is, is not} language
• if character set {is, is not} language
• if messages was received via account account
• if sender is in group nickname/alias alias
• if List-ID is not on mailing list registry
• if I've corresponded with sender in past number days (useful in vacation messages)
• if sender is on blacklist
• if message digital fingerprint is on blacklist -- useful for collaborative spam filtering
• if message passes/fails external plug-in script
  • scripts could include e.g. checking for "postage" or digital fingerprints
• if message is a response to a message you sent
  • You might want to have a different notification or color coding or prioritization for messages that are responses to messages. As the number of messages that you've sent gets larger, this will become more computationally expensive, so perhaps a good compromise would be "response to recent message" or "response to a specially flagged message". (Greg Noel points out that this should be no more expensive than threading, which could already have been done.)
  • This feature would make it easier to marry bounce messages to messages that you sent.
• if message (matches/doesn't match/can't tell) corpus corpus (see ProbabilisticFiltering)
  • for spam filtering
  • for automatic classification of messages
• if date in header header is {before/after} {relative/absolute} date (e.g, before three days ago, after 9/11/2001) (note that there are headers other than Date that contain interesting dates, such as Expires and Reply-By)

Filter Actions

• do/do not download message body
• change attribute attribute to value, e.g.
  • change Status to read/unread
  • change Category to Friends
  • change color to red
• prepend phrase to Subject line
• notify the user
• auto-respond with response template templateName
• auto-forward to address optionally with response template templateName
• open list of messages with saved View savedView (e.g. if message is from OSAF Design mailing list, open View that shows all OSAF Design mailing list messages)
• delete
• auto-suggest template templateName as a possible response
• add/subtract number points from attribute score
• quarantine/delete attachments
• move attachment to folder folder, overwriting same name
• move attachment to folder folder, prompting for new name if duplicate
• normalize (scale) message's received-importance-level based on sender's received-importance-level history
• prompt (for taking further actions) with text promptText
• use message to add a mailing list to your personal mailing list subscriptions registry
• add sender to group nickname/alias groupNickname
• add sender to address book addressBook
• execute built-in action action (e.g., Reply-to-sender, Dismiss-message, ...)
• execute script script
• burst message (i.e. split digests up into constituent messages)
• redirct with template template

External References

Web sites:

• Camram - digital postage
• Habeas - using copyright to whitelist
• http://www.wecanstopspam.org - using spam-fighting information to whitelist
• SpamNet aka Vipul's Razor - collaborative filtering with digital fingerprintints

Mailing list messages:

• different criteria for incoming and outgoing messages by Philip Trauring
• filters: req for filter action of "set category" by DuckySherwood
• filters: filter on raw/stripped text switch by DuckySherwood
• request: "if corresponded with recently" filter condition by DuckySherwood
• Rules Discovery by Mark Nielsen (about pre-loaded filters)
• Vista feedback -- relationship between sharing & act ions? by Chris Allum, quoting Charlie Joynt (about outgoing filters)
• Outbound rules by Joel Finkle
• general:import/export of filters by DuckySherwood
• filters: Subject to automation by Bruce Dykes (about prepending phrase to Subject line)
• filters: Subject to automation by Ray Ryan (discussing composition-time filters)
• Ray Ryan suggests that there be a condition "if user modified message attributes".
• filters: req for filter condition "is in address book X" by DuckySherwood
• view/filters: visually distinguish based on addressing type by DuckySherwood and Ed's rebuttal
• Scripting and Customization by Arnulv Rudland (about pre-installed filters)
• Arnulv Rudland suggests that Chandler add unfamiliar List-IDs to mailing list manager
• mailing list manager: intelligent digest parsing by Bruce Dykes and Kurt Hibbs' addition
• request for fine control over new-message alerts by DuckySherwood
• filters: request "suggest template" filter action by DuckySherwood
• DanG^? tells why he wants the "do/do not download messages body" filter action
• Jungle.WhereIsThatEmailScenario by DennisLynch about filters which sort email into virtual folders
• BrianWilson^? gives the preliminaries for a filtering plug-in API
• PhilipTrauring requests recognizing a response to a message the user sent
• PhilipTrauring requests outgoing filters
• BruceDykes^? asks for the "received via account account" filter condition
• filtering: request for "what filters trigger are triggered by this message?" by DuckySherwood

Contributors

• DuckySherwood - 8 Jan 2003
• ChaoLam - 10 Jan 2003

---

There is another page, EmailFilters, that overlaps with this one.

-- GregNoel - 03 Mar 2003

Yes, somewhat -- but the EmailFilters page is not designed to be a comprehensive list. Yes, it does have a competetive analysis section, but that's different.

-- DuckySherwood - 04 Mar 2003

My point was that EmailFilters lists some capabilities that aren't given here as features. I've brought over a few, but I don't think I got them all.

-- GregNoel - 05 Mar 2003