Bayesian filters or Bayesian method

How do Bayesian filters work?

Bayesian filters work by analyzing email content and assigning probabilities to certain characteristics or keywords that may be associated with spam or legitimate mail. Their operation can be summarized in a few steps:

1. Filter training: Initially, the filter is trained using a set of known email samples, both spam and legitimate mail. These samples serve as the basis for establishing initial probabilities.
2. Assigning weights: The filter assigns weights to various email characteristics, such as keywords, headers, sender addresses and so on. These weights are based on the probabilities calculated in the training step.
3. Probability calculation: When a new email arrives, the filter calculates the probability of it being spam or legitimate mail by combining the probabilities associated with its characteristics. A calculation formula based on Bayes’ theorem is used to obtain an overall probability.
4. Classification: Depending on the calculated probability, the email is classified as spam or legitimate mail. A predefined threshold can be used to determine the cut-off point between the two categories.
5. Continuous improvement: Bayesian filters improve over time by adjusting the weights assigned to features based on feedback and manual corrections made by users.

Bayesian filters are effective for filtering spam, as they adapt to new spam trends by updating their probabilities and weights. However, they can also produce false-positives or false-negatives, so other email security techniques are needed to complement their operation.

Examples

Two databases are created, one for spam and one for hams (legitimate messages). Through a learning phase, a dictionary of keywords is created in which each term is associated with a probability. For example: viagra 100%, security 20%, messaging 10% and free 60%.then, when analyzing an e-mail, if the words in the lexicon exist, the sum of the probabilities of each keyword found is assigned to the e-mail. Following our example, if an e-mail contains the words “free mail server security”, the e-mail will obtain a score of: (20% + 10% + 60%) / 3, i.e.: 30%. This is a legitimate message, since the score is less than 50%. With a large number of spam and hams, this technique provides very interesting analysis results.

Applications

Most e-mail clients with built-in anti-spam features (Thunderbird, Outlook, etc.) use Bayesian filters almost exclusively. In ALTOSPAM, Bayesian filters are among the 15 technologies used. Depending on the score obtained (between 0 and 100%), the e-mail is more likely to be classified as spam or ham.

Further information

• Wikipedia article on Bayesian filters
• Original article popularizing Bayesian methods in antispam filtering
• Overview of antispam technologies