How the blacklist works

One of the most commonly used techniques (alone or in combination with other techniques) in the fight against spam is address-based filtering. The principle is that hosts considered to be sources of spam are entered into databases (the blacklist) that are accessed by mail servers. It’s then up to the mail server to decide what to do with the mail from these hosts.

Long gone are the days when spammers used their own machines to send their e-mails. It was then easy for the recipient to blacklist the sender’s IP, or even ask his ISP to delete his account. To avoid blacklisting their own IP, but also to save money (bandwidth and processors are expensive), spammers started using open relays, hosts that agree to send mail from any sender. Over time, as the number of open relays dwindled, they used malware to take control of ordinary people’s computers and use them as mail servers. They therefore potentially have their hands on a vast number of hosts, and it is no longer practical for a single user (even a large organization) to keep an up-to-date blacklist of them.

There are therefore centralized databases of blacklists that can be accessed via a DNS query, hence their name DNSBL (Domain Name System Black Lists). The term RBL (Realtime Blackhole List) is sometimes used, after the first public blackhole list in 1997, but it is a trademarked term and is therefore less and less used. DNS is a practical, fast and flexible protocol for allowing geographically distributed clients (in this case, MTAs) to test whether a particular host is blacklisted. For example, the query to determine whether host 1.2.3.4 is listed is fairly straightforward: the MTA performs a DNS lookup of record A of name 4.3.2.1.liste-noire, in the same way as reverse lookups (in-addr.arpa).

The presence of an IP in a blacklist does not theoretically mean that emails from the host are systematically refused. It is up to the MTA to decide what policy to adopt accordingly: carry out one or more additional checks with other blacklist sources, mark the mail as SPAM and send it to the recipient, transfer the mail to a dedicated junk folder or refuse it. In practice, however, it’s simpler for the MTA to reject it immediately: this saves precious resources.

There is no exhaustive list of DNSBLs. Anyone can compile a blacklist, provided they control a DNS zone. Numerous lists appear (and disappear) every day. However, DNSBLs can be classified according to two criteria. The first criterion is the purpose of the blacklist(compliance with RFCsThe second criterion is the modality (some would say the aggressiveness) by which a host is added to the blacklist: proactive (following tests to find open relays), passive (added only when the host sends mails to spam traps), manual (with host verification before being added to the blacklist).

Are blacklists effective in the fight against spam? The answer is undoubtedly positive. In the face of the ever-increasing resources mobilized by spammers (e.g. herds of zombies), they represent a response that is both economical and simple for recipients. However, we mustn’t overlook the fact that, like any system, a blacklist is subject to failures, not least the addition of legitimate hosts that don’t spam immediately at the slightest sign of suspicion, as some very aggressive lists do. In a professional environment, the risk is that important mail could be lost from a machine that simply doesn’t comply with RFCs, for example, or that has been compromised from time to time. It is therefore important to implement a well-thought-out strategy for dealing with blacklisted e-mails, and not to systematically and blindly reject or delete every e-mail from a blacklisted IP.