About
The title
The concept
Decency is an all-in-one anti SPAM solution.
The general idea is to evaluate the probability of a mail being SPAM or HAM by applying multiple results from multiple vectors - each providing a scoring which will be cumulated until a defined threshold has been reached.
Decency is designed to be an extendable middle-ware between multiple existing third party filters (virus, SPAM), but also implements it's own filters and policies, following a strict modular design.
To achieve the most accurate decisions, the result of each component (wheter it is a single module in a Server or the result of a server, accounted by another server) is known at any point of time by any component.
Furthermore, Decency can be deployed in large distributed structures or on a single mailsystem - whatever you need.
The structure
Decency is implemented in three different servers:
- The Doorman - policy handling
- The Detective - content filtering
- The Defender - meta server, provides before-queue proxy or milter and uses both servers above
A normal setup would consist of either a Doorman and a Detective server or a Defender server implementing both. You can also just use the Doorman or the Detective without the other. Or you can build a "frontend" server, using Doorman and multiple "backend" servers in a row implementing Detective servers.
All actual logic (defending SPAM) is implemented in form of modules. The servers just give them a framework to work in and define behavior, such as when to reject a mail (threshold), what to do with virus infected mails and so on.
Why another anti-SPAM solution?
The main is reason is, that i was not satisfied with the existing open source solutions. Until now, you had to combine the results of a variety of separate, independent solutions manually to achieve a decent result. Decency intends to change this. All components and servers work together in a harmonic symphony, which only can be achieved, because they are "aware" of the bigger picture. For admins this means easier, because consistent configurations, clear guidelines and thus less mistakes and better understanding of what is going on. For end-users, this means less SPAM and less false positives.
The other reasons can be mainly attributed to my play instinct dictating me to try this and the needs of our hosting company • fortrabbit, in which we needed a more coherent solution than the patchwork of this and that we used before.
Is it "postfix only"?
No, or at least it does not has to be. For now, mainly Postfix is supported, but most other MTAs should be able to implement the Detective. Exim should be able to use the Doorman (how-to can be found in the Doorman section ).
Especially with the Defender, providing a milter interface, you can use Decency with probably most mailservers out there.
What about qpsmtpd?
Yes, there is qpsmtpd, which does implement the above mentioned techniques (content filtering, policy server) and is also open source. However the approach is entirely different: it is a replacement for your "outer" SMTP server. It does implements it's policy and content filters capabilities and then delivers the mails to your MTA (postfix, exim, qmail, ..). This is of course more flexible regarding the mail server support, but also reduces the "backend MTA" to a mailbox delivery server. Decency's approach is the other way around: it relies on your MTA as the "main SMTP server" and integrates itself below.
To put it exaggeratedly: with qpsmtpd you could substitute the backend MTA. With Decency, you could substitute your anti-SPAM measurements. Therefore i would not argue that both address the same issue, but that both have their warrant.
Which approach is better or worse is not the question, it is more: which can you integrate in your environment.
So, how does it work?
Again an overview of the servers:
- Doorman
- Sits at the very frontier and fights SPAM before it is received with various measurements like DNSBL, other black- and white lists, SPF, sender domain <-> ip assocation analysis, geo weighting and so on.
- Detective
- On the one hand, a middleware for applying all kinds of content filtering, such as SPAM filters and virus filter from third party. On the other hand, it directly implements functions like DKIM, archiving, honeypotting and deep DNSBL applying.
- Defender
- Alternative way to use Decency. Provides either a milter or a beforeq-queue proxy interface (Postfix only). It internally uses Doorman and Detective instances to do the filtering work.
Example: Simple setup
- A mail is to be received from the internet, the mailserver (eg Postfix) does not yet accept it.
- The mailserver applies his own policies and then asks the Doorman whether the mail shall pass.
- After checking against the Doorman, the mailserver accepts and receives (or rejects) the mail and delivers it to the Detective via SMTP.
- The Detective will scan and score the mail body and re-injects it back to the mailserver.
- Then the mail get's delivered to the mailbox (or forwarded) by the mailserver.
This is a very simplified diagram of the actual processes. However, it shows the position of both Decency components: the Doorman and the Detective. This setup is also an example for a simple, single mail server scenario.
Example: Multi server setup
The concepts here are the same, but all components are dedicated to a single task. The trick has to be to allow all components to communicate easily – and without configuration hassle – to assure that all their unique results are accounted for and seen in a "global" scope. Decency does that.
Modular concept
Each module works on it's own. It can affected the global score, but can not interrupt the server itself, if it has an error.
The module order is very important, because a module can stop the processing of the mail (eg pass the mail or reject it for good).
There are also other considerations, relating to what the module actually does. A bad order example would be to put the Greylisting module at the very first position, even before validity (Basic) checks. The result would be a huge greylist database but no better protection.
There are some general best-practices which you should keep in mind (as the bad-example above), but the fine tuning entirely depends on the quality of SPAM you receive, the scale (single user, company wide, shared hoster) of your mail setup, your location (eg Germany vs Japan) and so on.