Doorman
The Doorman is a policy server, which can act before the mail is full received. It works with the informations gathered on client connection (eg certificate, IP, ..) and the SMTP protocol before DATA (so RCPT TO, MAIL FROM, ..).
What is a policy server ?
The policy server implements anti SPAM measurements before the mail is received. It does not deal with actual mail contents but the SMTP protocol, eg: MAIL FROM, RCPT TO, the sender IP, the sender HELO name and so on.
Is a policy server "better" than a content filter ?
No, but different. You see, a content filter has to have the full mail received, whereas a policy server can decide whether the mail is SPAM far before that. Thus, your mail server has to spend less resources (bandwidth, CPU time, memory) to recject the mail.
There is also a legal implication. Mails rejected by a policy server are technically not received because they are not transmitted to you. Therefore, you have not accepted them and they cannot be viewed as delivered to you in the same way a mail rejected due to technical reasons (eg high load or a hard disk problem on the mail server) is not viewed as being delivered - because it was not.
If you delete a mail after receiving, for example because you consider it SPAM, you might have a legal problem. The mail has been successfully delivered to you and the sending server has received a 200 OK from you - now it is your responsibility. However, i am not a lawyer. This is how i understand it from the lecture of this book. Please ask a local lawyer for advice on this matter, this is only my opinion.
What can it do, what can't it do ?
As mentioned before, it acts before the mail is received, thus it cannot run any kind of content filtering. This means: neither any SPAM filters (eg SpamAssassin or DSPAM ) nor any virus filter (eg ClamAV ) nor can it read the MIME header for DKIM evaluation and so on.
It can check black-, grey- and whitelists and evaluate the IP geographically, because it knows the sender IP. It can check SPF records, because it knows also the sender email address. It can implement a honey-pot, because it knows the recipient, ...
What other policy servers are there ?
There are a couple of good and solid policy servers. Without this list being claimed to be exhaustive, here are some i know of:
- policyd-weight
- postfix-policyd
- postfwd
- Overview of multiple policy servers
Those are all "only" policy servers - that's one of the reason why i wrote Decency in the first place. They do not natively communicate with any content filter, such as amavisd-new, nor with each other. However, you might be able to implement this by yourself.
Configuration
Before i go in details, keep in mind: you don't have to run only one Doorman server on a single mail server. If your mail server requires multi policies on different spots in your mail server restrictions you can run multiple instances with different configurations files. Multiple Doorman servers have access to previous scoring informations - as long as they use the same cache! In the debian or the installer package, the Doorman configuration file can be found in /etc/decency/doorman.yml.
server
Allowed values: Hash
Default: -
Suggested: { host: "127.0.0.1″, port: 15000, instances: 3 }
Required: yes
This configures the TCP settings for the server and sets the amount of instances you want to run. The server host and port settings has to be adjusted in your postfix configuration (below) accordingly.
server:
host: 127.0.0.1
port: 15000
instances: 3
server.host
The hostname or IP address the server should be listen to.
Default: 127.0.0.1 (localhost)
server.port
The port the server should be listen to.
Default: 15000
server.instances
The amount of instances to be run in parallel. The more you run, the more mails can be handled in parallel and the memory will be used.
Default: 3
weight_threshold
Allowed values: negative Integer
Default: -100
This threshold determines the aggressiveness of your server. Any module which is capable of scoring adds a negative amount of score for a SPAM mail and a positive amount for a HAM. Adjust this value according to your needs. Most scoring values can be adjusted also in the module. Assume you use the DNSBL module with a blacklist which scores -100 and you want this hit to be sufficient to reject the mail, -100 in threshold will do just that (unless the mail has collected positve scorings before). Also keep in mind, that you can use unique scorings per domain (later).
force_check_local
Allowed values: Boolean
Default: 0
Normally mails received by the Doorman come from an external source (aka the sender mail server). Any mail coming from localhost (127.0.0.1 or ::1) are therefore ignored (because they most likely come from a local executed sendmail command). However, there are certain scenarios where this behavior is not designated. If you are unsure, leave it untouched. You can also overrule this default by most module configurations per module.
default_reject_message
Allowed values: String
Default: "use decency"
This is the default reject message which will be used in the rejection answer. It will be used if you disable rejection details or the module creating the rejection does not provide any detail (most do).
no_reject_detail
Allowed values: Boolean
Default: 0
You can disable details, which indicate why the mail has been rejected, due to security concerns. However, this is not helpful if you have to determine why a particular mail has been rejected and therefore disabled by default.
disable_prepend
Allowed values: Boolean
Default: 0
This parameter is required for communication between the Doorman and the Detective. If you don't use the Detective at all, it is recommended to disable prepending by setting this value to 1. If you use the Detective, leave it enabled, because it would break the communication otherwise. See also forward_sign_key with which you can secure the prepended scores.
forward_scoring
Allowed values: Boolean
Default: 0
This is required in a distributed environment, in which the Doorman does not access the same cache as the Detective. If it does, do not enable it! If you have to enable it, you should use a sign key (forward_sign_key, below).
forward_sign_key
Allowed values: File path
Default: -
If you have a distributed environment (more than one server, communicating over an insecure line - which is any non encrypted) in which your Doorman servers does not use the same caches as the Detective it is strongly advised to sign the forwarded scoring informations. Otherwise, spammers could possibly inject scores which will allow them to pass your Detective (at least the SPAM filters, never the virus filters).
How you can generate the forward sign key and more about this matter, read here (TODO).
enable_stats
Allowed values: Bool
Default: 0
Enables statistics role which stores stats about module usage in the database.
Here are the database SQL CREATE statements (SQLite):
-- TABLE: stats_doorman_response (SQLITE):
CREATE TABLE STATS_DOORMAN_RESPONSE (period varchar(10), type varchar(32),
start integer, module varchar(32), id INTEGER PRIMARY KEY);
CREATE UNIQUE INDEX STATS_DOORMAN_RESPONSE_MODULE_PERIOD_START_TYPE ON
STATS_DOORMAN_RESPONSE (module, period, start, type);
-- TABLE: stats_doorman_performance (SQLITE):
CREATE TABLE STATS_DOORMAN_PERFORMANCE (calls varchar(10), runtime real,
period varchar(10), type varchar(32), start integer, module
varchar(32), id INTEGER PRIMARY KEY);
CREATE UNIQUE INDEX STATS_DOORMAN_PERFORMANCE_MODULE_PERIOD_START_TYPE
ON STATS_DOORMAN_PERFORMANCE (module, period, start, type);
modules
Allowed values: List of Module Hashes or List of Module configuration file paths
Default: -
This is the most important configuration parameter, because you determine the used modules here. The order of the modules does matter. If you put the CWL (Custom White List) module behind the DNSBL it means, that your whitelist settings will be applied AFTER those blacklists, which renders them useless concerning real time blacklists. You can also include any module multiple times with different configurations which could be useful for performance tweaks.
Example with inline configuration
modules:
-
DNSBL:
harsh: 0
blacklist:
-
host: bl.spamcop.net
weight: -100
Example with outsourced configuration
modules:
- DNSBL: /etc/decency/doorman/dnsbl.yml
cache, database, logging and exclusions
Read here
Full YAML example
---
weight_threshold: -100
default_reject_message: "use decency"
#force_check_local: 0
#disable_prepend: 0
#no_reject_detail: 0
forward_scoring: 1
forward_sign_key: sign.key
enable_stats: 1
server:
host: 127.0.0.1
port: 15000
instances: 3
modules:
- CWL: doorman/cwl.yml
- DNSBL: doorman/dnsbl.yml
- CBL: doorman/cbl.yml
- SPF: doorman/spf.yml
- Association: doorman/association.yml
- GeoWeight: doorman/geo-weight.yml
- Honeypot: doorman/honeypot.yml
- Greylist: doorman/greylist.yml
- Throttle: doorman/throttle.yml
Integration in Postfix
Postfix can integrate the Doorman server within it's smtpd*restrictions in the main.cf. There are four of those restrictions which have a strict order in which they run and also a different behavior how the policy results will be handled.
Order of postfix restrictions
This is the order in which the restrictions are applied:
- smtpd_client_restrictions
Postfix calls those restrictions directly after connection of the client. Therefore here is only the client IP of the client available.- REJECT handling: will reject mail for good
- OK handling: will forward the mail in the next restriction
- smtpd_helo_restrictions
This will be applied after the client sends is HELO or EHLO command to postfix. Here you have the client IP and the helo name.- REJECT handling: will reject mail for good
- OK handling: will forward the mail in the next restriction
- smtpd_sender_restrictions
After the HELO or EHLO the client will send the MAIL FROM, which will give you the sender and calls those restrictions.- REJECT handling: will reject mail for good
- OK handling: will forward the mail in the next restriction
- smtpd_recipient_restrictions
The last restrictions in the queue have all informations available, including the recipient of the mail.- REJECT handling: will reject mail for good
- OK handling: will accept the mail finally
Before i go on, there is a very important postfix config param which enforces postfix to receive the SMTP commands until RCPT TO and run all the restrictions afterwards which makes it possible to access everything in any restriction (eg the recipient in the smtpd_client_restrictions):
# in main.cf smtpd_delay_reject = yes
If you want to use the Doorman server anywhere before the smtpd_recipient_restrictions you have to enable this!
Implementation of Decency
This is actually quite easy. After configuring the Doorman and setting a server host and port you simply have to put a check_client_access statement in the destined restriction class.
smtpd_recipient_restrictions =
# your other checks
check_client_access inet:127.0.0.1:15000
# more of your other checks
permit
You can put the policy check in any restriction class, but be aware of the limitations of the result handling (REJECT and OK) as described above. I provide some full examples in the mailserver section. For more information i advise to read the Postfix SMTP Access Delegation document.
One last interesting option in postfix would be the timeout parameter, which can limit the maximum time the Doorman can take to check a mail before postfix is given up on it (main.cf):
# for all Doorman servers in seconds policy_time_limit = 3600 # for a particular Doorman server 127.0.0.1:15000_time_limit = 3600