Configure Doorman Server

Table of content

Quickstart
1. The Doorman config file
2. Start the Doorman
  1. Running as daemon
Connect to Postfix
Why module-order matters
Finetunting
What's next ?

Quickstart

As a first and very basic scenario, just let's get started by setting up Doorman with three simple modules to explain you the actual usage and show how module order in your configuration matters.

We will use DNSBL (a DNS Blackhole List), Basic (validity checks) and Greylist (well, a greylist), because they are simple but provide extend configuration as well to be good for an example. Also, using those three modules can be enough for a simple SPAM defense.

The Doorman config file

I assume you've used the standard configuration directories, so you'll find the Doorman config file in /etc/decency/doorman.yml

It is already filled with some stuff, but to get you started, just remove it and begin with a blank slate.

First, you should set the environment stuff. This includes logging, database and cache.

One thing: All the config snippets are YAML. So DONT use TABS! I use 4 spaces, you can go with two if you are a lazy typer - as long as you stay consistent.

Database

I am going to use a SQLite database, cause it's either already installed or you can install it quite simple (via your distribution installer).

Insert into the (empty) configuration file the following:

---

# Using SQLite database
database:
    type: DBD
    args:
        - 'dbi:SQLite:dbname=/var/spool/decency/database.db'

I think it's quite readable. The 'dbi:SQLite:dbname=' gibberish is a Perl Data Source Name (DSN) - don't bother. You can edit the path to the database if you wish.

Cache

Having this done, let's use a cache. For the example, we'll got with the simplest one: a memory cache. The downside of this cache is, that i will be wiped on server restart and also it can grow large over time. For production i'd rather go with FastMmap or even Memcached.

So, add the following under the database configuration:

# Using Memory cache
cache:
    class: Memory

Logging

Lets use directory logging for the example. Normally, decency writes directory logs in /var/log/decency/, but you can change this as you like. Just make sure the decency user can write to the directory.

Here we go, add the following below the cache settings:

# Directory logging
logging:
    log_level: 1
    directory: /var/log/decency

Server settings

This part sets up the server itself, where it listens, how many mails it can work on in parallel and so on.

Here we go:

server:
    host: 127.0.0.1
    port: 15000
    instances: 3

Modules

The last part is setting up the modules. You could write the module configuration inline, but i am not a big fan of huge config files (better more, each short).

Announcing modules in the server config

The order of the modules is important. So we begin with the Basic checks, then the DNSBL and at last the Greylist.

modules:
    - Basic: /etc/decency/doorman/basic.yml
    - DNSBL: /etc/decency/doorman/dnsbl.yml
    - Greylist: /etc/decency/doorman/greylist.yml

One thing, while we're at the last config entry in the server config file: It should end with a blank link, or the Perl YAML parser will complain..

Basic-Module config

Ok, to the Basic module. If you are familiar with postfix basic validations, you will recognize them here. If not: all checks consider the validy of either the HELO name the connecting mail server has provider, the sender domain, the recipient domain oder the client's hostname. I will not go in depth, that you can find in the module description, so here are the settings for /etc/decency/doorman/basic:

---

weight_invalid_helo_hostname: -100
weight_non_fqdn_helo_hostname: -100
weight_non_fqdn_recipient: -100
weight_non_fqdn_sender: -100
weight_unknown_helo_hostname: -50
weight_unknown_recipient_domain: -50
weight_unknown_sender_domain: -50
weight_unknown_reverse_client_hostname: -25

Again, don't miss the last empty line..

DNSBL-Module config

DNSBL in short: check the connecting mailserver IP against a blackhole list. If it is on the list, it get's scored negatively. Which lists you use is your choice. The following are an example /etc/decency/doorman/dnsbl.

---

blacklist:
    -
        host: ix.dnsbl.manitu.net
        weight: -80
    -
        host: bl.spamcop.net
        weight: -80
    -
        host: dnsbl.sorbs.net
        weight: -60

Empty line ..

Greylist-Module config

Here is the last module for our example: the Greylist. It basicially reject any "first-contact" deliver attempt with a temporary error. If the sender is a real mailserver, it will queue the mail and try again later. Later the mail is accepted and everybody is happy. Here you go /etc/decency/doorman/greylist.yml :

---

recipient_domain_threshold: 5
sender_domain_threshold: 10

Empty line ..

Setup the database

Your Greylist module requires a database. Also you could enable later on statistics for your Doorman. So, assuming you have the sqlite commandline tools installed (if not: do now), open an empty file (eg /tmp/tables.sql) and add the SQL syntax:

-- TABLE: greylist_recipient (SQLITE):
CREATE TABLE GREYLIST_RECIPIENT ("to_domain" varchar(255),
    "from_address" varchar(255), "last_update" integer, "ip" varchar(39),
    "data" integer, id INTEGER PRIMARY KEY);
CREATE INDEX GREYLIST_RECIPIENT_LAST_UPDATE ON
    GREYLIST_RECIPIENT ("last_update");
CREATE UNIQUE INDEX GREYLIST_RECIPIENT_FROM_ADDRESS_IP_TO_DOMAIN ON
    GREYLIST_RECIPIENT ("from_address", "ip", "to_domain");

-- TABLE: greylist_sender (SQLITE):
CREATE TABLE GREYLIST_SENDER ("to_domain" varchar(255),
    "from_domain" varchar(255), "last_update" integer, "ip" varchar(39),
    "data" integer, id INTEGER PRIMARY KEY);
CREATE INDEX GREYLIST_SENDER_LAST_UPDATE ON GREYLIST_SENDER ("last_update");
CREATE UNIQUE INDEX GREYLIST_SENDER_FROM_DOMAIN_IP_TO_DOMAIN ON
    GREYLIST_SENDER ("from_domain", "ip", "to_domain");

-- TABLE: greylist_address (SQLITE):
CREATE TABLE GREYLIST_ADDRESS ("from_address" varchar(255),
    "last_update" integer, "ip" varchar(39), "data" integer,
    "to_address" varchar(255), id INTEGER PRIMARY KEY);
CREATE INDEX GREYLIST_ADDRESS_LAST_UPDATE ON GREYLIST_ADDRESS ("last_update");
CREATE UNIQUE INDEX GREYLIST_ADDRESS_FROM_ADDRESS_IP_TO_ADDRESS
    ON GREYLIST_ADDRESS ("from_address", "ip", "to_address");


-- TABLE: stats_doorman_results (SQLITE):
CREATE TABLE STATS_DOORMAN_RESULTS ("calls" integer, "period" varchar(10),
    "last_update" integer, "status" varchar(32), "module" varchar(32),
    "start" integer, id INTEGER PRIMARY KEY);
CREATE INDEX STATS_DOORMAN_RESULTS_START ON STATS_DOORMAN_RESULTS ("start");
CREATE UNIQUE INDEX STATS_DOORMAN_RESULTS_MODULE_PERIOD_STATUS_START
    ON STATS_DOORMAN_RESULTS ("module", "period", "status", "start");

-- TABLE: stats_doorman_performance (SQLITE):
CREATE TABLE STATS_DOORMAN_PERFORMANCE ("calls" integer, "period"
    varchar(10), "last_update" integer, "score" integer, "runtime" real,
    "module" varchar(32), "start" integer, id INTEGER PRIMARY KEY);
CREATE INDEX STATS_DOORMAN_PERFORMANCE_START ON
    STATS_DOORMAN_PERFORMANCE ("start");
CREATE UNIQUE INDEX STATS_DOORMAN_PERFORMANCE_MODULE_PERIOD_START
    ON STATS_DOORMAN_PERFORMANCE ("module", "period", "start");

-- TABLE: stats_doorman_final_state (SQLITE):
CREATE TABLE STATS_DOORMAN_FINAL_STATE ("amount" integer,
    "period" varchar(25), "status" varchar(10), "start" integer,
    id INTEGER PRIMARY KEY);
CREATE INDEX STATS_DOORMAN_FINAL_STATE_START ON
    STATS_DOORMAN_FINAL_STATE ("start");
CREATE UNIQUE INDEX STATS_DOORMAN_FINAL_STATE_PERIOD_STATUS_START
    ON STATS_DOORMAN_FINAL_STATE ("period", "status", "start");

Then create your database out of it:

$ sqlite3 /var/spool/decency/database.db < /tmp/tables.sql

Substitute /var/spool/decency/database.db with the path to your database. Also assure the Decency user has write access to the database:

# assuming your user and group are called "decency"
chown decency:decency /var/spool/decency/database.db

Start the Doorman

Thats it! You can now start the Doorman. For the first run, it's a good idea not to use the init.d-scripts or daemonize the server, because you will see all errors directly.

# direct
$ /opt/decency/bin/decency.pl -u decency -g decency -a doorman -l 6

You can stop the server by pressing "Ctrl+c".

Running as daemon

If everything works well, you can run the Doorman as daemon. Either use the init.d scripts or append the "-d" parameter:

# init scripts
$ /etc/init.d/decency-doorman start

# direct
$ /opt/decency/bin/decency.pl -u decency -g decency -a doorman -d

Connect to Postfix

To connect the Doorman to Postfix, you have to add a check-directive at a convenient point in your postfix config. If you configured config with smtpd_delay_reject = yes you can put it in any restriction class. If not, it should be placed in smtpd_recipient_restrictions. Here you go (in /etc/postfix/main.cf):

smtpd_recipient_restrictions =
    # .. something before
    check_policy_service inet:127.0.0.1:15000
    # .. something after

Of course: restart Postfix.

Why module-order matters

In the above example, we used Basic -> DNSBL -> Greylist. What, if we'd just reversed that order? Well, it would probably work the same, at least with the same accuracy .. BUT: your greylist database would grow enormously. From my experience, Basic and DNSBL will filter out about 90-95% of all SPAM beforehand (they are probably the most effective modules), so the Greylist has to deal with about 5-10% of the actual spams. If you reversed the order, Greylist has to cope with 10 to 20 times of the amount and will write about 10-20 times more database entries.

What if you'd put DNSBL before Basic? That will also work, but DNSBL has to query external blacklists, whereas Basic just does "local" validity checks. So you'd slow down your server, due to the more queries to external DNS servers. Basic would filter out somthing in the range of 50-70% beforehand, so DNSBL has to check less IPs if it runs after Basic.

Finetunting

You should familiarize yourself with the modules and the concepts they implement. Here is a list of all currently available modules:

Association
Tries to determine associations between the sending mailserver IP and the sender domain. If it finds none, the sender gets a negative score.
Basic
Implements basic checks of validity and existance. Eg checks whether the sender HELO name is a valid hostname or something like asd**asd; check whether the sender domain does exist (in term of: has an existing MX or A record)
CWL and BWL
White- and Blacklists by recipient domain based on sender IP, sender domain or sender address.
DNSBL
Checks the sending mailserver IP against weighted DNS Blackhole Lists.
Geo Weight
Applies a positive or negative score based on the location (country) of the sending mailserver.
Greylist
Takes advantage of the 450-temporary-not-available responses of the SMTP protocol to defend against simple SPAM bots.
Honeypot
Helps you tailoring your SPAM list to your needs by automated building of a black list.
Sender Permit
Implements a relay-like mailserver on a domain basis, using SMTPS certificates and/or IPs.
SPF
Uses the Sender Permit Framework to determine whether the sending server is allowed to send for the sender domain.

What's next ?

Also read the Doorman configuration, where you can find out about fine tuning your scoring, default responses and so on.

In the cache and database documentation you can read about the pros and cons about the supported databases and caches (and how to configure them).

Then of course go to the Detective tutorial, setup your Detective and understand how Doorman and Detective interact.

Decency

The spam fighting framework