Setting up Dovecot

Let us now configure Dovecot which will do several things for us:

  • get emails from Postfix and save them to disk
  • execute user-based "sieve" filter rules (can be used to put away emails to different folders)
  • allow the user to fetch emails using POP3 or IMAP

Before we get to the actual configuration for security reasons I recommend that you create a new system user that will own all virtual mailboxes. The following shell commands will create a system group "vmail" with GID (group ID) 5000 and a system user "vmail" with UID (user ID) 5000. (Make sure that UID and GID are not yet used or choose another - the number can be anything between 1000 and 65000 that is not yet used):

groupadd -g 5000 vmail
useradd -g vmail -u 5000 vmail -d /var/vmail -m

Also make sure that this directory has the proper permissions:

chown -R vmail:vmail /var/vmail
chmod u+w /var/vmail

The configuration files for Dovecot are found under /etc/dovecot. Start editing the main file...

/etc/dovecot/dovecot.conf

See the line protocols and define the protocols you want to offer. By default this line reads:

 protocols = imap imaps pop3 pop3s

so that Dovecot starts the IMAP and POP3 services and also its equivalents that work over an encrypted SSL (secure socket layer) connection. If you want to be strict about not allowing insecure connections then leave out the "imap" and "pop3" keywords here.

Although this is a less secure setting you will probably still need it:

 disable_plaintext_auth = no

This will allow plaintext passwords over an unsecured (non-SSL) connection. By default it is set to 'yes' for security reasons. Setting it to 'no' will mean less security but may help users of a "certain" Microsoft email software that is problematic in many ways.

An important setting is:

 mail_location = maildir:/var/vmail/%d/%n/Maildir

which will tell that the users' mailboxes are always found at /var/vmail/DOMAIN/USER/Maildir and that it should be in maildir format.

There is already a section "namespace private" in your dovecot.conf which is commented out by "#" characters. The "private" namespace is the personal mailbox of a certain user. You can leave this section disabled and get a maildir directory schema like:

/var/vmail/christoph.haas/email/Maildir/.spam

If you followed previous ISPmail tutorials then your directories may be different. If you rather have:

/var/vmail/christoph.haas/email/Maildir/.INBOX.spam

then you need to declare that in the "namespace private" section as follows. Enable this section and make sure these variables are set:

namespace private {
    separator = .
    inbox = yes
}

Next look for a section called "auth default". First define the allowed authentication mechanisms:

 mechanisms = plain login

Usually "plain" is used but a certain Micros*oft email client insists on using "login". Both mechanism use plain text so it is strongly recommended that your users use IMAPS and POP3S which are the SSL/TLS encrypted equivalents to IMAP and POP3.

As you browse through the section you see many backends that Dovecot can access to get the email users' data. We are using SQL lookups for the passdb (=password) but static information to get the users's (because all users follow the same scheme). Inside this section you need to set:

passdb sql {
    args = /etc/dovecot/dovecot-sql.conf
}

which tells Dovecot that the passwords are stored in an SQL database and:

userdb static {
    args = uid=5000 gid=5000 home=/var/vmail/%d/%n allow_all_users=yes
}

to tell Dovecot where the mailboxes are located. This is similar to the mail_location setting. The user gets authenticated in the "passdb sql" section. So the "userdb static" section defined where the mail folders are located. Using "userdb sql" is not needed as all mailboxes follow a fixed directory schema. This saves an SQL query for each access. The "allow_all_users=yes" setting means that it is not necessary for Dovecot to check if a certain user exists. We can do that because Postfix has already ensured (in the virtual_mailbox_maps query) that the users existed before their email was handed over to Dovecot's "deliver" agent.

You will want to comment out the section called "passdb pam that deals with system users. Otherwise Dovecot will also look for system users when someone fetches emails which leads to warnings in your log file.

Now look for another section called socket listen. Here you define socket files that are used to interact with Dovecot's authentication mechanism. Make the section read:

 socket listen {
    master {
        path = /var/run/dovecot/auth-master
        mode = 0600
        user = vmail
    }

    client {
        path = /var/spool/postfix/private/auth
        mode = 0660
        user = postfix
        group = postfix
    }
}

The "master" section is needed to give Dovecot's delivery agent (the program that saves a new mail to the user's mailbox) access to the userdb information. The "client" section creates a socket inside the "chroot" directory of Postfix. This socket file will be used by Postfix for SMTP authentication when users send their email through your mail server as a relay.

(chroot means that parts of Postfix are jailed into /var/spool/postfix and can only access files in that directory or its subdirectories. It is a good security measure so that even if Postfix had bugs and were hacked then the attacker would not be able to access /etc/passwd for example because it's outside of /var/spool/postfix.)

And the "protocol lda" section needs to be customized. The LDA (local delivery agent) is more capable than Postfix's built-in virtual delivery agent. It allows for quotas and Sieve (ships with the dovecot-common package) filtering. Let the section be:

 protocol lda {
    auth_socket_path = /var/run/dovecot/auth-master
    postmaster_address = postmaster@example.com
    mail_plugins = sieve
    log_path =
}

Please change the above postmaster email address to a valid address where a real human administrator can be reached.

The log_path setting is optional but may help you figure out why a certain server-side filter is not doing what you expect. Leaving it empty as shown above will log delivery details in your normal /var/log/mail.log but you can also use a file name here to create a seperate logfile.

Finally edit the /etc/dovecot/dovecot-sql.conf and change these settings:

driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mailuser2011
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';

Whenever Dovecot needs to check an email user's password if will run the above query. It will create an MD5 hash of the user's password and look for that in the "virtual_users" database table.

Restart Dovecot:

/etc/init.d/dovecot restart

Now look at your /var/log/mail.log logfile. You should see:

... dovecot: Dovecot v1.2.15 starting up (core dumps disabled)
... dovecot: auth-worker(default): mysql: Connected to 127.0.0.1 (mailserver)

Before you send a first test email you will need to fix file system permissions for the /etc/dovecot/dovecot.conf file so that the vmail user can access the Dovecot configuration. The reason is that Postfix starts the delivery agent with vmail permissions:

chgrp vmail /etc/dovecot/dovecot.conf
chmod g+r /etc/dovecot/dovecot.conf

We should also make sure that only root can access the SQL configuration file so nobody else is reading your database access passwords:

chown root:root /etc/dovecot/dovecot-sql.conf
chmod go= /etc/dovecot/dovecot-sql.conf

 

47 Comments

What about changing

What about changing permission for file /etc/dovecot/dovecot-sql.conf? It contains very important info like password to database. I know that mailuser has only SELECT but we should secure it.

dovecot

I installed yesterday to test your fine tutorial. Have Dovecot 2.0.12 installed and the configuration file is completely different. Stumped!!!!

This tutorial

is for squeeze where dovecot is version is 1.2.15.

And yes, in dovecot 2.0.12 the configuration file for dovecot are changed.

It's 'a bit' different

Hello,

debian version of dovecot 2.x  differing from 1.x in too many files so actually, if you follow this howto, it's not possibile.

However I do not think that it is impossible, but you need to adapt /change all the configuration files.

I don't have a test machine for trying, but it would be interesting know how this setup will be fine into dovecot2.

working dovecot.conf for Dovecot 2.0 from w3sz

Here is a working Dovecot 2.0 dovecot.conf file that I am using successfully.  Otherwise, the implementation was as per this excellent tutorial.

# 2.0.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.0.0-12-generic i686 Ubuntu 11.10 ext4
auth_verbose = yes
disable_plaintext_auth = no
login_greeting = Dovecot Available
mail_location = mbox:/mail:INBOX=/var/mail/%u
mail_privileged_group = mail
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
passdb {
  driver = pam
}
plugin {
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
protocols = imap pop3 imap pop3
service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
  }
  unix_listener auth-userdb {
    mode = 0600
    user = vmail
  }
}
ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_key = </etc/ssl/private/dovecot.pem
userdb {
  args = uid=5000 gid=5000 home=/mail allow_all_users=yes
  driver = static
}
userdb {
  driver = passwd
}
protocol lda {
  auth_socket_path = /var/run/dovecot/auth-master
  log_path = /var/log/dovecot.log
  mail_plugins = sieve
  postmaster_address = w3sz@w3sz.net
}

Perhaps not exactly...from w3sz

What I posted above was generated from my dovecot.conf file by using:
 doveconf -n > dovecot-new.conf.  I assumed that this just removed the comments from my newly created dovecot.conf file, but that is not the case.
 It changed some things, and added pam which I did not invoke.

This dovecot.conf file is created by cutting and pasting from my current working version.  Perhaps a guru can comment on the differences.  I have no intention of trying the 'new' version above, as what I am posting here works, and I am not curious enough to see if the automatically generated one does too.  Here is the actual working version:

protocols=imap pop3
disable_plaintext_auth = no

auth_verbose = yes
login_greeting = Dovecot Available

mail_location=mbox:/mail:INBOX=/var/mail/%u
mail_privileged_group = mail

auth_mechanisms=plain login

passdb {
    driver=sql
    args =/etc/dovecot/dovecot-sql.conf
   }

userdb {
    driver=static
    args = uid=5000 gid=5000 home=/mail allow_all_users=yes
   }

service auth {
  unix_listener auth-userdb {
    path = /var/run/dovecot/auth-master
    mode = 0600
    user = vmail # User running dovecot-lda
   }

  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
  }

}

protocol lda {
    auth_socket_path = /var/run/dovecot/auth-master
    postmaster_address = w3sz@w3sz.net
    mail_plugins = sieve
    log_path = /var/log/dovecot.log
}

dict {
  #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
  #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext
}

!include_try local.conf

dovecot.conf

is this really necessary ? 

chgrp vmail /etc/dovecot/dovecot.conf
chmod g+r /etc/dovecot/dovecot.conf

the default is already world readable (644)

error in mail.log

Hi, I have this error in mail.log 

 

dovecot: Dovecot v1.2.15 starting up (core dumps disabled)

dovecot: auth(default): bind(/var/spool/postifx/private/auth) failed: No such file or directory

dovecot: auth(default): Fatal: net_listen_unix(/var/spool/postifx/private/auth) failed: No such file or directory

dovecot: dovecot: Fatal: Auth process died too early - shutting down
and this in mail.err
	dovecot: auth(default): bind(/var/spool/postifx/private/auth) failed: No such file or directory

dovecot: auth(default): Fatal: net_listen_unix(/var/spool/postifx/private/auth) failed: No such file or directory

dovecot: Fatal: Auth process died too early - shutting down
an ideas?

mail_location problem

When I try to send an email, I always get the following response:

Diagnostic-Code: x-unix; procmail: Couldn't create "/var/mail/john"

I don't know why the mail_location option doesn't works. Meybe I made a mistake somewhere else, but I don't know how to debug this issue. Can everyone give me hint?

trouble with e-mails receive ...

I installed all as described. I was stuck in a time when I send mail. It goes to the proper directory, but the attempt to read the post fails.
referrals: telnet localhost pop3, then logs, password, and the commandos, "stat" returns "+ OK 0 0", whilethere are messages in the directory / home / vmail / domain / user / Maildir ... .
My impression is that not pop3 link referral to the proper directory, or is poorly set some path, but where?
Surely somewhere there is an error, because the logs do not currently have any irregularities.
Any idea?

This runs but...

Hello

i tried this tutorial until this Topic.

I see this logmessage but on which option will be set that the Mailhome is on /var/vmail....?

about the logfile

You should make a comment about which permissions to use if you want to use a different log-file than the mail.log from below

protocol lda {
    auth_socket_path = /var/run/dovecot/auth-master
    postmaster_address = postmaster@example.com
    mail_plugins = sieve
    log_path =
}

I got an error saying

(temporary failure. Command output: Can't open log file /var/log/another-filenamename.log: Permission denied )

And I don't know which permissions to set on this file.

I have error in

I have error in mail.log

dovecot: Dovecot v1.2.15 starting up (core dumps disabled)
dovecot: auth(default): Fatal: static userdb: Empty key (=)
dovecot: dovecot: Fatal: Auth process died too early - shutting down

What is the problem?

Dovecot 2.x "Translation"

I'm trying to setup dovecot 2.x right now and thought that I might share some information with you ..

protocols = imap imaps pop3 pop3s
>> to be found in /usr/share/dovecot/protocols.d/ (in 2 files; they are included by /etc/dovecot/dovecot.conf)

disable_plaintext_auth = no
mechanisms = plain login AS auth_mechanism = plain login
>> to be found in /etc/dovecot/conf.d/10-auth.conf

mail_location = maildir:/var/vmail/%d/%n/Maildir
namespace private {
separator = .
inbox = yes
}
>> /etc/dovecot/conf.d/10-mail.conf

passdb sql {
  args = /etc/dovecot/dovecot-sql.conf
}
userdb static {
  args = uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes
}
AS
passdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
  driver = static
  args = uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes
}
# you should create an empty file "conf.d/auth-sql.conf.ext" and uncomment the line "!include auth-sql.conf.ext" in 10-auth.conf (at the end of the file)
>> /etc/dovecot/conf.d/auth-sql.conf.ext

socket listen {
  master {
    path = /var/run/dovecot/auth-master
    mode = 0600
    user = vmail
  }
  client {
    path = /var/spool/postfix/private/auth
    mode = 0660
    user = postfix
    group = postfix
  }
}
(maybe) AS
Service auth {
  unix_listener /var/run/dovecot/auth-master {
    mode = 0600
    user = vmail
    group = vmail
  }
  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}
# have to test it ...
>> /etc/dovecot/conf.d/10-master.conf

protocol lda {
  auth_socket_path = /var/run/dovecot/auth-master
  postmaster_address = postmaster@example.com
  mail_plugins = sieve
  log_path =
}
#postmaster_adress isn't included in protocol lda, has it's own line at the beginning of the file.
#Maybe auth_socket_path is not needed here anymore, will test it and report ...
>> /etc/dovecot/conf.d/15-lda.conf

driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mailuser2011
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';
>> /etc/dovecot/dovecot-sql.conf.ext (all lines are commented out)

And finally you have to chgrp and chmod the dovecot.conf,the whole conf.d/ directory and /usr/share/dovecot/protocols.d/ (as it gets included during runtime or you just replace the include in dovecot.conf with "protocols = imap imaps pop3 pop3s")

Reporting ...

Except that

unix_listener /var/run/dovecot/auth-master

is just

unix_listener auth-master

everything seems to be correct and Dovecot works fine.



please note the changes from dovecot 2.0.12 to 2.0.13

Service auth {
  unix_listener /var/run/dovecot/auth-master {
    mode = 0600
    user = vmail
    group = vmail
  }

...

 }

is not  accepted anymore--->

unix_listener /var/run/dovecot/auth-master

<---is not accepted anymore.

it needs to be :

 Service  auth {

  #
  # be aware!: unix_listener auth-master <-
  # stays for /var/run/dovecot/auth-master !without the base-dir prefix!
  #
  unix_listener auth-master {
    mode = 0600
    user = vmail
    group = vmail
    }

  ...

That's true

Thanks for this comment. I changed it: 

mail_location = maildir:/var/vmail/%d/%n/mail

and in the userdb static { section I used 

args = uid=5000 gid=5000 home=/var/vmail/%d/%n/home allow_all_users=yes

Hope this is ok ?

user vmail

hi,

does the user vmail really needs a valid shell or can i type

useradd -g vmail -u 5000 vmail -d /var/vmail -m -s /bin/false

best regards

Created New Howto for Dovecot2 as Proxy to DBMail.

Hi,

I'm wondering If I may post here the link to my Howto.

The Howto uses Workaround setup as a basis to go further and save the mail messages itself inside the MySQL database, using Dovecot2 for the user authentication, that is organised the way the Workaround tutorial does only afterwards the Dovecot does proxy to the DBMail listening on the localhost only.

content.fens.org/index.php?q=admin-howto/mail/dovecot2dbmail-proxy

hope I may post the urls here.

a yes, thanks for your howto, it has helped me in life!

:)

Thanks, MySQL slownes - may be a warning, but outOf experiance..

Hi Christoph,

out of my experiance I can say following:

MySQL slowness may very well be - hardware architecture dependant.

I have noted that AMD Quad CPU  processor machine for instance, - was much slower with MySQL then the Intel Core Duo CPU machine.

INTEL did the same queries in no time at all.

Perhaps MySQL is C - programmed that way that it is just very much CPU - second level cache hungry?

I have tested to work with it on the biggest database tables I have experienced so far, apr. 32.000 entries a table, very much columns inside, useing some left-joins to some others of the same size ( it may sound as a joke for other administrators that are working on something much bigger, but it was enough for me to instantly see the difference.)

Since then I am using only Intel CPU architecture for the important MySQL database solutions.

And now, why am I that "married" with the MySQL databases? 

The answer is: like the black magician in 'Connan the Barbar' of Mileus's film was searchng for the sectret of still, I am on my quest of searching the cheap DNS Failover solution for the firm I am working for at the time being. ;)))

And MySQL database supports Master - Master data replication with the asyncronous settings for the data counters. One can find more of what I am thinking on that topic inside the last not very perfect written entry of my tutorial where I am try to elaborate about it.

I have no experiance with the Postgres at all does it do database replication?

Thanks for your wellcome, I am earned.

Leonid

Auth Problem?

Hi. After restarting Dovecot, I look in my mail.log and see this:

 

Apr 14 22:27:23 server1 dovecot: Dovecot v1.2.15 starting up (core dumps disabled)
Apr 14 22:27:23 server1 dovecot: auth(default): Error in configuration file /etc/dovecot/dovecot-sql.conf line 65: Unknown setting: password
Apr 14 22:27:23 server1 dovecot: dovecot: child 1882 (auth) returned error 89 (Fatal failure)
Apr 14 22:27:23 server1 dovecot: dovecot: Fatal: Auth process died too early - shutting down

I'm not sure what's wrong.

  # This file is opened as

 

# This file is opened as root, so it should be owned by root and mode 0600.
#
# http://wiki.dovecot.org/AuthDatabase/SQL
#
# For the sql passdb module, you'll need a database with a table that
# contains fields for at least the username and password. If you want to
# use the user@domain syntax, you might want to have a separate domain
# field as well.
#
# If your users all have the same uig/gid, and have predictable home
# directories, you can use the static userdb module to generate the home
# dir based on the username and domain. In this case, you won't need fields
# for home, uid, or gid in the database.
#
# If you prefer to use the sql userdb module, you'll want to add fields
# for home, uid, and gid. Here is an example table:
#
# CREATE TABLE users (
# username VARCHAR(128) NOT NULL,
# domain VARCHAR(128) NOT NULL,
# password VARCHAR(64) NOT NULL,
# home VARCHAR(255) NOT NULL,
# uid INTEGER NOT NULL,
# gid INTEGER NOT NULL,
# active CHAR(1) DEFAULT 'Y' NOT NULL
# );

# Database driver: mysql, pgsql, sqlite
#driver = mysql

# Database connection string. This is driver-specific setting.
#
# pgsql:
# For available options, see the PostgreSQL documention for the
# PQconnectdb function of libpq.
#
# mysql:
# Basic options emulate PostgreSQL option names:
# host, port, user, password, dbname
#
# But also adds some new settings:
# client_flags - See MySQL manual
# ssl_ca, ssl_ca_path - Set either one or both to enable SSL
# ssl_cert, ssl_key - For sending client-side certificates to server
# ssl_cipher - Set minimum allowed cipher security (default: HIGH)
# option_file - Read options from the given file instead of
# the default my.cnf location
# option_group - Read options from the given group (default: client)
#
# You can connect to UNIX sockets by using host: host=/var/run/mysqld/mysqld.sock
# Note that currently you can't use spaces in parameters.
#
# MySQL supports multiple host parameters for load balancing / HA.
#
# sqlite:
# The path to the database file.
#
# Examples:
# connect = host=192.168.1.1 dbname=users
# connect = host=sql.example.com dbname=virtual user=virtual password=blarg
# connect = /etc/dovecot/authdb.sqlite
#
driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser
LINE 65-password=mailusercesar
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM virtual_usersWHERE email='%u';
# Default password scheme.
#
# List of supported schemes is in
# http://wiki.dovecot.org/Authentication/PasswordSchemes
#
#default_pass_scheme = MD5

# passdb query to retrieve the password. It can return fields:
# password - The user's password. This field must be returned.
# user - user@domain from the database. Needed with case-insensitive lookups.
# username and domain - An alternative way to represent the "user" field.
#
# The "user" field is often necessary with case-insensitive lookups to avoid
# e.g. "name" and "nAme" logins creating two different mail directories. If
# your user and domain names are in separate fields, you can return "username"
# and "domain" fields instead of "user".
#
# The query can also return other fields which have a special meaning, see
# http://wiki.dovecot.org/PasswordDatabase/ExtraFields
#
# Commonly used available substitutions (see http://wiki.dovecot.org/Variables
# for full list):
# %u = entire user@domain
# %n = user part of user@domain
# %d = domain part of user@domain
#
# Note that these can be used only as input to SQL query. If the query outputs
# any of these substitutions, they're not touched. Otherwise it would be
# difficult to have eg. usernames containing '%' characters.
#
# Example:
# password_query = SELECT userid AS user, pw AS password \
# FROM users WHERE userid = '%u' AND active = 'Y'
#
#password_query = \
# SELECT username, domain, password \
# FROM users WHERE username = '%n' AND domain = '%d'

# userdb query to retrieve the user information. It can return fields:
# uid - System UID (overrides mail_uid setting)
# gid - System GID (overrides mail_gid setting)
# home - Home directory
# mail - Mail location (overrides mail_location setting)
#
# None of these are strictly required. If you use a single UID and GID, and
# home or mail directory fits to a template string, you could use userdb static
# instead. For a list of all fields that can be returned, see
# http://wiki.dovecot.org/UserDatabase/ExtraFields
#
# Examples:
# user_query = SELECT home, uid, gid FROM users WHERE userid = '%u'
# user_query = SELECT dir AS home, user AS uid, group AS gid FROM users where userid = '%u'
# user_query = SELECT home, 501 AS uid, 501 AS gid FROM users WHERE userid = '%u'
#
#user_query = \
# SELECT home, uid, gid \
# FROM users WHERE username = '%n' AND domain = '%d'

# If you wish to avoid two SQL lookups (passdb + userdb), you can use
# userdb prefetch instead of userdb sql in dovecot.conf. In that case you'll
# also have to return userdb fields in password_query prefixed with "userdb_"
# string. For example:
#password_query = \
# SELECT userid AS user, password, \
# home AS userdb_home, uid AS userdb_uid, gid AS userdb_gid \
# FROM users WHERE userid = '%u'

This is how I fixed

This is how I fixed it,

driver = mysql
connect = host=127.0.0.1 dbname=mailserver user=mailuser
password=mailusercesar
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM virtual_usersWHERE email='%u';

I combined the:

connect = host=127.0.0.1 dbname=mailserver user=mailuser

with password=mailusercesar (of course with a space).