ipng/ipng.ch
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
3c69130cea21779d1e7a0e70c7fb0d0924771f06
ipng.ch/content/articles/2024-05-17-smtp.md
Pim van Pelt 8b23bba61d
All checks were successful
continuous-integration/drone/push Build is passing
Details
Redo the logos to have transparent background
2024-08-13 01:11:55 +02:00

1137 lines
50 KiB
Markdown
Raw Blame History

---
date: "2024-05-17T10:56:54Z"
title: 'Case Study: IPng''s mail servers'
aliases:
- /s/articles/2024/05/17/smtp.html
---
## Intro
I have seen companies achieve great successes in the space of consumer internet and entertainment
industry. I've been feeling less enthusiastic about the stronghold that these corporations have over
my digital presence. I am the first to admit that using "free" services is convenient, but these
companies are sometimes taking away my autonomy and exerting control over society. To each their own
of course, but for the last few years, I've been more and more inclined to take back a little bit of
responsibility for my online social presence, away from centrally hosted services and to privately
operated ones.
{{< image width="120px" float="right" src="/assets/smtp/gmail_logo.png" alt="GMail" >}}
First off - I **love** Google's Workspace products. I started using GMail just after it launched,
back in 2004. Its user interface is sleek, performant, and very intuitive. Its filtering, granted,
could be a bit less ... robotic, but that's made up by labels and an incredibly comprehensive
search function. I would dare say that between GMail and Photos, those are my absolute favorite
products on the internet.
That said, I have been running e-mail servers since well before Google existed as a company. I
started off at M.C.G.V. Stack, the computer club of the University of Eindhoven, in 1995. We ran
sendmail back then, and until about two months ago, I have continuously run sendmail in production
using the PaPHosting platform [[ref](https://paphosting.net/)] that I wrote with my buddies Paul
and Jeroen.
However, two things happened, both of them somewhat nerdsnipe-esque:
1. Mrs IPngNetworks said "Well if you are going to use NextCloud and PeerTube and PixelFed and
Mastodon, why would you not run your own mailserver?"
1. I added a forward for `event@frys-ix.net` on my Sendmail relays at PaPHosting, and was tipped
by my buddy Jelle that his e-mail to it was bouncing due to SPF strictness.
### I tried to resist ...
{{< image width="150px" float="left" src="/assets/smtp/pulling_hair.png" alt="Pulling Hair" >}}
My main argument against running a mailserver has been the mailspool. Before I moved to GMail, I had
the misfortune of having my mail and primary DNS fail, running on `bfib.ipng.nl` at the time, a
server 700km away from me, without redundancy. Even the nameserver slaves went beyond their zone
refresh. It was not a good month for me, even though it was twenty years or so ago :)
Last year, during a roadtrip with Fred, he and I spent a few long hours restoring a backup after a
catastrophic failure of a hypervisor at IP-Max on which his mailserver was running. Luckily, backups
were awesome and saved the day, but having to go into a Red Alert mode and not being able to
communicate, really can be stressful. I don't want to run mailservers!!!1
### .. but resistance is futile
After this nerdsnipe, I had a short conversation with Jeroen who mentioned that since I last had a
look at this, Dovecot, a popular imap/pop3 server, had gained the ability to do mailbox
synchronization across multiple machines. That's a really nifty feature - but also it meant that
there will be no more single points of failure, if I do this properly. Oh crap, there's no longer an
argument of resistance? Nerd-snipe accepted!
Let me first introduce the mail^W main characters of my story:
| {{< image src="/assets/smtp/postfix_logo.png" width="8em" >}} | {{< image src="/assets/smtp/dovecot_logo.png" width="8em" >}} | {{< image src="/assets/smtp/nginx_logo.png" width="8em" >}} | {{< image src="/assets/smtp/rspamd_logo.png" width="8em" >}} | {{< image src="/assets/smtp/unbound_logo.png" width="8em" >}} | {{< image src="/assets/smtp/roundcube_logo.png" width="8em" >}} |
| ---- | ---- | ---- | ---- | ---- | ---- |
* ***Postfix***: is Wietse Venema's mail server that started life at IBM research as an
alternative to the widely-used Sendmail program. After eight years at Google, Wietse continues
to maintain Postfix.
* ***Dovecot***: an open source IMAP and POP3 email server for Linux/UNIX-like systems, written
with security primarily in mind. Dovecot is an excellent choice for both small and large
installations.
* ***NGINX***: an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy
server, originally written by Igor Sysoev.
* ***Rspamd***: an advanced spam filtering system and email processing framework that allows
evaluation of messages by a number of rules including regular expressions, statistical analysis
and custom services such as URL black lists.
* ***OpenDKIM***: is a community effort to develop and maintain a C library for producing
DKIM-aware applications and an open source milter for providing DKIM service.
* ***Unbound***: a validating, recursive, caching DNS resolver. It is designed to be fast and
lean and incorporates modern features based on open standards.
* ***Roundcube***: a web-based IMAP email client. Roundcube's most prominent feature is the
pervasive use of Ajax technology.
In the rest of this article, I'll go over four main parts that I used to build a fully redundant
and self-healing mail service at IPng Networks:
1. <span style="color:green">**Green**</span>: `smtp-in.ipng.ch` which handles inbound e-mail
1. <span style="color:red">**Red**</span>: `imap.ipng.ch` which serves mailboxes to users
1. <span style="color:blue">**Blue**</span>: `smtp-out.ipng.ch` which handles outbound e-mail
1. <span style="color:magenta">**Magenta**</span>: `webmail.ipng.ch` which exposes the mail in a web browser
Let me start with a functional diagram, using those colors:
{{< image src="/assets/smtp/IPng Mail Cluster.svg" alt="IPng Mail Cluster" >}}
As you can see in this diagram, I will be separating concerns and splitting the design into three
discrete parts, which will also be in three sets of redundantly configured backend servers running
on IPng's hypervisors in Zurich (CH), Lille (FR) and Amsterdam (NL).
### 1. Outbound: smtp-out
I'm going to start with a relatively simple component first: outbound mail. This service will be
listening on the _smtp submission_ port 587, require TLS and user authentication from clients,
validate outbound e-mail using a spam detection agent, and finally provide DKIM signing on all
outbound e-mails. It should spool and retry the delivery in case there is a temporary issue (like
greylisting, or server failure) on the receiving side.
Because the only way to send e-mail will be using TLS and user authentication, the smtp-out servers
themselves will not need to do any DNSBL lookups, which is convenient because it means I can put
them behind a loadbalancer and serve them entirely within IPng Site Local. If you're curious as to
what this site local thing means, basically it's an internal network spanning all IPng's points of
presence, with an IPv4, IPv6 and MPLS backbone that is disconnected from the internet. For more
details on the design goals, take a look at the [[article]({{< ref "2023-03-11-mpls-core" >}})] I wrote about it last year.
#### Debian VMs
I'll take three identical virtual machines, hosted on three separate hypervisors each in their own
country.
```
pim@summer:~$ dig ANY smtp-out.net.ipng.ch
smtp-out.net.ipng.ch. 60 IN A 198.19.6.73
smtp-out.net.ipng.ch. 60 IN A 198.19.4.230
smtp-out.net.ipng.ch. 60 IN A 198.19.6.135
smtp-out.net.ipng.ch. 60 IN AAAA 2001:678:d78:50e::9
smtp-out.net.ipng.ch. 60 IN AAAA 2001:678:d78:50a::6
smtp-out.net.ipng.ch. 60 IN AAAA 2001:678:d78:510::7
````
I will give them each 8GB of memory, 4 vCPUs, and 16GB of bootdisk. I'm pretty confident that the
whole system will be running in only a fraction of that. I will install a standard issue Debian
Bookworm (12.5), and while my VMs by default have 4 virtual NICs, I only need one, connected to the
IPng Site Local:
```
pim@smtp-out-chrma0:~$ ip -br a
lo UNKNOWN 127.0.0.1/8 ::1/128
enp1s0f0 UP 198.19.6.135/27 2001:678:d78:510::7/64 fe80::5054:ff:fe99:81b5/64
enp1s0f1 UP fe80::5054:ff:fe99:81b6/64
enp1s0f2 UP fe80::5054:ff:fe99:81b7/64
enp1s0f3 UP fe80::5054:ff:fe99:81b8/64
pim@smtp-out-chrma0:~$ mtr -6 -c5 -r dns.google
Start: 2024-05-17T13:49:28+0200
HOST: smtp-out-chrma0 Loss% Snt Last Avg Best Wrst StDev
1.|-- msw1.chrma0.net.ipng.ch 0.0% 5 1.6 1.5 1.3 1.6 0.1
2.|-- msw0.chrma0.net.ipng.ch 0.0% 5 1.4 1.3 1.3 1.4 0.1
3.|-- msw0.chbtl0.net.ipng.ch 0.0% 5 3.2 3.1 2.8 3.2 0.2
4.|-- hvn0.chbtl0.net.ipng.ch 0.0% 5 1.5 1.5 1.4 1.5 0.0
5.|-- chbtl0.ipng.ch 0.0% 5 1.6 1.7 1.6 1.7 0.0
6.|-- chrma0.ipng.ch 0.0% 5 2.4 2.4 2.4 2.5 0.0
7.|-- as15169.lup.swissix.ch 0.0% 5 3.2 3.8 3.2 5.6 1.0
8.|-- 2001:4860:0:1::6083 0.0% 5 4.5 4.5 4.5 4.5 0.0
9.|-- 2001:4860:0:1::12f9 0.0% 5 3.4 3.5 3.4 3.5 0.0
10.|-- dns.google 0.0% 5 3.8 3.9 3.8 4.0 0.1
```
One cool observation: these machines are not really connected to the internet - you'll note that
their IPv4 address is from reserved space, and their IPv6 supernet (2001:678:d78:500::/56) is
filtered at the border. I'll get to that later!
#### Postfix
I will install Postfix, and make a few adjustments to its config. First off, this mailserver will only
be receiving `submission` mail, which is port 587. It will not participate or listen to the regular
`smtp` port 25, nor `smtps` port 465, as such the `master.cf` file for Postfix becomes:
```
#smtp inet n - y - - smtpd
# -o smtpd_sasl_auth_enable=no
submission inet n - y - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_reject_unlisted_recipient=no
-o smtpd_client_restrictions=permit_sasl_authenticated,permit_mynetworks,reject
-o milter_macro_daemon_name=ORIGINATING
#smtps inet n - y - - smtpd
# -o syslog_name=postfix/smtps
# -o smtpd_tls_wrappermode=yes
# -o smtpd_sasl_auth_enable=yes
```
The only thing I will make note of is that the `submission` service has a set of client
restrictions. In other words, to be able to use this service, the client must either be SASL
authenticated, or from a list of network prefixes that are allowed to relay. If neither of those two
conditions are satisfied, relaying will be denied.
Now, I understand that pasting in the entire postfix configuration is a bit verbose, but honestly
I've spent many an hour trying to puzzle together an end-to-end valid configuration, so I'm just
going to swim upstream and post the whole `main.cf`, which I'll try to annotate the broad strokes
of, in case there's anybody out there trying to learn:
```
myhostname = smtp-out.ipng.ch
myorigin = smtp-out.ipng.ch
mydestination = $myhostname, smtp-out.chrma0.net.ipng.ch, localhost.net.ipng.ch, localhost
mynetworks = 127.0.0.0/8, [::1]/128
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
biff = no
# appending .domain is the MUA's job.
append_dot_mydomain = no
readme_directory = no
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on fresh installs.
compatibility_level = 3.6
# SMTP Server
smtpd_banner = $myhostname ESMTP $mail_name (smtp-out.chrma0.net.ipng.ch)
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_tls_cert_file = /etc/certs/ipng.ch/fullchain.pem
smtpd_tls_key_file = /etc/certs/ipng.ch/privkey.pem
smtpd_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtpd_tls_CApath = /etc/ssl/certs
smtpd_use_tls = yes
smtpd_tls_received_header = yes
smtpd_tls_auth_only = yes
smtpd_tls_session_cache_database = btree:$data_directory/smtpd_scache
smtpd_client_connection_count_limit = 4
smtpd_client_connection_rate_limit = 10
smtpd_client_message_rate_limit = 60
smtpd_client_event_limit_exceptions = $mynetworks
# Dovecot auth
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_authenticated_header = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous, noplaintext
smtpd_sasl_tls_security_options = noanonymous
# SMTP Client
smtp_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtp_tls_cert_file = /etc/certs/ipng.ch/fullchain.pem
smtp_tls_key_file = /etc/certs/ipng.ch/privkey.pem
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_CApath = /etc/ssl/certs
smtp_tls_mandatory_ciphers = medium
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_security_level = encrypt
header_size_limit = 4096000
message_size_limit = 52428800
mailbox_size_limit = 0
# OpenDKIM, Rspamd
smtpd_milters = inet:localhost:8891,inet:rspamd.net.ipng.ch:11332
non_smtpd_milters = $smtpd_milters
# Local aliases
alias_maps = hash:/etc/postfix/aliases
alias_database = hash:/etc/postfix/aliases
```
***Hostnames***: The full (internal) hostname for the server is `smtp-out.$(site).net.ipng.ch`, in
this case for `chrma0` in R&uuml;mlang, Switzerland. However, when clients connect to the public hostname
`smtp-out.ipng.ch`, they will expect that the TLS certificate matches _that_ hostname. This is why I
let the server present itself as simply `smtp-out.ipng.ch`, which will also be its public DNS name
later, but put the internal FQDN for debugging purposes between parenthesis. See the `smtpd_banner`
and `myhostname` for the destinction. I'll load up the `*.ipng.ch` wildcard certificate which I
described in my Let's Encrypt [[DNS-01]({{< ref "2023-03-24-lego-dns01" >}})] article.
***Authorization***: I will make Postfix accept relaying for those users that are either in the
`mynetworks` (which is only localhost) OR `sasl_authenticated` (ie. presenting a username and
password). This password exchange will only be possible after encryption has been triggered using
the `STARTTLS` SMTP feature. This way, user/pass combos will be safe on the network.
***Authentication***: Those username and password combos can come from a few places. One popular way
to do this is via a `dovecot` authentication service. Via the `smtpd_sasl_path`, I tell Postfix to
ask these authentication questions using the dovecot protocol on a certain file path. I'll let
Dovecot listen in the `/var/spool/postfix/private/auth` directory. This is how Postfix will know
which user to relay for, and which to deny.
***DKIM/SPF***: These days, most (large and small) mail providers will be suspicious of e-mail that is
delivered to them without proper SPF and DKIM fields. ***DKIM*** is a mechanism to create a cryptographic
signature over some of the E-Mail header fields (usually From/Subject/Date), which can be checked
by the recipient for validity. ***SPF*** is a mechanism to use DNS to inform receiving mailservers
of which are the valid IPv4/IPv6 addresses that should be used to deliver mail for a given sender
domain.
#### Dovecot (auth)
The configuration for Dovecot is incredibly simple. The only thing I do is create a mostly empty
`dovecot.conf` file which defines the `auth` service listening in the place where Postfix expects
it. Then, I add a password file called `sasl-users` which will contain user:password tuples:
```
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0660
# Assuming the default Postfix user and group
user = postfix
group = postfix
}
}
passdb {
driver = passwd-file
args = username_format=%n /etc/dovecot/sasl-users
}
```
I can use `doveadm pw` to generate such passwords. I do this in an upstream Ansible repository and
then push out the same configuration to any number of `smtp-out` servers, so they are all configured
identically to this one.
#### OpenDKIM (signing)
Now that I can authorize (via SASL) and authenticate (via Dovecot backend) a user, it will be
entitled to use the `smtp-out` Postfix to send e-mail. However, there's a good chance that
recipients will bounce the e-mail, unless it comes with a DKIM signature, and from the _correct_ IP
addresses.
To configure DKIM signing, I use OpenDKIM, which I give the following `/etc/opendkim.conf` file:
```
pim@smtp-out-chrma0:~$ cat /etc/opendkim.conf
Syslog yes
LogWhy yes
UMask 007
Mode sv
AlwaysAddARHeader yes
SignatureAlgorithm rsa-sha256
X-Header no
KeyTable refile:/etc/opendkim/keytable
SigningTable refile:/etc/opendkim/signers
RequireSafeKeys false
Canonicalization relaxed
TrustAnchorFile /usr/share/dns/root.key
UserID opendkim
PidFile /run/opendkim/opendkim.pid
Socket inet6:8891
```
It opens a socket at port 8891, which is where Postfix expects it, based on its `smtpd_milter`
configuration option. It will look at the so-called **SigningTable** to determine which outbound
e-mail addresses it can sign. This table looks up From addresses, including wildcards, and informs
which symbolic keyname in the **KeyTable** to use for the signature, like so:
```
pim@smtp-out-chrma0:/etc/opendkim$ cat signers
*@*.ipng.nl ipng-nl
*@ipng.nl ipng-nl
*@*.ipng.ch ipng-ch
*@ipng.ch ipng-ch
*@*.ublog.tech ublog
*@ublog.tech ublog
...
pim@smtp-out-chrma0:/etc/opendkim$ cat keytable
ipng-nl ipng.nl:DKIM2022:/etc/opendkim/keys/DKIM2022-ipng.nl-private
ipng-ch ipng.ch:DKIM2022:/etc/opendkim/keys/DKIM2022-ipng.ch-private
ublog ublog.tech:DKIM2022:/etc/opendkim/keys/DKIM2022-ublog.tech-private
...
```
This allows OpenDKIM to sign messages for any number of domains, using the correct key. Slick!
#### NGINX
Now that I have three of these identical VMs, I am ready to hook them up to the internet. On the way
in, I will point `smtp-out.ipng.ch` to our NGINX cluster. I wrote about that cluster in a [[previous
article]({{< ref "2023-03-17-ipng-frontends" >}})]. I will add a snippet there, that exposes these
VMs behind a TCP loadbalancer like so:
```
pim@squanchy:~/src/ipng-ansible/roles/nginx/files/streams-available$ cat smtp-out.ipng.ch.conf
upstream smtp_out {
server smtp-out.chrma0.net.ipng.ch:587 fail_timeout=10s max_fails=2;
server smtp-out.frggh0.net.ipng.ch:587 fail_timeout=10s max_fails=2 backup;
server smtp-out.nlams2.net.ipng.ch:587 fail_timeout=10s max_fails=2 backup;
}
server {
listen [::]:587;
listen 0.0.0.0:587;
proxy_pass smtp_out;
}
```
I make use of the `backup` keyword, which will make the loadbalancer choose, if it's available, the
primary server in `chrma0`. If it were to go down, no problem, two connection failures within ten
seconds will make NGINX choose the alternative ones in `frggh0` or `nlams2`.
#### IPng Site Local gateway
When the `smtp-out` server receives the e-mail from the customer/client, it'll spool it and start to
deliver it to the remote MX record. To do this, it'll create an outbound connection from its cozy
spot within IPng Site Local (which, you will remember, is not connected directly to the internet).
There are three redundant gateways in IPng Site Local (in Geneva, Br&uuml;ttisellen and Amsterdam).
If any of these were to go down for maintenance or fail, the network will use OSPF E1 to find the
next closest default gateway. I wrote about how this entire european network is connected via three
gateways that are self-repairing in this [[article]({{< ref "2023-03-11-mpls-core" >}})], in case
you're curious.
But, for the purposes of SMTP, it means that each of the internal `smtp-out` VMs will be seen by
remote mailservers as NATted via ***one of these egress points***. This allows me to determine the
SPF records in DNS. With that, I'm ready to share the publicly visible details for this service:
```
_spf.ipng.ch. 3600 IN TXT "v=spf1 include:_spf4.ipng.ch include:_spf6.ipng.ch ~all"
_spf4.ipng.ch. 3600 IN TXT "v=spf1 ip4:46.20.246.112/28 ip4:46.20.243.176/28 ip4:94.142.245.80/29"
"ip4:94.142.241.184/29 ip4:194.1.163.0/24 ~all"
_spf6.ipng.ch. 3600 IN TXT "v=spf1 ip6:2a02:2528:ff00::/40 ip6:2a02:898:146::/48"
"ip6:2001:678:d78::/48 ~all"
smtp-out.ipng.ch. 3600 IN CNAME nginx0.ipng.ch.
nginx0.ipng.ch. 600 IN A 194.1.163.151
nginx0.ipng.ch. 600 IN A 46.20.246.124
nginx0.ipng.ch. 600 IN A 94.142.241.189
nginx0.ipng.ch. 600 IN AAAA 2001:678:d78:7::151
nginx0.ipng.ch. 600 IN AAAA 2a02:2528:ff00::124
nginx0.ipng.ch. 600 IN AAAA 2a02:898:146::5
```
To re-iterate one point: the _inbound_ path of the mail is via the redundant cluster of `nginx0`
entrypoints, while the _outbound_ path will be seen from `gw0.chbtl0.ipng.ch`, `gw0.chplo0.ipng.ch`
or `gw0.nlams3.ipng.ch`, which are all covered by the SPF records for IPv4 and IPv6.
#### Bonus: opensmtpd on clients
By the way, every single server (VM, hypervisor, router) at IPng Neworks will all use smtp-out to
send e-mail. I use `opensmtpd` for that, and it's incredibly simple:
```
pim@squanchy:~$ cat /etc/mail/smtpd.conf
table aliases file:/etc/mail/aliases
table secrets file:/etc/mail/secrets
listen on localhost
action "local_mail" mbox alias <aliases>
action "outbound" relay host "smtp+tls://ipng@smtp-out.ipng.ch:587" auth <secrets> mail-from "@ipng.ch"
match from local for local action "local_mail"
match from local for any action "outbound"
pim@squanchy:~$ sudo cat /etc/mail/secrets
ipng bastion:<haha-made-you-look>
```
{{< image width="120px" float="left" src="/assets/shared/lightbulb.svg" alt="Lightbulb" >}}
What happens here is, every time this server `squanchy` wants to send an e-mail, it will use an SMTP
session with TLS, on port 587, of the machine called `smtp-out.ipng.ch`, and it'll authenticate
using the opensmtpd realm called `ipng`, which maps to a username:password tuple in the _secrets_
file. It will also rewrite the envelope to be always from `@ipng.ch`. As a best practice I organize
my SMTP users by Ansible group. Squanchy is in the group `bastion`, hence its username. By doing it
this way, I can make use of the DKIM and SPF, which makes all mails properly formatted, routed,
signed and delivered. I love it, so much!
### 2. Inbound: smtp-in
The `smtp-out` service I described in the previous section is completely standalone. That is to say,
its purpose is only to receive submitted mail from humans and servers, sign it, spool it if need be,
and deliver it. But users also want to deliver e-mail to me and my customers. For this, I'll build a
second cluster of redundant _inbound_ mailservers: `smtp-in`.
Here, the base setup is not too different from above, so I won't repeat it. I'll take three
identical VMs, in three different datacenters, and install them with Debian and Postfix as well.
But, contrary to the outbound servers, here I will make them listen to the _smtp_ port 25 and the
_smtps_ port 465, and I'll turn off the ability to authenticate with SASL (and thereby, refuse to
forward any e-mail that I'm not the MX record for), making `master.cf` look like this:
```
smtp inet n - y - - smtpd
-o smtpd_sasl_auth_enable=no
#submission inet n - y - - smtpd
# -o syslog_name=postfix/submission
# -o smtpd_tls_security_level=encrypt
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_reject_unlisted_recipient=no
# -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mynetworks,reject
# -o milter_macro_daemon_name=ORIGINATING
smtps inet n - y - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=no
```
Many of the `main.cf` attributes are the same, unsurprisingly the `myhostname` configuration option
is set to `smtp-in.ipng.ch`, which is going to be expected to match the wildcard SSL certificate
from the `smtpd_tls_cert_file` config option. The banner is a bit more telling, as it shows also the
FQDN hostname (eg. `smtp-in.frggh0.net.ipng.ch`), helpful when debugging.
```
# Impose DNSBL restrictions at SMTP time
smtpd_recipient_restrictions = permit_mynetworks,
reject_invalid_helo_hostname,
reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
reject_unauth_pipelining,
reject_unauth_destination,
reject_rbl_client zen.spamhaus.org=127.0.0.[2..11],
reject_rhsbl_sender dbl.spamhaus.org=127.0.1.[2..99],
reject_rhsbl_helo dbl.spamhaus.org=127.0.1.[2..99],
reject_rhsbl_reverse_client dbl.spamhaus.org=127.0.1.[2..99],
warn_if_reject reject_rbl_client zen.spamhaus.org=127.255.255.[1..255],
reject_rbl_client dnsbl-1.uceprotect.net,
reject_rbl_client bl.0spam.org=127.0.0.[7..9],
permit
# Milter for rspamd
smtpd_milters = inet:rspamd.net.ipng.ch:11332
milter_default_action = accept
# PostSRSd
sender_canonical_maps = tcp:localhost:10001
sender_canonical_classes = envelope_sender
recipient_canonical_maps = tcp:localhost:10002
recipient_canonical_classes= envelope_recipient,header_recipient
# Virtual domains
virtual_alias_domains = hash:/etc/postfix/virtual-domains
virtual_alias_maps = hash:/etc/postfix/virtual
```
The config arguably is quite compact. but I will hilight four specific pieces.
***DNSBL***: When connecting and receiving the envelope (ie. `MAIL FROM` and `RCPT TO` in the SMTP
transaction), I'll ask Postfix to do a bunch of DNS blocklist lookups. Many sender domains, and
infected hosts/networks are mapped in public DNSBL zones, notably
[[Spamhaus](https://www.spamhaus.org/)], [[UCEProtect](http://www.uceprotect.net/en/index.php)], and
[[0Spam](https://0spam.org/)] do a great job at identifying malicious and spammy domain-names and
networks. So I'll tell Postfix to reject folks attempting to connect from these low-reputation
places.
***Rspamd***: Here's where I hook up a redundant cluster of rspamd servers. Each e-mail, once
accepted, will be routed through this milter, and after thinking about it a little bit, the Rspamd
server will either answer:
* ***greylisted***: where Rspamd recommends a _tempfail_ so the remote mailserver comes back
after a few minutes after connecting for the first time, many spammers will not do this.
* ***blocked***: if Rspamd finds the e-mail is egregious, it'll simply recommend a _permfail_
so Postfix immediately rejects it.
* ***tagged***: if Rspamd is iffy about the e-mail, it may insert an `X-Spam` header, so that
downstream mail clients like Thunderbird or Mail.app can decide for themselves to consider the
e-mail junk or not.
***PostSRS***: This is a really useful feature which allows Postfix to safely _forward_ an e-mail to
another mailhost. Perhaps best explained with an example, notably the aforementioned nerdsnipe from
my buddy Jelle:
1. Let's say `jelle@luteijn.email` sends an e-mail to `event@frys-ix.net` for which IPng is the
mailhost.
1. Jelle configured his SPF records to allow mail to come from either `ip4:185.36.229.0/24`
or `ip6:2a07:cd40::/29`, and if it comes from neither of those, to hard fail the SPF check
`-all`.
1. My spiffy `smtp-in.ipng.ch` receives this e-mail and decides to forward it internally by
rewriting it to `foo@eritap.com`.
1. The mailserver for `eritap.com` now sees an e-mail coming From: `jelle@luteijn.email` going to
its `foo@eritap.com`. It does an SPF check and concludes: Yikes! That mailserver
`smtp-in.ipng.ch` is NOT authorized to send e-mail on behalf of Jelle, so reject it! A kitten
gets hurt, which is obviously unacceptable.
To handle this, PostSRSd detects when such a forward is about to happen, and rewrites the envelope
From: header to be something that `smtp-in.ipng.ch` might be allowed to deliver mail for: something
in the `@ipng.ch` domain! Using a secret (shared between the replicas of IPng's `smtp-in` cluster),
it can insert a little cryptographic signature as it does this rewrite.
In the example above, the e-mail from `jelle@luteijn.email` will be rewritten to an envelope such as
`SRS0=CCIM=MT=luteijn.email=jelle@ipng.ch` and while hideous, it **is** in the `@ipng.ch` domain. If
a bounce for this e-mail were to be generated, PostSRSd can also rewrite in reverse, re-assembling
the original envelope From when sending the bounce on to Jelle's mailserver.
I configure Postfix to do this using the _sender_ and _recipient_ canonical maps. I read these from
a server running on localhost port 10001 and 10002 respectively. This is where PostSRSd does its
magic.
Oh, what's that I hear? The telephone is ringing! 1982 called, and it wants to change the title of
[[RFC821](https://datatracker.ietf.org/doc/html/rfc821)] from SMTP to CMTP (**Convoluted Mail
Transfer Protocol**).
***Virtual***: With all of that out of the way, I can now receive and forward aliased e-mails. I
won't be using local mail delivery (to unix users on the local machine), but rather I will forward
the mails for my local users onwards to what is called a redundant `maildrop` server. So for the
virtualized part of the Postfix config, I have things like this:
```
pim@smtp-in-chrma0:~$ cat /etc/postfix/virtual-domains
ublog.tech ublog.tech
frys-ix.net frys-ix.net
ipng.nl ipng.nl
ipng.ch ipng.ch
...
pim@smtp-in-chrma0:~$ cat /etc/postfix/virtual
## Virtual domain: ipng.ch
postmaster@ipng.ch pim+postmaster@maildrop.net.ipng.ch
hostmaster@ipng.ch pim+hostmaster@maildrop.net.ipng.ch
abuse@ipng.ch pim+abuse@maildrop.net.ipng.ch
pim@ipng.ch pim+ipng@maildrop.net.ipng.ch
noreply@ipng.ch /dev/null
...
## Virtual domain: ipng.nl
@ipng.nl @ipng.ch
## Virtual domain: frys-ix.net
postmaster@frys-ix.net pim+postmaster@maildrop.net.ipng.ch
hostmaster@frys-ix.net pim+hostmaster@maildrop.net.ipng.ch
abuse@frys-ix.net pim+abuse@maildrop.net.ipng.ch
noc@frys-ix.net pim+frysix@maildrop.net.ipng.ch,noc@eritap.com
pim@frys-ix.net pim+frysix@maildrop.net.ipng.ch
arend@frys-ix.net arend+frysix@eritap.com
event@frys-ix.net someplace@example.com
...
```
The first file here `virtual_alias_domains`, simply explains to Postfix which domains it is to
accept e-mail for. This avoids users trying to use it as a relay. If the domain is not listed in
the lefthand side of the table, it's not welcome here. But then once Postfix knows it's supposed to
be accepting e-mail for this domain, it will consult the `virtual_alias_maps` configuration. Here, I
showed three domains, and a few features:
* I can simply forward along `pim@ipng.ch` to `pim+ipng@maildrop.net.ipng.ch`. Cool.
* I can toss the email by passing it to `/dev/null` (useful for things like `noreply@` and
`nobody@`)
* I can forward it to multiple recipients as well, for example `noc@frys-ix.net` goes to me and
Eritap (hoi, Arend!)
When such a forward happens, PostSRSd kicks in, and for that e-mail, the envelope rewrite will
happen such that `smtp-in` can safely deliver this to even the strictest of SPF users.
#### Why no NGINX ?
There's an important technical reason for me not to be able to use an inbound loadbalancer, even
though I'd love to frontend port 25 and 465 on IPng's nginx cluster. I have enabled the use of
DNSBL, which implies that Postfix needs to know the remotely connecting IPv4 and IPv6 addresses.
While for domain-based blocklists this is not important, for IP based ones like _zen.spamhaus.org_
it is critical. Therefore, I will assign a public IPv4 and IPv6 address to each of the machines in
the cluster. They will be used in a round-robin way, and if one of them is down for a while, remote
mail servers will automatically and gracefully use another replica.
With that, the public DNS entries:
```
ublog.tech. 86400 IN MX 10 smtp-in.ipng.ch.
ipng.nl. 86400 IN MX 10 smtp-in.ipng.ch.
ipng.ch. 86400 IN MX 10 smtp-in.ipng.ch.
smtp-in.ipng.ch. 60 IN A 46.20.246.125
smtp-in.ipng.ch. 60 IN A 94.142.245.85
smtp-in.ipng.ch. 60 IN A 194.1.163.141
smtp-in.ipng.ch. 60 IN AAAA 2a02:2528:ff00::125
smtp-in.ipng.ch. 60 IN AAAA 2a02:898:146:1::5
smtp-in.ipng.ch. 60 IN AAAA 2001:678:d78:6::141
```
### 3. Dovecot: maildrop
Remember when I said that mail to `pim@ipng.ch` is forwarded to `pim+ipng@maildrop.net.ipng.ch`?
Doing this allows me to have replicated, fully redundant, IMAP servers! As it turns out, Dovecot, a
very popular open source pop3/imap server, has the ability to do realtime synchronization between
multiple machines serving the same user.
On these servers, I'll start with enabling Postfix only using the `smtp` and `smtps` transport in
`master.cf`. The maildrop servers will be entirely within IPng Site Local, and cannot be reached
from the internet directly, just the same as the `smtp-out` server replicas.
Postfix on the server receives mail from the `smtp-in` servers as the final destination for an
e-mail. It does this very similar to the `smtp-in` server pool I described above, with two notable
differences:
1. It does not need to do DNSBL lookups or spam analysis -- those have already happened upstream
from these maildrop servers by the `smtp-in` servers. That's also why these can be safely
tucked away in IPng Site Local.
1. Their virtual maps point to what is called an ***LMTP***: Local Mail Transport Protocol, where
I'll ask Postfix to pump them into a redundalty replicated Dovecot pair.
```
# Completely virtual
virtual_alias_maps = hash:/etc/postfix/virtual-maildrop
virtual_mailbox_domains = maildrop.net.ipng.ch
virtual_transport = lmtp:unix:private/dovecot-lmtp
pim@maildrop0-chbtl0:$ cat /etc/postfix/virtual-maildrop
pim@maildrop.net.ipng.ch pim
```
What I've done here is define only one `virtual_mailbox_domains` entry, for which I look up the
users in the `virtual_alias_maps` and use a `virtual_transport` to deliver the enduser (`pim`) to a
unix domain socket in `/var/spool/postfix/private/dovecot-lmtp`. Once again, mail servers are super
simple after you've spent ten hours reading configuration manuals and RFCs and asked at least three
other people how they did theirs.... Super... Simple!
#### Dovecot
By default, Dovecot ships with a very elaborate configuration file hierarchy. I decide to replace it
with an autogenerated one from Ansible that has fewer includes (namely: none at all). Here's the
features that I want to enable in Dovecot:
* ***UserDB***: To define username, password and mail directory for users.
* ***LMTP***: To be a local recepticle for the Postfix delivery
* ***IMAP***: To serve SSL enabled IMAP to mail clients like Mail.app, Thunderbird, Roundcube,
etc.
* ***Replicator***: To replicate mailboxes between pairs of Dovecot servers.
* ***Sieve***: To allow users to create mail filters using the Sieve protocol.
Starting from the easier bits, here's how I configure the User Database in `dovecot.conf`:
```
passdb {
driver = passwd-file
args = username_format=%n /etc/dovecot/maildrop-users
}
userdb {
driver = passwd-file
args = username_format=%n /etc/dovecot/maildrop-users
default_fields = uid=vmail gid=vmail home=/var/dovecot/users/%n
}
mail_plugins = $mail_plugins notify push_notification replication
mail_location = mdbox:~/mdbox
```
I can add a user `pim` with an encrypted password from `doveadm pw` like so:
```
pim@maildrop0-chbtl0:/etc/dovecot$ sudo cat maildrop-users
...
pim:{CRYPT}$2y$<some encrypted password goes here>::::
```
Due to the `passdb` option, this user can authenticate with username and password, and due to the
`userdb` option, this user receives a mailbox homedir in the specified location. One important
observation is that the _unix_ user is `vmail:vmail` for every mailbox. This is pretty cool as it
allows the whole mail delivery system to be virtualized under Dovecot's guidance. Slick!
There are two tried-and-tested mailbox formats: Maildir and mbox. mbox is one giant file per mail
folder, and can be expensive to search and sort and delete mails out of. Maildir is cheaper to
search and sort and delete, but is essentially one file per e-mail, which is bulky. Dovecot has its
own high performance mailbox, which is the best of both worlds: an indexed append-only chunked mail
format called mdbox. I learned more about the options and trade offs reading [[this
doc](https://doc.dovecot.org/admin_manual/mailbox_formats/dbox/)].
#### Dovecot: LMTP
The following `dovecot.conf` snippet ties Postfix into Dovecot:
```
protocols = $protocols lmtp
protocol lmtp {
mail_plugins = $mail_plugins sieve
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
mode = 0660
user = postfix
group = postfix
}
}
```
Recall that in Postfix above, the `virtual_transport` field specified the same location. This is how
user `pim` gets mail handed to Dovecot. One other tidbit here is that the LMTP protocol enables a
plugin called `sieve`. What this does, is upon receipt of each e-mail, a list of filters is run
through, on the server side! It is here that I can tell Dovecot that some mail goes to different folders
and sub-folders, some might be forwarded, marked read or discarded entirely. I'll get to that in a
minute.
#### Dovecot: IMAP
Then, I enable SSL enabled IMAP in `dovecot.conf`:
```
disable_plaintext_auth = yes
auth_mechanisms = plain login
protocols = $protocols imap
protocol imap {
mail_max_userip_connections = 50
mail_plugins = $mail_plugins imap_sieve
}
service imap-login {
inet_listener imap {
port = 0 ## Disabled
}
inet_listener imaps {
port = 993
}
}
```
With this snippet, I instruct Dovecot to disable any plain-text authentication, and use either
`plain` or `login` challenges to authenticate users. I'll disable the un-encrypted IMAP listener
by setting its port to 0, and I'll allow for an IMAP+SSL listener on the common port 993, which will
be presenting a `*.ipng.ch` wildcard certificate that's shared between all sorts of services at
IPng.
#### Dovecot: Replication
And now for something really magical. Dovecot can be instructed to replicate in multi-master (ie.
read/write) mailboxes to remote machines also running Dovecot. This is called ***dsync*** and it's
hella cool! In reading the [[docs](https://doc.dovecot.org/configuration_manual/replication/)], I
take note that the same user should be directed to a stable replica in normal use, but changes do
not get lost even if the same user modifies mails simultaneously on both replicas, some mails just
might have to be redownloaded in that case. The replication is done by looking at Dovecot index
files (not what exists in filesystem), so no mails get lost due to filesystem corruption or an
accidental `rm -rf`, they will simply be replicated back.
This is amazing!! The configuration for it is remarkably straight forward:
```
mail_plugins = $mail_plugins notify replication
# Replication details
replication_max_conns = 10
replication_full_sync_interval = 1h
service aggregator {
fifo_listener replication-notify-fifo {
user = vmail
group = vmail
mode = 0660
}
unix_listener replication-notify {
user = vmail
group = vmail
mode = 0660
}
}
# Enable doveadm replicator commands
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
mode = 0660
user = vmail
group = vmail
}
}
doveadm_port = 63301
doveadm_password = <some password here>
service doveadm {
vsz_limit=512 MB
inet_listener {
port = 63301
}
}
plugin {
mail_replica = tcp:maildrop0.ddln0.net.ipng.ch
}
```
To try to explain this - The first service, the `aggregator` opens some notification FIFOs that will
notify listeners of new replication events. Then, Dovecot will start a process called `replicator`,
which gets these cues when there is work to be done. It will connect to a `mail_replica` on another
host, on the `doveadm_port` (in my case 63301) which is protected by a shared password. And with
that, all e-mail that is delivered via **LMTP** on this machine, is both retrievable via **IMAPS**
but also gets copied to the remote machine `maildrop0.ddln0.net.ipng.ch` (and in _its_
configuration, it'll synchronize mail to `maildrop0.chbtl0.net.ipng.ch`). Nice!
#### Dovecot: Sieve
Having a flat mailbox is just no fun (unless you're using GMail, in which case: tolerable). Enter
_Sieve_, described in [[RFC5228](https://datatracker.ietf.org/doc/html/rfc5228)]. Scripts written in
Sieve are executed during final delivery, when the message is moved to the user-accessible mailbox.
In systems where the Mail Transfer Agent (MTA) does final delivery, such as traditional Unix mail,
it is reasonable to filter when the MTA deposits mail into the user's mailbox.
```
protocols = $protocols sieve
plugin {
sieve = ~/.dovecot.sieve
sieve_global_path = /etc/dovecot/sieve/default.sieve
sieve_dir = ~/sieve
sieve_global_dir = /etc/dovecot/sieve/
sieve_extensions = +editheader
sieve_before = /etc/dovecot/sieve/before.d
sieve_after = /etc/dovecot/sieve/after.d
}
plugin {
sieve_plugins = sieve_imapsieve sieve_extprograms
# From elsewhere to Junk folder
imapsieve_mailbox1_name = Junk
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
# From Junk folder to elsewhere
imapsieve_mailbox2_name = *
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve
sieve_pipe_bin_dir = /etc/dovecot/sieve
sieve_global_extensions = +vnd.dovecot.pipe
}
```
This is a mouthful, but only because it's hella cool. By default, each mailbox will have a
`.dovecot.sieve` file that is consulted at each delivery. If no file exists there, the default sieve
will be used. But also, some sieve filters might happen either `sieve_before` the users' one is
called, or `sieve_after`. Then, in the plugin I create two specific triggers:
1. if a file is copied to the Junk folder, I will run it through a script called
`report-spam.sieve`.
1. similarly, if it is moved *out of* the Junk folder, I'll run the `report-ham.sieve` script.
Using an `rspamc` client, I can wheel over the cluster of Rspamd servers one by one and offer them
these two events (the train-spam and train-ham are similar, so I'll only show one):
```
pim@maildrop0-chbtl0:~$ cat /etc/dovecot/sieve/report-spam.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.email" "*" {
set "email" "${1}";
}
pipe :copy "train-spam.sh" [ "${email}" ];
pim@maildrop0-chbtl0:~$ cat /etc/dovecot/sieve/train-spam.sh
logger learning spam
/usr/bin/rspamc -h rspamd.net.ipng.ch:11332 learn_spam
```
Users of Dovecot can now add their Sieve configs to their mailbox:
```
pim@maildrop0-chbtl0:~$ sudo ls -la /var/dovecot/users/pim/
lrwxrwxrwx 1 vmail vmail 19 Apr 2 17:27 .dovecot.sieve -> sieve/ipng_v1.sieve
-rw------- 1 vmail vmail 2113 Mar 29 11:35 .dovecot.sieve.log
-rw------- 1 vmail vmail 5001 May 14 14:34 .dovecot.svbin
drwx------ 4 vmail vmail 4096 May 17 16:02 mdbox
drwx------ 3 vmail vmail 4096 May 14 14:30 sieve
```
but seeing as (a) it's tedious to have to edit these files on multiple dovecot replicas, and (b) my
users will not receive access to `vmail` user in order to actually do that, as it would be a
security risk, I need one more thing.
#### Dovecot: IMAP Sieve
Dovecot has an implementation of a replication-aware Sieve filter editor called `managesieve`:
```
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
service managesieve {
process_limit = 256
}
protocol sieve {
}
```
It will use the IMAP credentials to allow users to edit their _Sieve_ filter online. For example,
Thunderbird has a plugin for it, which does syntax checking and what-not. When the filter is edited,
it is syntax checked, compiled and replicated to the other Dovecot instance.
{{< image src="/assets/smtp/sieve_thunderbird.png" alt="Sieve Thunderbird" >}}
#### NGINX
I have an imap server and a mangesieve server, redundantly running on two Dovecot machines. I recall
reading in the Dovecot manual that it is slightly preferable to have users go to a consistent
replica and not bounce around between them. Luckily, I can do exactly that using the NGINX
frontends:
```
upstream imap {
server maildrop0.chbtl0.net.ipng.ch:993 fail_timeout=10s max_fails=2;
server maildrop0.ddln0.net.ipng.ch:993 fail_timeout=10s max_fails=2 backup;
}
server {
listen [::]:993;
listen 0.0.0.0:993;
proxy_pass imap;
}
upstream sieve {
server maildrop0.chbtl0.net.ipng.ch:4190 fail_timeout=10s max_fails=2;
server maildrop0.ddln0.net.ipng.ch:4190 fail_timeout=10s max_fails=2 backup;
}
server {
listen [::]:4190;
listen 0.0.0.0:4190;
proxy_pass sieve;
}
```
I keep port 993 for `maildrop` as well as port 587 for `smtp-out` unfiltered on the NGINX cluster.
I'm a little bit more protective of the `managesieve` service, so port 4190 is allowed only when
users are connected to the VPN or the internal (office/home) network.
Now, you'll recall that in the `smtp-in` servers, I forward mail to `pim@maildrop.net.ipng.ch`,
for which the redundant Dovecot servers are both accepting mail. On the way in, I can see to it that
the primary replica is used , by giving it a slightly lower preference in DNS MX records:
```
maildrop.net.ipng.ch. 300 IN MX 10 maildrop0.chbtl0.net.ipng.ch.
maildrop.net.ipng.ch. 300 IN MX 20 maildrop0.ddln0.net.ipng.ch.
imap.ipng.ch. 60 IN CNAME nginx0.ipng.ch.
nginx0.ipng.ch. 600 IN A 194.1.163.151
nginx0.ipng.ch. 600 IN A 46.20.246.124
nginx0.ipng.ch. 600 IN A 94.142.241.189
nginx0.ipng.ch. 600 IN AAAA 2001:678:d78:7::151
nginx0.ipng.ch. 600 IN AAAA 2a02:2528:ff00::124
nginx0.ipng.ch. 600 IN AAAA 2a02:898:146::5
```
This will make the `smtp-in` hosts prefer to use the `chbtl0` maildrop replica when it's available.
If ever it were to go down, they will automatically fail over and use `ddln0`, which will replicate
back any changes while `chbtl0` is down for maintenance or hardware failure. On the way to out, the
nginx cluster will prefer to use `chbtl0` as well, as it has marked the `ddln0` replica as `backup`.
### 4. Webmail: Roundcube
Now that I have all of the infrastructure up and running, I thought I'd put some icing on the cake
with Roundcube, a web-based IMAP email client. Roundcube's most prominent feature is the pervasive
use of Ajax technology. It also comes with an online Sieve editor, and runs in Docker. What more
can I ask for?
Installing it is really really easy in my case. Since I have an nginx cluster to frontend it and do
the SSL offloading, I choose the simplest version with the following `docker-compose.yaml`:
```
version: '2'
services:
roundcubemail:
image: roundcube/roundcubemail:latest
container_name: roundcubemail
volumes:
- ./www:/var/www/html
- ./db/sqlite:/var/roundcube/db
ports:
- 9002:80
environment:
- ROUNDCUBEMAIL_DB_TYPE=sqlite
- ROUNDCUBEMAIL_SKIN=elastic
- ROUNDCUBEMAIL_DEFAULT_HOST=ssl://maildrop0.net.ipng.ch
- ROUNDCUBEMAIL_DEFAULT_PORT=993
- ROUNDCUBEMAIL_SMTP_SERVER=tls://smtp-out.net.ipng.ch
- ROUNDCUBEMAIL_SMTP_PORT=587
```
There's a small snag, in that by default the SMTP user and password are expected to be the same as
for the IMAP server, which is not the case for my design. So, I create a user `roundcube` on the
`smtp-out` cluster and give it a suitable password. I nose around a little bit, and decide my
preference is to have _threaded_ view by default, and I also enable the `managesieve` plugin:
```
$config['log_driver'] = 'stdout';
$config['zipdownload_selection'] = true;
$config['des_key'] = '<this key of sorts>';
$config['enable_spellcheck'] = true;
$config['spellcheck_engine'] = 'pspell';
$config['smtp_user'] = 'roundcube';
$config['smtp_pass'] = '<something or other>';
$config['plugins'] = array('managesieve');
$config['managesieve_host'] = 'tls://maildrop0.net.ipng.ch:4190';
$config['default_list_mode'] = 'threads';
```
I start the docker containers, and very quickly after, Roundcube shoots to life. I can expose it
behind the nginx cluster, while keeping it accessible only for VPN + office/home network users:
```
server {
listen [::]:80;
listen 0.0.0.0:80;
server_name webmail.ipng.ch webmail.net.ipng.ch webmail;
access_log /var/log/nginx/webmail.ipng.ch-access.log;
include /etc/nginx/conf.d/ipng-headers.inc;
location / {
return 301 https://webmail.ipng.ch$request_uri;
}
}
geo $allowed_user {
default 0;
include /etc/nginx/conf.d/geo-ipng.inc;
}
server {
listen [::]:443 ssl http2;
listen 0.0.0.0:443 ssl http2;
ssl_certificate /etc/certs/ipng.ch/fullchain.pem;
ssl_certificate_key /etc/certs/ipng.ch/privkey.pem;
include /etc/nginx/conf.d/options-ssl-nginx.inc;
ssl_dhparam /etc/nginx/conf.d/ssl-dhparams.inc;
server_name webmail.ipng.ch;
access_log /var/log/nginx/webmail.ipng.ch-access.log upstream;
include /etc/nginx/conf.d/ipng-headers.inc;
if ($allowed_user = 0) { rewrite ^ https://ipng.ch/ break; }
location / {
proxy_pass http://docker0.frggh0.net.ipng.ch:9002;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
The configuration has one neat trick in it -- it uses the `geo` module in NGINX to assert that the
client address is used to set the value of `allowed_user`. It will be 1 if the client connected from
any network defined in the `geo-ipng.inc` file, and 0 otherwise. I then use it to bounce unwanted
visitors back to the main [[website](https://ipng.ch/)], and expose Roundcube for those that are
welcome.
While the Roundcube instance is not replicated, it's also non-essential. I will be using
Thunderbird, Mail.app and other clients more regularly than Roundcube. It may just be handy in a
pinch to either check mail using a browser, but also to edit the _Sieve_ filters easily.
In my defense, considering roundcube is pretty much stateless, I can actually just run multiple
copies of it on a few docker hosts at IPng -- then in the nginx configs I might use a similar
construct as for the `maildrop` and `smtp-out` services, with a primary and hot standby. But that
will be for that one day that the docker host in Lille dies AND I decided I absolutely require
Roundcube precisely on that day :)
Reference in New Issue View Git Blame Copy Permalink