Repository files navigation

This repository hosts code for two docker imagesmaxking/mailman-core andmaxking/mailman-web both of which are meant to deployGNU Mailman 3 ina production environment.

Docker is a container ecosystem which can run containers on severalplatforms. It consists of a tool calleddocker-compose which can be used torun multi-container applications. This repository consists of adocker-compose.yaml file which is a set ofconfigurations that can be used to deploy theMailman 3 Suite.

Please seerelease pagefor the releases and change log.

The tags for the images are assumed to be release versions for images. This isgoing to be a somewhat common philosophy of distributing Container images wherethe images with same tags are usually updated with the new functionality.

Releases will follow the following rules:

• Images tagged like A.B.C will never change. If you want to pin down versionsof Images, use these tags.

• Images tagged with A.B will correspond to the latest A.B.C versionreleased. Releases in A.B series are supposed to be backwards compatible,i.e., any existing installation should not break when upgrading betweensubversions of A.B.C. So, if you want the latest updates and want tofrequently update your installation without having to change the versionnumbers, you can use this.

• Any changes in the minor version of Mailman components of the images willcause a bump in the minor version, e.g., A.(B+1) can have one or moreupdated Mailman components from A.B. Also, significant change in functionality,that might change how Images work or how people interact with the containerscan also cause a bump in the minor version.

• Major versions will change either when there are backwards incompatiblechanges or when the releases reach a certain set milestone or when there arebugfix releases for the internal components or both.

The container images are available from multiple container registries. Do specify anexplicit version tag (e.g.0.4.5 , MAJOR.MINOR like0.4 also works as floating tag pointing to latest patch version) as taglatest isnot updated anymore.

• ghcr.io/maxking/mailman-core
• docker.io/maxking/mailman-core

• ghcr.io/maxking/mailman-web
• docker.io/maxking/mailman-web

• ghcr.io/maxking/postorius
• docker.io/maxking/postorius

Rolling releases are made up of Mailman Components installed fromgitsource.Note that these releases are made up ofun-released software and should be assumed to be beta quality.

Every commit is tested with Mailman's CI infrastructure and is included inrolling releases only if they have passed the complete test suite.

$ docker pull docker.io/maxking/mailman-web:rolling$ docker pull docker.io/maxking/mailman-core:rolling

Rolling releases are built with every commit and also re-generated nightly. Youcan inspect the images to get which commit it was built using:

$ docker inspect --format'{{json .Config.Labels }}' mailman-core| python -m json.tool{"version.git_commit":"45a4d7805b2b3d0e7c51679f59682d64ba02f05f",}$ docker inspect --format'{{json .Config.Labels }}' mailman-web| python -m json.tool{"version.git_commit":"45a4d7805b2b3d0e7c51679f59682d64ba02f05f",}

• version.git_commit : This is the commit hash of the Dockerfile in theGithub repo

• Docker
• Docker-compose-plugin

To install these on Ubuntu/Debian:

$ sudo apt install docker.io docker-compose-plugin

For other systems, you can read the official Docker documentation to installDocker from here anddocker compose from here.

Most of the common configuration is handled through environment variables in thedocker-compose.yaml. However, there is need for some extra configuration thatinteracts directly with the application. There are two configuration files onthe host that interact directly with Mailman's settings. These files exist onthe host running the containers and are imported at runtime in the containers.

• /opt/mailman/core/mailman-extra.cfg : This is the configuration for MailmanCore and anything that you add here will be added to Core's configuration. Youneed to restart your mailman-core container for the changes in this file totake effect.

• /opt/mailman/web/settings_local.py : This is the Django configuration thatis imported by theexisting configurationprovided by the mailman-web container.This file is referred to assettings.py in most of the Postorius and Django documentation. To changeor override any settings in Django/Postorius, you need to create/edit this file.A useful configuration for troubleshooting isDEBUG = True.

Also, note that if you need any other files to be accessible from the host toinside the container, you can place them at certain directories which aremounted inside the containers.

• /opt/mailman/core in host maps to/opt/mailman/ in mailman-core container.
• /opt/mailman/web in host maps to/opt/mailman-web-data in mailman-webcontainer.

These are the settings that you MUST change in your docker-compose.yaml before deploying:

• SERVE_FROM_DOMAIN: The domain name from which Django will be served. To beadded toALLOWED_HOSTS in django settings. Default value is not set. Thisalso replaces Django's defaultexample.com SITE and becomes the default SITE(with SITE_ID=1).

• HYPERKITTY_API_KEY: Hyperkitty's API Key, should be set to the same valueas set for the mailman-core. (Not needed in case of Postorius-only version.)

• MAILMAN_ADMIN_USER: The username for the admin user to be created by default.

• MAILMAN_ADMIN_EMAIL: The email for the admin user to be created by default.

• SECRET_KEY: Django's secret key, mainly used for signing cookies and others.

Please note here that if you choose to create the admin user using theenvironment variables mentioned above (MAILMAN_ADMIN_USER &MAILMAN_ADMIN_EMAIL), no password is set for your admin account. To set apassword, plese follow the "Forgot Password" link on the "Sign In" page.

Mailman web is already configured to send emails through$SMTP_HOST as theMTA's address. If you want to modify it, you can set the value in underdocker-compose.yaml for mailman-web container. By default,SMTP_HOST pointsto the gateway of the web container, which is the host itself.

You can also use the environment variablesSMTP_HOST (defaults tothe container's gateway),SMTP_PORT (defaults to25),SMTP_HOST_USER (defaults toan empty string),SMTP_HOST_PASSWORD (defaults to an empty string),SMTP_USE_TLS (defaults toFalse) andSMTP_USE_SSL (defaults toFalse).

This is required in addition to theSetup your MTAsection below, which covers email setup for Mailman Core.

For more details on how to configure this image, please look atMailman-web's Readme

These are the variables that you MUST change in your docker-compose.yaml before deploying:

• HYPERKITTY_API_KEY: Hyperkitty's API Key, should be set to the same value asset for the mailman-web. Skip the variable in case of non-Hyperkitty deployment.

• DATABASE_URL: URL of the typedriver://user:password@hostname:port/databasename for the django to use. Ifnot set, the default is set tosqlite:///opt/mailman-web-data/mailmanweb.db. The standarddocker-compose.yaml comes with it set to a postgres database. There is no needto change this if you are happy with PostgreSQL.

• DATABASE_TYPE: Its value can be one ofsqlite,postgres ormysql asthese are the only three database types that Mailman 3 supports. Its defaultvalue is set tosqlite along with the default database class and defaultdatabase url above.

• DATABASE_CLASS: Default value ismailman.database.sqlite.SQLiteDatabase. The values for this can be found inthe mailman's documentationhere.

• SMTP_HOST: outgoing host for SMTP connections

• SMTP_PORT: use this port. 25, 587, whatever your host asks for.

• SMTP_HOST_USER: authenticate this user

• SMTP_HOST_PASSWORD: and use this password

• SMTP_SECURE_MODE: security mode for smtp connection - can besmtp (no encryption),smtps orstarttls

• SMTP_VERIFY_HOSTNAME: defaults totrue - verify, that certificate hostname is identical toSMTP_HOST

• SMTP_VERIFY_CERT: defaults totrue - verify, that certificate is valid

For more details on how to configure this image, please lookMailman-core'sReadme

While the above configuration will allow you to run the images and possibly viewthe Web Frontend, it won't be functional until it is fully configured to to sendemails.

To configure the mailman-core container to send emails, see theSetting your MTAsection below.

To run the containers, simply run:

$ mkdir -p /opt/mailman/core$ mkdir -p /opt/mailman/web$ git clone https://github.com/maxking/docker-mailman$cd docker-mailman# Change some configuration variables as mentioned above.$ docker compose up -d

Note that the web frontend in the mailman-web container is, by default, onlyconfigured to serve dynamic content. Anything static like stylesheets, etc., isexpected to be served directly by the web server. The static content exists at/opt/mailman/web/static and should bealiased to/static/ in the webserver configuration.

Seethe nginx configuration as an example.

This command will do several things, most importantly:

• Run a wsgi server usinguwsgi for the Mailman's Django-based webfrontend listening on port 8000. It will run 2 workerprocesses with 4 threads each. You may want to change the settingALLOWED_HOSTS in the settings before deploying the application inproduction.

• Run a PostgreSQL server with a default database, username, and password asmentioned in thedocker-compose.yaml. You will have to change configurationfiles too if you change any of these.

• Run mailman-core listening on port 8001 for REST API and port 8024 (LMTPserver) for messages from your MTA. You will have to configure your MTA tosend messages at this address.

Some more details about what the above system achieves is mentioned below. If youare only going to deploy a simple configuration, you don't need to readthis. However, these are very easy to understand if you know how docker works.

• First create a bridge network calledmailman in thedocker-compose.yaml. It will probably be named something else in yourmachine. All the containersmentioned (mailman-core, mailman-web, database) will join this network and areassigned static IPs. The host operating system is the default gatewayfrom within these containers.

• Spin off a mailman-core container attached to the mailman bridge network created above. It hasGNU Mailman 3 core running inside it. Mailman core's REST API is available atport 8001 and LMTP server listens at port 8024.

• Spin off a mailman-web container which has a Django application running withboth Mailman's web frontend Postorius and Mailman's web-based Archiverrunning.Uwsgi server is used to run a web server with the configurationprovided in this repositoryhere. You may want tochange the settingALLOWED_HOSTS in the settings before deploying theapplication in production. You can do that by adding a/opt/mailman/web/settings_local.py which is imported by the Django whenrunning.

• Spin off a PostgreSQL database container which is used by both mailman-coreand mailman-web as their primary database.

• mailman-core mounts/opt/mailman/core from host OS at/opt/mailman in thecontainer. Mailman's var directory is stored there so that it is accessiblefrom the host operating system. Configuration for Mailman core is generated onevery run from the environment variables provided. Extra configuration canalso be provided at/opt/mailman/core/mailman-extra.cfg (on host), and willbe added to generated configuration file. Mailman also needs anotherconfiguration file calledmailman-hyperkitty.cfg and is alsoexpected to be at/opt/mailman/core/ on the host OS.

• mailman-web mounts/opt/mailman/web from the host OS to/opt/mailman-web-data in the container. It consists of the logs andsettings_local.py file for Django.

• database mounts/opt/mailman/database at/var/lib/postgresql/data so thatPostgreSQL can persist its data even if the database containers areupdated/changed/removed.

The provided docker containers do not have an MTA in-built. You can either runyour own MTA inside a container and have them relay emails to the mailman-corecontainer or just install an MTA on the host and have them relay emails.

To useExim4, it should be setup to relay emails from mailman-core andmailman-web. The mailman specific configuration is provided in therepository atcore/assets/exim. There are three files

• 25_mm3_macros to be placed at/etc/exim4/conf.d/main/25_mm3_macros in a typical Debian install ofexim4. Please change MY_DOMAIN_NAME to the domain name that will be used toserve mailman. Multi-domain setups will be added later.

• 455_mm3_router to be placed at/etc/exim4/conf.d/router/455_mm3_router in a typical Debian install of exim4.

• 55_mm3_transport to be placed at/etc/exim4/conf.d/transport/55_mm3_transport in a typical Debian install of exim4.

Also, the default configuration inside the mailman-core image has the MTA setto Exim, but just for reference, it looks like this:

# mailman.cfg[mta]incoming: mailman.mta.exim4.LMTPoutgoing: mailman.mta.deliver.deliverlmtp_host: $MM_HOSTNAMElmtp_port: 8024smtp_host: $SMTP_HOSTsmtp_port: $SMTP_PORTconfiguration: python:mailman.config.exim4

To usePostfix, edit themain.cf configuration file, which is typicallyat/etc/postfix/main.cf on Debian-based operating systems. Addmailman-core and mailman-web tomynetworks so it will relay emails fromthe containers and add the following configuration lines:

# main.cf# Support the default VERP delimiter.recipient_delimiter = +unknown_local_recipient_reject_code = 550owner_request_special = notransport_maps =    regexp:/opt/mailman/core/var/data/postfix_lmtplocal_recipient_maps =    regexp:/opt/mailman/core/var/data/postfix_lmtprelay_domains =    regexp:/opt/mailman/core/var/data/postfix_domains

To configure Mailman to use Postfix, addMTA=postfix under mailman-core'senvironment section in thedocker-compose.yaml:

  mailman-core:    <snip>    environment:    - MTA=postfix

This will auto-generate the configuration to talk to Postfix assuming thatPostfix is available at the gateway address for the container's bridge networkat port 25. The final configuration can be found by executing:

$ docker exec mailman-core cat /etc/mailman.cfg

The postfix configuration that is generated looks like this:

[mta]incoming: mailman.mta.postfix.LMTPoutgoing: mailman.mta.deliver.deliverlmtp_host: $MM_HOSTNAMElmtp_port: 8024smtp_host: $SMTP_HOSTsmtp_port: $SMTP_PORTconfiguration: /etc/postfix-mailman.cfg

So, if you need to update the values, you can setSMTP_HOST,SMTP_PORT,MM_HOSTNAME environment variables inmailman-core container.

Please verify the output for[mta] section to ensure that it points tothe rightsmtp_host (address to reach postfix from mailman-core container)andlmtp_host (address to reach mailman-core container from postfix).

The configuration file/etc/postfix-mailman.cfg is also generated automaticallyinside themailman-core container and contains the configuration specificfor Postfix.

Setup site owner address. By default, mailman is setup with the site_owner set to 'changeme@example.com'. This should be pointing to a valid mailbox. Add the following to the '/opt/mailman/core/mailman-extra.cfg'.

[mailman]# This address is the "site owner" address.  Certain messages which must be# delivered to a human, but which can't be delivered to a list owner (e.g. a# bounce from a list owner), will be sent to this address.  It should point to# a human.site_owner: changeme@example.com

Hyperkitty in mailman-web image support full-text indexing. The current defaultindexing engine isWhooshfor historical reasons. It is highly recommended that you instead use Xapian forproduction use cases. The default will change when the next major version bumphappens.

To configure your Mailman-web container to use Xapian, add the following to yoursettings_local.py:

HAYSTACK_CONNECTIONS= {'default': {'ENGINE':'xapian_backend.XapianEngine','PATH':"/opt/mailman-web-data/fulltext_index",    },}

If you have been using the default search indexing engine, you might have tore-index emails using the following command:

$ docker composeexec mailman-web ./manage.py rebuild_index

This command can take some time if you a lot of emails, so please be patient!

It is advisable to run your Django (interfaced through WSGI server) through anactual webserver in production for better performance.

If you are using v0.1.0, the uwsgi server is configured to listen to requests atport8000 using theHTTP protocol. Make sure that you preserve theHOSTheader when you proxy the requests from your Web Server. In Nginx you can dothat by adding the following to your configuration:

    # Nginx configuration.    location /static {        alias /opt/mailman/web/static;        autoindex off;    }    location / {  proxy_pass http://127.0.0.1:8000;  include uwsgi_params;  uwsgi_read_timeout 300;  proxy_set_header Host $host;  proxy_set_header X-Forwarded-For $remote_addr;    }

Make sure you are usingproxy_pass for theHTTP protocol.

Starting from v0.1.1, the uwsgi server is configured to listen to requests atport8000 with the http protocol and port8080 for the uwsgiprotocol.

Please make sure that you are using port 8080 for uwsgi protocol.

It is advised to use the uwsgi protocol as it has better performance. BothApache and Nginx have native support for the uwsgi protocol through plugins whichare generally included in the distro packages.

To move to uwsgi protocol in the above nginx configuration use this

    # Nginx configuration.    location /static {        alias /opt/mailman/web/static;        autoindex off;    }    location / {  uwsgi_pass localhost:8080;  include uwsgi_params;  uwsgi_read_timeout 300;    }

Please make sure that you are using v0.1.1 or greater if you use this configuration.

UWSGI by default doesn't serve static files so, when runningmailman-web using the provideddocker-compose.yaml file, you won't see anyCSS or JS files being served.

To enable serving of static files using UWSGI, add the following environmentvariable to yourdocker-compose.yaml file undermailman-web:

UWSGI_STATIC_MAP=/static=/opt/mailman-web-data/static

It is recommended to use web-server to serve static files instead of UWSGI forbetter performance. You will have to add an alias rule in your web server toserve the static files. Seehere for instructions on how to configure yourweb server. The STATIC_ROOT for you would be/opt/mailman/web/static.

SSL Certificates from Lets Encrypt need to be renewed every 90 days. You cansetup a cron job to do the job. I have this small shell script (certbot-renew.sh)that you can put up in/etc/cron.monthly to get the job done.

#! /bin/bashcd /opt/letsencrypt/./certbot-auto --config /etc/letsencrypt/renewal/MY_DOMAIN_NAME.conf certonlyif [ $? -ne 0 ] then        ERRORLOG=`tail /var/log/letsencrypt/letsencrypt.log`        echo -e "The Let's Encrypt cert has not been renewed! \n \n" \                 $ERRORLOG else        nginx -s reloadfiexit 0

Please do not forget to make the script executable (chmod +x certbot-renew.sh).

This repository is licensed under the MIT License. Please see the LICENSE file formore details.