Repository files navigation

This will build a container for backing up multiple types of DB Servers

Backs up CouchDB, InfluxDB, MySQL/MariaDB, Microsoft SQL, MongoDB, Postgres, Redis servers.

• dump to local filesystem or backup to S3 Compatible services, and Azure.
• multiple backup job support
  • selectable when to start the first dump, whether time of day or relative to container start time
  • selectable interval
  • selectable omit scheduling during periods of time
  • selectable database user and password
  • selectable cleanup and archive capabilities
  • selectable database name support - all databases, single, or multiple databases
  • backup all to separate files or one singular file
• checksum support choose to have an MD5 or SHA1 hash generated after backup for verification
• compression support (none, gz, bz, xz, zstd)
• encryption support (passphrase and public key)
• notify upon job failure to email, matrix, mattermost, rocketchat, custom script
• zabbix metrics support
• hooks to execute pre and post backup job for customization purposes
• companion script to aid in restores

• Dave Conroy

• About
• Maintainer
• Table of Contents
• Prerequisites and Assumptions
• Installation
  • Build from Source
  • Prebuilt Images
    • Multi Architecture
• Configuration
  • Quick Start
  • Persistent Storage
  • Environment Variables
    • Base Images used
    • Container Options
    • Job Defaults
      • Compression Options
      • Encryption Options
      • Scheduling Options
      • Default Database Options
        • CouchDB
        • InfluxDB
        • MariaDB/MySQL
        • Microsoft SQL
        • MongoDB
        • Postgresql
        • Redis
      • Default Storage Options
        • Filesystem
        • S3
        • Azure
      • Hooks
        • Path Options
        • Pre Backup
        • Post backup
    • Job Backup Options
      • Compression Options
      • Encryption Options
      • Scheduling Options
      • Specific Database Options
        • CouchDB
        • InfluxDB
        • MariaDB/MySQL
        • Microsoft SQL
        • MongoDB
        • Postgresql
        • Redis
        • SQLite
      • Specific Storage Options
        • Filesystem
        • S3
        • Azure
      • Hooks
        • Path Options
        • Pre Backup
        • Post backup
    • Notifications
      • Custom Notifications
      • Email Notifications
      • Matrix Notifications
      • Mattermost Notifications
      • Rocketchat Notifications
• Maintenance
  • Shell Access
  • Manual Backups
  • Restoring Databases
• Support
  • Usage
  • Bugfixes
  • Feature Requests
  • Updates
• License

• You must have a working connection to one of the supported DB Servers and appropriate credentials

Clone this repository and build the image withdocker build <arguments> (imagename) .

Builds of the image are available onDocker Hub

Builds of the image are also available on theGithub Container Registry

docker pull ghcr.io/tiredofit/docker-db-backup:(imagetag)

The following image tags are available along with their tagged release based on what's written in theChangelog:

docker pull docker.io/tiredofit/db-backup:(imagetag)

Images are built primarily foramd64 architecture, and may also include builds forarm/v7,arm64 and others. These variants are all unsupported. Considersponsoring my work so that I can work with various hardware. To see if this image supports multiple architectures, typedocker manifest (image):(tag)

• The quickest way to get started is usingdocker-compose. See the examples folder for a series of example compose.yml that can be modified for development or production use.

• Set variousenvironment variables to understand the capabilities of this image.

• Mappersistent storage for access to configuration and data files for backup.

The following directories are used for configuration and can be mapped for persistent storage.

This image relies on anAlpine Linux base image that relies on aninit system for added capabilities. Outgoing SMTP capabilities are handled viamsmtp. Individual container performance monitoring is performed byzabbix-agent. Additional tools include:bash,curl,less,logrotate,nano.

Be sure to view the following repositories to understand all the customizable options:

If these are set and no other defaults or variables are set explicitly, they will be added to any of the backup jobs.

Encryption occurs after compression and the encrypted filename will have a.gpg suffix

You may need to wrap yourDEFAULT_BACKUP_BEGIN value in quotes for it to properly parse. There have been reports of backups that start with a0 get converted into a different format which will not allow the timer to start at the correct time.

Options that are related to the value ofDEFAULT_BACKUP_LOCATION

IfDEFAULT_BACKUP_LOCTION =FILESYSTEM then the following options are used:

IfDEFAULT_BACKUP_LOCATION =S3 then the following options are used:

• WhenDEFAULT_S3_KEY_ID and/orDEFAULT_S3_KEY_SECRET is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.

IfDEFAULT_BACKUP_LOCATION =blobxfer then the following options are used:.

• WhenDEFAULT_BLOBXFER_MODE is set to auto it will use blob containers by default. If theDEFAULT_BLOBXFER_REMOTE_PATH path does not exist a blob container with that name will be created.

This service uploads files from backup targed directoryDEFAULT_FILESYSTEM_PATH.If the a cleanup configuration inDEFAULT_CLEANUP_TIME is defined, the remote directory on Azure storage will also be cleaned automatically.

If you want to execute a custom script before a backup starts, you can drop bash scripts with the extension of.sh in the location defined inDB01_SCRIPT_LOCATION_PRE. See the following example to utilize:

$ cat pre-script.sh##!/bin/bash# #### Example Pre Script# #### $1=DBXX_TYPE (Type of Backup)# #### $2=DBXX_HOST (Backup Host)# #### $3=DBXX_NAME (Name of Database backed up# #### $4=BACKUP START TIME (Seconds since Epoch)# #### $5=BACKUP FILENAME (Filename)echo"${1} Backup Starting on${2} for${3} at${4}. Filename:${5}"

## script DBXX_TYPE DBXX_HOST DBXX_NAME STARTEPOCH BACKUP_FILENAME${f} "${backup_job_db_type}" "${backup_job_db_host}" "${backup_job_db_name}" "${backup_routines_start_time}" "${backup_job_file}"

Outputs the following on the console:

mysql Backup Starting on example-db for example at 1647370800. Filename: mysql_example_example-db_202200315-000000.sql.bz2

If you want to execute a custom script at the end of a backup, you can drop bash scripts with the extension of.sh in the location defined inDB01_SCRIPT_LOCATION_POST. Also to support legacy users/assets/custom-scripts is also scanned and executed.See the following example to utilize:

$ cat post-script.sh##!/bin/bash# #### Example Post Script# #### $1=EXIT_CODE (After running backup routine)# #### $2=DBXX_TYPE (Type of Backup)# #### $3=DBXX_HOST (Backup Host)# #### #4=DBXX_NAME (Name of Database backed up# #### $5=BACKUP START TIME (Seconds since Epoch)# #### $6=BACKUP FINISH TIME (Seconds since Epoch)# #### $7=BACKUP TOTAL TIME (Seconds between Start and Finish)# #### $8=BACKUP FILENAME (Filename)# #### $9=BACKUP FILESIZE# #### $10=HASH (If CHECKSUM enabled)# #### $11=MOVE_EXIT_CODEecho"${1}${2} Backup Completed on${3} for${4} on${5} ending${6} for a duration of${7} seconds. Filename:${8} Size:${9} bytes MD5:${10}"

  ## script EXIT_CODE DB_TYPE DB_HOST DB_NAME STARTEPOCH FINISHEPOCH DURATIONEPOCH BACKUP_FILENAME FILESIZE CHECKSUMVALUE  ${f} "${exit_code}" "${dbtype}" "${backup_job_db_host}" "${backup_job_db_name}" "${backup_routines_start_time}" "${backup_routines_finish_time}" "${backup_routines_total_time}" "${backup_job_file}" "${filesize}" "${checksum_value}" "${move_exit_code}

Outputs the following on the console:

0 mysql Backup Completed on example-db for example on 1647370800 ending 1647370920 for a duration of 120 seconds. Filename: mysql_example_example-db_202200315-000000.sql.bz2 Size: 7795 bytes Hash: 952fbaafa30437494fdf3989a662cd40 0

If you wish to change the size value from bytes to megabytes set environment variableDB01_SIZE_VALUE=megabytes

You must make your scripts executable otherwise there is an internal check that will skip trying to run it otherwise.If for some reason your filesystem or host is not detecting it right, use the environment variableDB01_POST_SCRIPT_SKIP_X_VERIFY=TRUE to bypass.

IfDEFAULT_ variables are set and you do not wish for the settings to carry over into your jobs, you can set the appropriate environment variable with the value ofunset.Otherwise, override them per backup job. Additional backup jobs can be scheduled by usingDB02_,DB03_,DB04_ ... prefixes. SeeSpecific Database Options which may overrule this list.

Encryption will occur after compression and the resulting filename will have a.gpg suffix

Your Organization will be mapped toDB_USER and your root token will need to be mapped toDB_PASS.You may useDB_NAME=ALL to backup the entire set of databases.ForDB_HOST use syntax ofhttp(s)://db-name

Options that are related to the value ofDB01_BACKUP_LOCATION

IfDB01_BACKUP_LOCTION =FILESYSTEM then the following options are used:

IfDB01_BACKUP_LOCATION =S3 then the following options are used:

WhenDB01_S3_KEY_ID and/orDB01_S3_KEY_SECRET is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.

IfDB01_BACKUP_LOCATION =blobxfer then the following options are used:.

• WhenDEFAULT_BLOBXFER_MODE is set to auto it will use blob containers by default. If theDEFAULT_BLOBXFER_REMOTE_PATH path does not exist a blob container with that name will be created.

This service uploads files from backup directoryDB01_BACKUP_FILESYSTEM_PATH.If the a cleanup configuration inDB01_CLEANUP_TIME is defined, the remote directory on Azure storage will also be cleaned automatically.

If you want to execute a custom script before a backup starts, you can drop bash scripts with the extension of.sh in the location defined inDB01_SCRIPT_LOCATION_PRE. See the following example to utilize:

$ cat pre-script.sh##!/bin/bash# #### Example Pre Script# #### $1=DB01_TYPE (Type of Backup)# #### $2=DB01_HOST (Backup Host)# #### $3=DB01_NAME (Name of Database backed up# #### $4=BACKUP START TIME (Seconds since Epoch)# #### $5=BACKUP FILENAME (Filename)echo"${1} Backup Starting on${2} for${3} at${4}. Filename:${5}"

## script DB01_TYPE DB01_HOST DB01_NAME STARTEPOCH BACKUP_FILENAME${f} "${backup_job_db_type}" "${backup_job_db_host}" "${backup_job_db_name}" "${backup_routines_start_time}" "${backup_job_filename}"

Outputs the following on the console:

mysql Backup Starting on example-db for example at 1647370800. Filename: mysql_example_example-db_202200315-000000.sql.bz2

If you want to execute a custom script at the end of a backup, you can drop bash scripts with the extension of.sh in the location defined inDB01_SCRIPT_LOCATION_POST. Also to support legacy users/assets/custom-scripts is also scanned and executed.See the following example to utilize:

$ cat post-script.sh##!/bin/bash# #### Example Post Script# #### $1=EXIT_CODE (After running backup routine)# #### $2=DB_TYPE (Type of Backup)# #### $3=DB_HOST (Backup Host)# #### #4=DB_NAME (Name of Database backed up# #### $5=BACKUP START TIME (Seconds since Epoch)# #### $6=BACKUP FINISH TIME (Seconds since Epoch)# #### $7=BACKUP TOTAL TIME (Seconds between Start and Finish)# #### $8=BACKUP FILENAME (Filename)# #### $9=BACKUP FILESIZE# #### $10=HASH (If CHECKSUM enabled)# #### $11=MOVE_EXIT_CODEecho"${1}${2} Backup Completed on${3} for${4} on${5} ending${6} for a duration of${7} seconds. Filename:${8} Size:${9} bytes MD5:${10}"

  ## script EXIT_CODE DB_TYPE DB_HOST DB_NAME STARTEPOCH FINISHEPOCH DURATIONEPOCH BACKUP_FILENAME FILESIZE CHECKSUMVALUE  ${f} "${exit_code}" "${dbtype}" "${dbhost}" "${dbname}" "${backup_routines_start_time}" "${backup_routines_finish_time}" "${backup_routines_total_time}" "${backup_job_filename}" "${filesize}" "${checksum_value}" "${move_exit_code}

Outputs the following on the console:

0 mysql Backup Completed on example-db for example on 1647370800 ending 1647370920 for a duration of 120 seconds. Filename: mysql_example_example-db_202200315-000000.sql.bz2 Size: 7795 bytes Hash: 952fbaafa30437494fdf3989a662cd40 0

If you wish to change the size value from bytes to megabytes set environment variableDB01_SIZE_VALUE=megabytes

You must make your scripts executable otherwise there is an internal check that will skip trying to run it otherwise.If for some reason your filesystem or host is not detecting it right, use the environment variableDB01_POST_SCRIPT_SKIP_X_VERIFY=TRUE to bypass.

This image has capabilities on sending notifications via a handful of services when a backup job fails. This is a global option that cannot be individually set per backup job.

The following is sent to the custom script. Use how you wish:

$1 unix timestamp$2 logfile$3 errorcode$4 subject$5 body/error message

See more details in the base image listed above for more mail environment variables.

Fetch aMATRIX_ACCESS_TOKEN:

curl -XPOST -d '{"type":"m.login.password", "user":"myuserid", "password":"mypass"}' "https://matrix.org/_matrix/client/r0/login"

Copy the JSON responseaccess_token that will look something like this:

{"access_token":"MDAxO...blahblah","refresh_token":"MDAxO...blahblah","home_server":"matrix.org","user_id":"@myuserid:matrix.org"}

For debugging and maintenance purposes you may want access the containers shell.

bash docker exec -it (whatever your container name is) bash

Manual Backups can be performed by entering the container and typingbackup-now. This will execute all the backup tasks that are scheduled by means of theBACKUPXX_ variables. Alternatively if you wanted to execute a job on its own you could simply typebackup01-now (or whatever your number would be). There is no concurrency, and jobs will be executed sequentially.

• Recently there was a request to have the container work with Kubernetes cron scheduling. This can theoretically be accomplished by setting the containerMODE=MANUAL and then settingMANUAL_RUN_FOREVER=FALSE - You would also want to disable a few features from the upstream base images specificallyCONTAINER_ENABLE_SCHEDULING andCONTAINER_ENABLE_MONITORING. This should allow the container to start, execute a backup by executing and then exit cleanly. An alternative way to running the script is to execute/etc/services.available/10-db-backup/run.

Entering in the container and executingrestore will execute a menu based script to restore your backups - MariaDB, Postgres, and Mongo supported.

You will be presented with a series of menus allowing you to choose:

• What file to restore
• What type of DB Backup
• What Host to restore to
• What Database Name to restore to
• What Database User to use
• What Database Password to use
• What Database Port to use

The image will try to do auto detection based on the filename for the type, hostname, and database name.The image will also allow you to use environment variables or Docker secrets used to backup the images

The script can also be executed skipping the interactive mode by using the following syntax/

`restore <filename> <db_type> <db_hostname> <db_name> <db_user> <db_pass> <db_port>`

If you only enter some of the arguments you will be prompted to fill them in.

These images were built to serve a specific need in a production environment and gradually have had more functionality added based on requests from the community.

• TheDiscussions board is a great place for working with the community on tips and tricks of using this image.
• Sponsor me for personalized support

• Please, submit aBug Report if something isn't working as expected. I'll do my best to issue a fix in short order.

• Feel free to submit a feature request, however there is no guarantee that it will be added, or at what timeline.
• Sponsor me regarding development of features.

• Best effort to track upstream changes, More priority if I am actively using the image in a production environment.
• Sponsor me for up to date releases.

MIT. SeeLICENSE for more details.