Repository files navigation

• help build development and testing environments involving full-size database,
• provide temporary full-size database clones for SQL query optimization (see also:Joe bot, which works on top of Database Lab),
• help verify database migrations (DB schema changes) and massive data operations.

As an example, the cloning of 10 TiB PostgreSQL database takes less than 2 seconds.

In rapidly developing businesses, excellent production health requires powerful non-production environments.With Database Lab, provisioning of multi-terabyte database clones doesn't imply muchwaiting time or extra budgets spent anymore. Therefore, Database Lab gives necessary power to developers, DBAs,and QA engineers, boosting development and testing processes.

The project is in its early stage. However, it is already being extensively usedin some teams in their daily work. Since production is not involved, it isquite easy to try and start using it.

Please support the project giving a GitLab star (it's onthe main page,at the upper right corner):

To discuss Database Lab, try a demo, or ask any questions,join our community Slack.

• Perform SQL optimization in a convenient way (seepostgres-ai/joe)
• Check database schema changes (database migrations) on full-sized database clones using Database Lab in CI (seepostgres-ai/ci-example)

TL;DR: you need:

• any machine with a separate disk that is big enough to store a single copy of your database,
• Linux with Docker,
• ZFS to enable thin cloning (the default option; also, LVM is supported as an alternative),
• initial copy of your Postgres database.

Details:

• for each Database Lab instance, a separate machine (either physical or virtual) is needed,
• both on-premise and cloud setups are possible,
• for each Postgres cluster (a single Postgres server with databases), a separate Database Lab instance is required,
• the machine needs to have a separate disk partition with size enough to store the target Postgres directory (PGDATA),
• any modern Linux is supported,
• Docker needs to be installed on the machine,
• currently, you need to take care yourself of the initial copying of the database to this disk ("thick cloning" stage),use pg_basebackup, restoration from an archive (such as WAL-G, Barman, pgBackRest or any), or dump/restore (the only waysupported for RDS, until AWS guys decide to allow replication connections),
• upon request, Database Lab will do "thin cloning" of PGDATA, providing fully independent writablePostgres clones to users;
• currently, two technologies are supported for thin cloning:
  • ZFS:ZFS on Linux needs to be installed on the machine,
  • LVM: withLVM, all thin clones will be based on the latest version of the database (with ZFS, you can have multiple snapshots prepared and choose which one to use when requesting a new thin clone).
• it is easy to extend and add, Ceph - please contact us if you need it;
• comments, bug reports, and contributions are highly welcome.

Seedetailed tutorialin our documentation.

For stable Docker images seepostgresai/dblab-server repository on DockerHub.

Install Database Lab client CLI on a Linux architecture (e.g., Ubuntu):

curl https://gitlab.com/postgres-ai/database-lab/-/raw/master/scripts/cli_install.sh| bash

Also, binaries available for download:Alpine,other.

See the full client CLI referencehere.

Once you have Database Lab server(s) intalled, and client CLI installed on your machine,initialize CLI and start communicating with the Database Lab server(s).

dblab init \  --environment-id=tutorial \  --url=http://$IP_OR_HOSTNAME:2345 \  --token=secret_token

Access your Database Lab instance and check its status.

dblab instance status

When your Database Lab instance is up and running you can use it to create thinclones, work with them, delete the existing clones, and see the list ofexisting clones. To create a thin clone, you need to execute adblab clone createand fill all the required options, as illustrated below:

dblab clone create \  --username dblab_user_1 \  --password secret_password \  --id my_first_clone

After a few seconds, if everything is configured correctly, you will seethat the clone is ready to be used:

{"id":"botcmi54uvgmo17htcl0","snapshot": {"id":"dblab_pool@initdb",...    },"status": {"code":"OK","message":"Clone is ready to accept Postgres connections."    },"db": {"connStr":"host=localhost port=6000 user=dblab_user_1","host":"localhost","port":"6000","username":"dblab_user_1","password":""    },...}

Now you can work with this clone using any PostgreSQL client, for examplepsql:

PGPASSWORD=secret_password \  psql"host=${IP_OR_HOSTNAME} port=6000 user=dblab_user_1 dbname=test" \  -c'\l+'

• Server
• CLI client
• Custom Postgres images

• API reference
• CLI reference

• Check ourQ&A
• or join our community (links below)

• Community Slack (English)
• Telegram (Russian)

See ourGitLab Container Registry for develop Docker images.

1. Installgolangci-lint:https://github.com/golangci/golangci-lint#install