Repository files navigation

Copy data from any source to any destination without any code

ingestr is a command-line app that allows you to ingest data from any source into any destination using simple command-line flags, no code necessary.

• ✨ copy data from your database into any destination
• ➕ incremental loading:append,merge ordelete+insert
• 🐍 single-command installation

ingestr takes away the complexity of managing any backend or writing any code for ingesting data, simply run the command and watch the data land on its destination.

We recommend usinguv to runingestr.

pip install uvuvx ingestr

Alternatively, if you'd like to install it globally:

uv pip install --system ingestr

While installation with vanillapip is possible, it's an order of magnitude slower.

ingestr ingest \    --source-uri'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \    --source-table'public.some_data' \    --dest-uri'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \    --dest-table'ingestr.some_data'

That's it.

This command:

• gets the tablepublic.some_data from the Postgres instance.
• uploads this data to your BigQuery warehouse under the schemaingestr and tablesome_data.

You can see the full documentationhere.

Join our Slack communityhere.

Pull requests are welcome. However, please open an issue first to discuss what you would like to change. We maybe able to offer you help and feedback regarding any changes you would like to make.

Feel free to create an issue if you'd like to see support for another source or destination.

This project would not have been possible without the amazing work done by theSQLAlchemy anddlt teams. We relied on their work to connect to various sources and destinations, and builtingestr as a simple, opinionated wrapper around their work.