persistent-postgresql-streaming:Memory-constant streaming of Persistent entities from PostgreSQL
This library provides a set of APIs for performing queries on Persistententities in constant memory, streaming the results usingconduit.
The library relies on PostgreSQL-specific features to avoid loading all theresults of a query into memory at once. This allows for accessing tables ofmillions of entities from Haskell without a memory blow-up.
[Skip to Readme]
Downloads
- persistent-postgresql-streaming-0.1.0.0.tar.gz [browse] (Cabal source package)
- Package description (as included in the package)
Maintainer's Corner
For package maintainers and hackage trustees
Candidates
| Versions[RSS] | 0.1.0.0 |
|---|---|
| Change log | CHANGELOG.md |
| Dependencies | base (>=4.7 && <5),conduit (>=1.3 && <1.4),monad-logger (>=0.3.28.4 && <0.4),mtl (>=2 && <2.3),persistent (>=2.13.2 && <2.14),persistent-postgresql (>=2.13.2 && <2.14),postgresql-simple (>=0.6.1 && <0.7),resourcet (>=0.3 && <2),text (>=1 && <1.3),transformers (>=0.5 && <0.6) [details] |
| License | BSD-3-Clause |
| Copyright | (c) 2021 Supercede Ltd. |
| Author | Isaac van Bakel |
| Maintainer | support@supercede.com |
| Category | Conduit,Database |
| Home page | https://github.com/SupercedeTech/persistent-postgresql-streaming#readme |
| Source repo | head: git clonehttps://github.com/SupercedeTech/persistent-postgresql-streaming |
| Uploaded | byisaac_supercede at2021-10-20T15:41:26Z |
| Distributions | |
| Reverse Dependencies | 1 direct, 0 indirect [details] |
| Downloads | 327 total (5 in the last 30 days) |
| Rating | (no votes yet)[estimated byBayesian average] |
| Your Rating |
|
| Status | Docs available[build log] Last success reported on 2021-10-20[all 1 reports] |
Readme for persistent-postgresql-streaming-0.1.0.0
[back to package description]persistent-postgresql-streaming
This library allows for memory-constant streaming ofpersistent entities from PostgreSQL databases.
This code makes use of the PostgreSQL-onlycursors, which allow for batched access to the result set of a query at speeds comparable to loading all the results into memory at once.
See themain project README for more.
Streaming Persistent queries
The main function of this library isselectStream, which can be used in place ofselectSource.selectSource runs in aConduitT - consuming the conduit will pull results from the database in constant memory.
FAQ
Why isselectStream so slow?
Have you configured PostgreSQL correctly? See the section in theREADME about it.