This release expands the closed intervals that were released for SQL sources in v0.12.5 to all the other platforms we support. In addition, it includes an improved Google Analytics source that can load past intervals as well.

What's Changed

• Made the access_token optional by@sanjushahgupta in#82
• Patch/closed intervals by@karakanb in#83

Full Changelog:v0.12.5...v0.12.6

Contributors

This release introduces an important change on how intervals worked in ingestr.

Important - Closed Intervals

This release introduces a change in how--interval-start and--interval-end works in ingestr. Previously, these flags were treated as[interval-start, interval-end), meaning that the values withinterval-start would be included, the ones withinterval-end would be excluded, which means the ranges would not be overlapping.

In more tangible terms, the previous version would create a query like this to get the data based on anupdated_at field:

SELECT*FROM mytableWHERE updated_at>= :interval_startAND updated_at< :interval_end

In the new version, this is how the query looks like:

SELECT*FROM mytableWHERE updated_at>= :interval_startAND updated_at<= :interval_end

This means certain bits of data will be fetched unnecessarily, but ingestr will deduplicate the data anyway. This is especially helpful for date-based incremental keys where there's no high-precision timestamp available.

Full Changelog:v0.12.4...v0.12.5

What's Changed

• New source:Google Analytics by@sanjushahgupta in#70
• New source:GitHub by@sanjushahgupta in#61
• Added support for a new--columns flag that can be used to pass hints for the destination table structure.
• Fixed a bug with incremental load handling for date columns.

Full Changelog:v0.12.3...v0.12.4

Contributors

This release introduces a new Kafka source to ingestr!

Using ingestr, you can ingest data from any Kafka topic into your DWH, seedocs here.

This release addsGorgias as a new source to ingestr!

The current release supports 4 resources at the moment:

• customers
• tickets
• ticket messages
• satisfaction surveys

Adding new resources would be pretty easy, feel free to contribute either directly to the code or creating an issue with your needs.

This release adds support for Shopify as a new source! 🚀

Givethe docs a look, anda share your feedback.

What's Changed

• updated snowflake account identifier usage by@trakmaker in#17
• add shopify as a source by@karakanb in#20

New Contributors

• @trakmaker made their first contribution in#17
• @karakanb made their first contribution in#20

Full Changelog:v0.5.1...v0.6.0

Contributors

This release adds support for SAP Hana as a source. You can use the Hana databases as a source using the following URI structure:

hana://username:password@host:port/dbname

Enjoy!