SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

• UpdatedMar 24, 2025
• Java

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

• UpdatedMar 24, 2025
• Python

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

• UpdatedMar 25, 2025
• Java

Concurrent and multi-stage data ingestion and data processing with Elixir

• UpdatedMar 18, 2025
• Elixir

Pravega - Streaming as a new software defined storage primitive

• UpdatedMar 2, 2025
• Java

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

• UpdatedMar 24, 2025
• Go

Copy to/from Parquet in S3 or Azure Blob Storage from within PostgreSQL

• UpdatedMar 24, 2025
• Rust

Orbital automates integration between data sources (APIs, Databases, Queues and Functions). BFF's, API Composition and ETL pipelines that adapt as your specs change.

• UpdatedMar 21, 2025
• TypeScript

Use SQL to build ELT pipelines on a data lakehouse.

• UpdatedMay 25, 2022
• JavaScript

A Python library that enables ML teams to share, load, and transform data in a collaborative, flexible, and efficient way 🌰

• UpdatedMar 22, 2025
• Python

The Data Engineering Book - หนังสือวิศวกรรมข้อมูล ของคนไทย เพื่อคนไทย

• UpdatedOct 28, 2023
• JavaScript

Apache Paimon Rust The rust implementation of Apache Paimon.

• UpdatedOct 1, 2024
• Rust

Apache Spark examples exclusively in Java

• UpdatedApr 21, 2023
• Java

• UpdatedFeb 11, 2020
• Java

Squirrel dataset hub

• UpdatedSep 7, 2023
• Python

Sample code for the AWS Big Data Blog Post Building a scalable streaming data processor with Amazon Kinesis Data Streams on AWS Fargate

• UpdatedMar 25, 2025
• Python

Enables custom tracing of Java applications in Dynatrace

• UpdatedSep 3, 2024
• Java

OpenKit Java Reference Implementation

• UpdatedAug 2, 2024
• Java

Download and warehouse historical trading data

• UpdatedMar 17, 2023
• Elixir

The Data Integration Library project provides a library of generic components based on a multi-stage architecture for data ingress and egress.

• UpdatedMar 3, 2025
• Java