Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

• UpdatedMar 17, 2025
• Python

Workflow Engine for Kubernetes

• UpdatedMar 13, 2025
• Go

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

• UpdatedMar 14, 2025
• Java

PipelineAI

• UpdatedApr 17, 2024
• Jsonnet

Build data pipelines, the easy way 🛠️

• UpdatedJun 6, 2023
• TypeScript

Docker Apache Airflow

• UpdatedMar 1, 2023
• Shell

Curated list of resources about Apache Airflow

• UpdatedAug 20, 2024
• Shell

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

• UpdatedMar 13, 2025
• Java

Elyra extends JupyterLab with an AI centric approach.

• UpdatedMar 16, 2025
• Python

A series of DAGs/Workflows to help maintain the operation of Airflow

• UpdatedJun 18, 2024
• Python

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

• UpdatedAug 26, 2022
• Python

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

• UpdatedMar 9, 2020
• Python

More than 2000+ Data engineer interview questions.

• UpdatedJan 26, 2025

Dynamically generate Apache Airflow DAGs from YAML configuration files

• UpdatedMar 10, 2025
• Python

Example end to end data engineering project.

• UpdatedDec 8, 2022
• Python

A Data Engineering & Machine Learning Knowledge Hub

• UpdatedFeb 2, 2024
• Python

Personal Data Engineering Projects

• UpdatedFeb 8, 2023
• Jupyter Notebook

🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

• UpdatedApr 3, 2024
• Python

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

• UpdatedMar 13, 2025
• Python

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

• UpdatedJun 8, 2024
• Go