pipeline

Star

Here are 5,880 public repositories matching this topic...

Language:All

Filter by language

All5,880 Python1,567 Jupyter Notebook723 Shell333 Go286 JavaScript284 TypeScript274 Nextflow239 Java236 HTML154 R148

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

jina-ai /serve

Star21.7k

☁️ Build multimodal AI applications with cloud-native stack

docker kubernetes machine-learning framework deep-learning microservice pipeline cncf grpc prometheus orchestration cloud-native jaeger multimodal mlops fastapi opentelemetry neural-search generative-ai llmops

UpdatedMar 24, 2025
Python

argoproj /argo-cd

Star20.1k

Declarative Continuous Deployment for Kubernetes

docker kubernetes devops pipeline continuous-delivery cd helm continuous-deployment ci-cd argo cicd hacktoberfest jsonnet gitops kustomize argo-cd

UpdatedJul 18, 2025
Go

vectordotdev /vector

Star20k

A high-performance observability data pipeline.

rust parser events router pipeline metrics vector logs stream-processing forwarder observability

UpdatedJul 18, 2025
Rust

kestra-io /kestra

Star19.9k

⚡ Workflow Automation Platform. Orchestrate & Schedule code in any language, run anywhere, 600+ plugins. Alternative to Airflow, n8n, Rundeck, VMware vRA, Zapier ...

java workflow devops automation pipeline orchestration infrastructure-as-code high-availability low-code pipeline-as-code lowcode data-orchestration

UpdatedJul 18, 2025
Java

PrefectHQ /prefect

Star19.8k

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

python infrastructure workflow data-science data automation pipeline workflow-engine orchestration data-engineering observability prefect data-ops ml-ops

UpdatedJul 18, 2025
Python

airbytehq /airbyte

Star18.9k

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

mysql python java bigquery data pipeline etl postgresql s3 snowflake self-hosted data-engineering data-analysis mssql data-integration data-collection redshift elt data-pipeline change-data-capture

UpdatedJul 18, 2025
Python

Avaiga /taipy

Star18.2k

Turns Data and AI algorithms into production-ready web applications in no time.

python workflow automation pipeline job-scheduler pipelines data-visualization orchestration datascience data-engineering developer-tools data-integration scenario hacktoberfest mlops data-ops scenario-analysis hacktoberfest2023 taipy-core taipy-gui

UpdatedJun 29, 2025
Python

marimo-team /marimo

Sponsor

Star14.5k

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. All in a modern, AI-native editor.

python data-science machine-learning reactive sql pipeline web-app data-visualization artificial-intelligence developer-tools dataflow notebooks dag

UpdatedJul 18, 2025
Python

great-expectations /great_expectations

Star10.6k

Always know what to expect from your data.

data-science pipeline exploratory-data-analysis eda data-engineering data-quality data-profiling datacleaner exploratory-analysis cleandata dataquality datacleaning mlops pipeline-tests pipeline-testing dataunittest data-unit-tests exploratorydataanalysis pipeline-debt data-profilers

UpdatedJul 18, 2025
Python

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

python machine-learning pipeline hacktoberfest machine-learning-engineering mlops kedro experiment-tracking