stream-processing

Star

Here are 1,092 public repositories matching this topic...

Language:All

Filter by language

All1,092 Java266 Python213 Go103 Scala60 C++58 JavaScript56 Jupyter Notebook43 Rust43 TypeScript28 C#22

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

pathwaycom /pathway

Star50.6k

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

python rust streaming real-time kafka etl machine-learning-algorithms stream-processing data-analytics dataflow data-processing data-pipelines batch-processing pathway iot-analytics etl-framework time-series-analysis

UpdatedDec 17, 2025
Python

google-ai-edge /mediapipe

Star32.5k

Cross-platform, customizable ML solutions for live and streaming media.

android c-plus-plus calculator machine-learning framework computer-vision deep-learning inference pipeline-framework stream-processing video-processing perception mobile-development audio-processing graph-framework graph-based mediapipe

UpdatedDec 16, 2025
C++

vectordotdev /vector

Star20.9k

A high-performance observability data pipeline.

rust parser events router pipeline metrics vector logs stream-processing forwarder hacktoberfest observability

UpdatedDec 17, 2025
Rust

flink learning blog.http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

mysql redis elasticsearch streaming kafka spark influxdb rabbitmq clickhouse hbase stream-processing opentsdb loki flink rocketmq

UpdatedMar 12, 2025
Java

oxnr /awesome-bigdata

Star14.1k

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

UpdatedNov 27, 2025

redpanda-data /redpanda

Star11.5k

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

go kubernetes streaming microservices kafka cpp seastar storage-engine containers realtime stream-processing event-driven redpanda

UpdatedDec 17, 2025
C++

madd86 /awesome-system-design

Star11.3k

A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

distributed-systems microservices nosql interview stream-processing microservices-architecture relational-database message-broker hadoop-ecosystem

UpdatedJun 27, 2024

ThreeDotsLabs /watermill

Star9.3k

Building event-driven applications the easy way in Go.

go golang events reactive kafka cqrs sagas rabbitmq event-sourcing nats stream-processing event-driven watermill

UpdatedNov 28, 2025
Go

risingwavelabs /risingwave

Star8.6k

Streaming data platform. Real-time stream processing, low-latency serving, and Iceberg table management.

rust database kafka postgresql stream-processing data-engineering materialized-view etl-pipeline apache-iceberg

UpdatedDec 17, 2025
Rust

redpanda-data /connect

Star8.5k

Fancy stream processing made operationally mundane

go golang kafka cqrs etl rabbitmq amqp logs message-bus event-sourcing nats stream-processing message-queue data-engineering streaming-data stream-processor data-ops

UpdatedDec 17, 2025
Go

fluent /fluent-bit

Star7.5k

Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows

c metrics logging logs prometheus data-collector stream-processing fluentd fluent-bit cloudnative sql-queries forwarder traces opentelemetry

UpdatedDec 17, 2025
C

robinhood /faust

Star6.8k

Python Stream Processing

python distributed-systems kafka stream-processing asyncio kafka-streams

UpdatedJul 27, 2024
Python

hazelcast /hazelcast

Star6.5k

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

java caching distributed-systems real-time big-data hazelcast scalability distributed-computing distributed stream-processing in-memory low-latency hacktoberfest data-insights data-in-motion

UpdatedDec 17, 2025
Java

MaterializeInc /materialize

Star6.2k

The live data layer for apps and AI agents Create up-to-the-second views into your business, just using SQL

mysql rust distributed-systems streaming sql sql-server database kafka postgresql data-store stream-processing cdc streaming-data materialized-view data-mesh postgresql-dialect operational-data-store

UpdatedDec 17, 2025
Rust

apache /hudi

Star6k

Upserts, Deletes And Incremental Processing on Big Data.

bigdata stream-processing data-integration datalake apachespark hudi apachehudi incremental-processing apacheflink

UpdatedDec 17, 2025
Java

online-ml /river

Star5.6k

🌊 Online machine learning in Python

python data-science machine-learning streaming stream-processing online-learning streaming-data concept-drift real-time-processing incremental-learning online-machine-learning online-statistics

UpdatedDec 16, 2025
Python

fluvio-community /fluvio

Star5.1k

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

rust distributed-systems streaming real-time serverless webassembly data-flow stream-processing data-analytics data-integration cloud-native data-pipelines stateful streaming-data stream-processing-engine event-driven-architecture streaming-analytics streaming-data-processing streaming-data-pipelines

UpdatedDec 16, 2025
Rust

javascriptdata /danfojs

Star5k

Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.

javascript data-science tensorflow table pandas stream-processing data-analytics data-analysis data-manipulation tensors dataframe stream-data plotting-charts danfojs

UpdatedAug 6, 2025
TypeScript

ag2ai /faststream

Star4.8k

FastStream is a powerful and easy-to-use asynchronous Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.

python redis distributed-systems kafka rabbitmq nats stream-processing asyncio asyncapi faststream

UpdatedDec 15, 2025
Python

ArroyoSystems /arroyo

Star4.7k

Distributed stream processing engine in Rust

infrastructure rust data sql kafka stream-processing stream-processing-engine dev-tools data-stream-processing

UpdatedDec 13, 2025
Rust

Improve this page

Add a description, image, and links to thestream-processing topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thestream-processing topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly