stream-processing

Star

Here are 1,146 public repositories matching this topic...

Language:All

Filter by language

All1,146 Java272 Python231 Go105 Scala61 C++60 JavaScript56 Rust48 Jupyter Notebook43 TypeScript33 C#22

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

pathwaycom /pathway

Star59.7k

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

python rust streaming real-time kafka etl machine-learning-algorithms stream-processing data-analytics dataflow data-processing data-pipelines batch-processing pathway iot-analytics etl-framework time-series-analysis

UpdatedFeb 20, 2026
Python

google-ai-edge /mediapipe

Star33.8k

Cross-platform, customizable ML solutions for live and streaming media.

android c-plus-plus calculator machine-learning framework computer-vision deep-learning inference pipeline-framework stream-processing video-processing perception mobile-development audio-processing graph-framework graph-based mediapipe

UpdatedFeb 4, 2026
C++

vectordotdev /vector

Star21.3k

A high-performance observability data pipeline.

rust parser events router pipeline metrics vector logs stream-processing forwarder hacktoberfest observability

UpdatedFeb 20, 2026
Rust

flink learning blog.http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

mysql redis elasticsearch streaming kafka spark influxdb rabbitmq clickhouse hbase stream-processing opentsdb loki flink rocketmq

UpdatedMar 12, 2025
Java

oxnr /awesome-bigdata

Star14.2k

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

UpdatedFeb 5, 2026

redpanda-data /redpanda

Star11.8k

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

go kubernetes streaming microservices kafka cpp seastar storage-engine containers realtime stream-processing event-driven redpanda

UpdatedFeb 21, 2026
C++

madd86 /awesome-system-design

Star11.7k

A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

distributed-systems microservices nosql interview stream-processing microservices-architecture relational-database message-broker hadoop-ecosystem

UpdatedJun 27, 2024

ThreeDotsLabs /watermill

Star9.5k

Building event-driven applications the easy way in Go.

go golang events reactive kafka cqrs sagas rabbitmq event-sourcing nats stream-processing event-driven watermill

UpdatedJan 5, 2026
Go

risingwavelabs /risingwave

Star8.8k

Event streaming platform for agents, apps, and analytics. Continuously ingest, transform, and serve event data in real time, at scale.

rust database kafka postgresql stream-processing data-engineering materialized-view etl-pipeline apache-iceberg

UpdatedFeb 20, 2026
Rust

redpanda-data /connect

Star8.6k

Fancy stream processing made operationally mundane

go golang kafka cqrs etl rabbitmq amqp logs message-bus event-sourcing nats stream-processing message-queue data-engineering streaming-data stream-processor data-ops

UpdatedFeb 20, 2026
Go

fluent /fluent-bit

Star7.7k

Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows

c metrics logging logs prometheus data-collector stream-processing fluentd fluent-bit cloudnative sql-queries forwarder traces opentelemetry

UpdatedFeb 20, 2026
C

robinhood /faust

Star6.8k

Python Stream Processing

python distributed-systems kafka stream-processing asyncio kafka-streams

UpdatedJul 27, 2024
Python

hazelcast /hazelcast

Star6.6k

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

java caching distributed-systems real-time big-data hazelcast scalability distributed-computing distributed stream-processing in-memory low-latency hacktoberfest data-insights data-in-motion

UpdatedFeb 20, 2026
Java

MaterializeInc /materialize

Star6.2k

The live data layer for apps and AI agents Create up-to-the-second views into your business, just using SQL

mysql rust distributed-systems streaming sql sql-server database kafka postgresql data-store stream-processing cdc streaming-data materialized-view data-mesh postgresql-dialect operational-data-store

UpdatedFeb 21, 2026
Rust

apache /hudi

Star6.1k

Upserts, Deletes And Incremental Processing on Big Data.

bigdata stream-processing data-integration datalake apachespark hudi apachehudi incremental-processing apacheflink

UpdatedFeb 20, 2026
Java

online-ml /river

Star5.7k

🌊 Online machine learning in Python

python data-science machine-learning streaming stream-processing online-learning streaming-data concept-drift real-time-processing incremental-learning online-machine-learning online-statistics

UpdatedFeb 9, 2026
Python

fluvio-community /fluvio

Star5.2k

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

rust distributed-systems streaming real-time serverless webassembly data-flow stream-processing data-analytics data-integration cloud-native data-pipelines stateful streaming-data stream-processing-engine event-driven-architecture streaming-analytics streaming-data-processing streaming-data-pipelines

UpdatedFeb 17, 2026
Rust

javascriptdata /danfojs

Star5k

Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.

javascript data-science tensorflow table pandas stream-processing data-analytics data-analysis data-manipulation tensors dataframe stream-data plotting-charts danfojs

UpdatedFeb 18, 2026
TypeScript

ag2ai /faststream

Star5k

FastStream is a powerful and easy-to-use asynchronous Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.

python redis distributed-systems kafka rabbitmq nats stream-processing asyncio asyncapi faststream

UpdatedFeb 19, 2026
Python

ArroyoSystems /arroyo

Star4.8k

Distributed stream processing engine in Rust

infrastructure rust data sql kafka stream-processing stream-processing-engine dev-tools data-stream-processing

UpdatedFeb 19, 2026
Rust

Improve this page

Add a description, image, and links to thestream-processing topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thestream-processing topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly