stream-processing

Star

Here are 1,029 public repositories matching this topic...

Language:All

Filter by language

All1,029 Java256 Python193 Go99 Scala59 C++56 JavaScript55 Jupyter Notebook39 Rust34 TypeScript25 C#20

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

google-ai-edge /mediapipe

Star30.7k

Cross-platform, customizable ML solutions for live and streaming media.

android c-plus-plus calculator machine-learning framework computer-vision deep-learning inference pipeline-framework stream-processing video-processing perception mobile-development audio-processing graph-framework graph-based mediapipe

UpdatedJul 18, 2025
C++

pathwaycom /pathway

Star28.8k

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

python rust streaming real-time kafka etl machine-learning-algorithms stream-processing data-analytics dataflow data-processing data-pipelines batch-processing pathway iot-analytics etl-framework time-series-analysis

UpdatedJul 18, 2025
Python

vectordotdev /vector

Star20k

A high-performance observability data pipeline.

rust parser events router pipeline metrics vector logs stream-processing forwarder observability

UpdatedJul 18, 2025
Rust

flink learning blog.http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

mysql redis elasticsearch streaming kafka spark influxdb rabbitmq clickhouse hbase stream-processing opentsdb loki flink rocketmq

UpdatedMar 12, 2025
Java

oxnr /awesome-bigdata

Star13.7k

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

UpdatedFeb 14, 2025

redpanda-data /redpanda

Star10.6k

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

go kubernetes streaming microservices kafka cpp seastar storage-engine containers realtime stream-processing event-driven redpanda

UpdatedJul 18, 2025
C++

madd86 /awesome-system-design

Star10.5k

A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

distributed-systems microservices nosql interview stream-processing microservices-architecture relational-database message-broker hadoop-ecosystem

UpdatedJun 27, 2024

ThreeDotsLabs /watermill

Star8.7k

Building event-driven applications the easy way in Go.

go golang events reactive kafka cqrs sagas rabbitmq event-sourcing nats stream-processing event-driven watermill

UpdatedJul 3, 2025
Go

redpanda-data /connect

Star8.4k

Fancy stream processing made operationally mundane

go golang kafka cqrs etl rabbitmq amqp logs message-bus event-sourcing nats stream-processing message-queue data-engineering streaming-data stream-processor data-ops

UpdatedJul 18, 2025
Go

risingwavelabs /risingwave

Star8.1k

Stream processing and management platform.

rust database kafka postgresql stream-processing data-engineering materialized-view apache-iceberg

UpdatedJul 18, 2025
Rust

fluent /fluent-bit

Star7.1k

Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows

c metrics logging logs prometheus data-collector stream-processing fluentd fluent-bit cloudnative sql-queries forwarder traces opentelemetry

UpdatedJul 18, 2025
C

robinhood /faust

Star6.8k

Python Stream Processing

python distributed-systems kafka stream-processing asyncio kafka-streams

UpdatedJul 27, 2024
Python

hazelcast /hazelcast

Star6.4k

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

java caching distributed-systems real-time big-data hazelcast scalability distributed-computing distributed stream-processing in-memory low-latency hacktoberfest data-insights data-in-motion

UpdatedJul 17, 2025
Java

MaterializeInc /materialize

Star6.1k

Real-time Data Integration and Transformation: use SQL to transform, deliver, and act on fast-changing data.

rust distributed-systems streaming sql database kafka postgresql data-store stream-processing streaming-data materialized-view postgresql-dialect operational-data-store

UpdatedJul 18, 2025
Rust

apache /hudi

Star5.9k

Upserts, Deletes And Incremental Processing on Big Data.

bigdata stream-processing data-integration datalake apachespark hudi apachehudi incremental-processing apacheflink

UpdatedJul 18, 2025
Java

online-ml /river

Star5.4k

🌊 Online machine learning in Python

python data-science machine-learning streaming stream-processing online-learning streaming-data concept-drift real-time-processing incremental-learning online-machine-learning online-statistics

UpdatedJul 2, 2025
Python

infinyon /fluvio

Star5k

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

rust distributed-systems streaming real-time serverless webassembly data-flow stream-processing data-analytics data-integration cloud-native data-pipelines stateful streaming-data stream-processing-engine event-driven-architecture streaming-analytics streaming-data-processing streaming-data-pipelines

UpdatedJul 15, 2025
Rust

javascriptdata /danfojs

Star5k

Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.

javascript data-science tensorflow table pandas stream-processing data-analytics data-analysis data-manipulation tensors dataframe stream-data plotting-charts danfojs

UpdatedJun 24, 2025
TypeScript

ArroyoSystems /arroyo

Star4.4k

Distributed stream processing engine in Rust

infrastructure rust data sql kafka stream-processing stream-processing-engine dev-tools data-stream-processing

UpdatedJul 12, 2025
Rust

ag2ai /faststream

Star4.2k

FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.

python redis distributed-systems kafka rabbitmq nats stream-processing asyncio asyncapi propan faststream fastkafka

UpdatedJul 18, 2025
Python

Improve this page

Add a description, image, and links to thestream-processing topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thestream-processing topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly