streaming-data

Star

Here are 539 public repositories matching this topic...

Language:All

Filter by language

All539 Python150 Java56 Jupyter Notebook53 JavaScript44 TypeScript29 Go27 C#20 C++16 HTML12 Rust12

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

oxnr /awesome-bigdata

Star14.2k

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

UpdatedFeb 5, 2026

provectus /kafka-ui

Star11.9k

Open-Source Web UI for Apache Kafka Management

opensource kafka big-data web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

UpdatedJul 26, 2024
Java

johnkerl /miller

Star9.7k

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

tsv devops json statistics csv command-line json-data tabular-data data-reduction unix-toolkit statistical-analysis csv-format devops-tools data-regression data-processing command-line-tools data-cleaning streaming-algorithms streaming-data miller

UpdatedFeb 18, 2026
Go

redpanda-data /connect

Star8.6k

Fancy stream processing made operationally mundane

go golang kafka cqrs etl rabbitmq amqp logs message-bus event-sourcing nats stream-processing message-queue data-engineering streaming-data stream-processor data-ops

UpdatedFeb 20, 2026
Go

MaterializeInc /materialize

Star6.2k

The live data layer for apps and AI agents Create up-to-the-second views into your business, just using SQL

mysql rust distributed-systems streaming sql sql-server database kafka postgresql data-store stream-processing cdc streaming-data materialized-view data-mesh postgresql-dialect operational-data-store

UpdatedFeb 20, 2026
Rust

online-ml /river

Star5.7k

🌊 Online machine learning in Python

python data-science machine-learning streaming stream-processing online-learning streaming-data concept-drift real-time-processing incremental-learning online-machine-learning online-statistics

UpdatedFeb 9, 2026
Python

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

mysql rust caching postgres sql backend cache postgresql databases rust-lang mysql-database postgresql-database streaming-data caching-proxy

UpdatedFeb 20, 2026
Rust

fluvio-community /fluvio

Star5.2k

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

rust distributed-systems streaming real-time serverless webassembly data-flow stream-processing data-analytics data-integration cloud-native data-pipelines stateful streaming-data stream-processing-engine event-driven-architecture streaming-analytics streaming-data-processing streaming-data-pipelines

UpdatedFeb 17, 2026
Rust

memgraph /memgraph

Star3.7k

Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.

kafka graph graph-algorithms nosql stream-processing graph-database kafka-streams cypher graph-analysis streaming-data opencypher

UpdatedFeb 20, 2026
C++

piskvorky /smart_open

Sponsor

Star3.4k

Utils for streaming large files (S3, HDFS, gzip, bz2...)

python streaming s3 file hdfs hacktoberfest webhdfs boto gzip-stream bz2 streaming-data

UpdatedJan 28, 2026
Python

reugn /go-streams

Star2.2k

A lightweight stream processing library for Go

redis workflow kafka pipeline etl websocket data-stream streams stream-processing pulsar windowing aerospike streaming-api kafka-streams throttling data-pipeline streaming-data nats-streaming stream-processor low-code

UpdatedJan 14, 2026
Go

kafbat /kafka-ui

Sponsor

Star2.1k

Open-Source Web UI for managing Apache Kafka clusters

opensource kafka big-data foss web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

UpdatedFeb 19, 2026
TypeScript

pravega /pravega

Star2k

Pravega - Streaming as a new software defined storage primitive

streaming distributed-storage real-time-data streaming-data data-ingestion

UpdatedMar 2, 2025
Java

bytewax /bytewax

Star2k

Python Stream Processing

python rust data-science machine-learning stream-processing data-engineering dataflow data-processing streaming-data

UpdatedMar 27, 2025
Python

quixio /quix-streams

Star1.5k

Python Streaming DataFrames for Kafka

python data-science machine-learning kafka stream-processing data-engineering streaming-data stream-processor event-driven-architecture streaming-data-processing time-series-data data-intensive-applications streaming-data-pipelines real-time-data-processing

UpdatedFeb 18, 2026
Python

python-streamz /streamz

Star1.3k

Real-time stream processing for python

python real-time async streaming-data

UpdatedFeb 5, 2026
Python

microsoft /Trill

Star1.3k

Trill is a single-node query processor for temporal or streaming data.

streaming-data temporal-data

UpdatedJan 8, 2024
C#

DoneDeal0 /superdiff

Sponsor

Star1.1k

Superdiff provides a rich and readable diff for both arrays and objects. It supports stream and file inputs for handling large datasets efficiently, is battle-tested, has zero dependencies, and offer a top-tier performance.

react nodejs diff streaming typescript comparison json-diff streaming-data comparison-tool object-comparison deep-diff array-comparison object-diff objectdifference objectdiff