streaming-data

Star

Here are 511 public repositories matching this topic...

Language:All

Filter by language

All511 Python141 Java53 Jupyter Notebook52 JavaScript45 Go26 TypeScript26 C#17 C++16 Scala12 HTML11

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

oxnr /awesome-bigdata

Star14.1k

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

UpdatedNov 27, 2025

provectus /kafka-ui

Star11.7k

Open-Source Web UI for Apache Kafka Management

opensource kafka big-data web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

UpdatedJul 26, 2024
Java

johnkerl /miller

Star9.6k

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

tsv devops json statistics csv command-line json-data tabular-data data-reduction unix-toolkit statistical-analysis csv-format devops-tools data-regression data-processing command-line-tools data-cleaning streaming-algorithms streaming-data miller

UpdatedDec 17, 2025
Go

redpanda-data /connect

Star8.5k

Fancy stream processing made operationally mundane

go golang kafka cqrs etl rabbitmq amqp logs message-bus event-sourcing nats stream-processing message-queue data-engineering streaming-data stream-processor data-ops

UpdatedDec 17, 2025
Go

MaterializeInc /materialize

Star6.2k

The live data layer for apps and AI agents Create up-to-the-second views into your business, just using SQL

mysql rust distributed-systems streaming sql sql-server database kafka postgresql data-store stream-processing cdc streaming-data materialized-view data-mesh postgresql-dialect operational-data-store

UpdatedDec 17, 2025
Rust

online-ml /river

Star5.6k

🌊 Online machine learning in Python

python data-science machine-learning streaming stream-processing online-learning streaming-data concept-drift real-time-processing incremental-learning online-machine-learning online-statistics

UpdatedDec 16, 2025
Python

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

mysql rust caching postgres sql backend cache postgresql databases rust-lang mysql-database postgresql-database streaming-data caching-proxy

UpdatedDec 16, 2025
Rust

fluvio-community /fluvio

Star5.1k

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

rust distributed-systems streaming real-time serverless webassembly data-flow stream-processing data-analytics data-integration cloud-native data-pipelines stateful streaming-data stream-processing-engine event-driven-architecture streaming-analytics streaming-data-processing streaming-data-pipelines

UpdatedDec 16, 2025
Rust

memgraph /memgraph

Star3.5k

Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.

kafka graph graph-algorithms nosql stream-processing graph-database kafka-streams cypher graph-analysis streaming-data opencypher

UpdatedDec 17, 2025
C++

piskvorky /smart_open

Sponsor

Star3.4k

Utils for streaming large files (S3, HDFS, gzip, bz2...)

python streaming s3 file hdfs hacktoberfest webhdfs boto gzip-stream bz2 streaming-data

UpdatedDec 1, 2025
Python

reugn /go-streams

Star2.1k

A lightweight stream processing library for Go

redis workflow kafka pipeline etl websocket data-stream streams stream-processing pulsar windowing aerospike streaming-api kafka-streams throttling data-pipeline streaming-data nats-streaming stream-processor low-code

UpdatedNov 20, 2025
Go

pravega /pravega

Star2k

Pravega - Streaming as a new software defined storage primitive

streaming distributed-storage real-time-data streaming-data data-ingestion

UpdatedMar 2, 2025
Java

bytewax /bytewax

Star1.9k

Python Stream Processing

python rust data-science machine-learning stream-processing data-engineering dataflow data-processing streaming-data

UpdatedMar 27, 2025
Python

kafbat /kafka-ui

Sponsor

Star1.8k

Open-Source Web UI for managing Apache Kafka clusters

opensource kafka big-data foss web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

UpdatedDec 15, 2025
Java

quixio /quix-streams

Star1.5k

Python Streaming DataFrames for Kafka

python data-science machine-learning kafka stream-processing data-engineering streaming-data stream-processor event-driven-architecture streaming-data-processing time-series-data data-intensive-applications streaming-data-pipelines real-time-data-processing

UpdatedNov 18, 2025
Python

python-streamz /streamz

Star1.3k

Real-time stream processing for python

python real-time async streaming-data

UpdatedDec 16, 2025
Python

microsoft /Trill

Star1.3k

Trill is a single-node query processor for temporal or streaming data.

streaming-data temporal-data

UpdatedJan 8, 2024
C#

zpl-c /zpl

Star1k

📐 Pushing the boundaries of simplicity

c hashing cli helper time cross-platform math cpp timer coroutines tar header-only threading thread-pool memory-management zpl memory-allocation csv-parser streaming-data json5-parser

UpdatedApr 26, 2025
C

DoneDeal0 /superdiff

Sponsor

Star924

Superdiff provides a complete and readable diff for both arrays and objects. Plus, it supports stream and file inputs for handling large datasets efficiently, is battle-tested, has zero dependencies, and is super fast.

react nodejs diff streaming typescript comparison json-diff streaming-data comparison-tool object-comparison deep-diff array-comparison object-diff objectdifference objectdiff