bigdata

🚀2.3x faster than MinIO for 4KB object payloads. RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.

rust filesystem bigdata s3 minio cloud-native objectstorage object-storage amazon-s3

UpdatedFeb 20, 2026
Rust

apache /shardingsphere

Star20.7k

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

mysql sql database bigdata postgresql shard distributed-database encrypt data-pipeline data-encryption database-cluster distributed-transaction read-write-splitting database-middleware distributed-sql-database database-gateway

UpdatedFeb 20, 2026
Java

heibaiying /BigData-Notes

Star16.9k

大数据入门指南 ⭐

phoenix scala kafka big-data spark yarn hive hadoop storm bigdata hbase zookeeper hdfs mapreduce flume azkaban sqoop

UpdatedJan 5, 2024
Java

oxnr /awesome-bigdata

Star14.2k

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

UpdatedFeb 5, 2026

juicedata /juicefs

Star13.2k

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

go redis golang distributed-systems storage filesystem bigdata s3 posix hdfs cloud-native object-storage

UpdatedFeb 17, 2026
Go

wangzhiwubigdata /God-Of-BigData

Star10.4k

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

kafka spark hive hadoop bigdata hbase zookeeper hdfs flume flink azkaban

UpdatedAug 7, 2023

databendlabs /databend

Star9.2k

Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

rust elasticsearch sql database ai serverless geospatial bigdata snowflake olap cloud-native vector-search vector-database lakehouse

UpdatedFeb 20, 2026
Rust

vaexio /vaex

Star8.5k

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

visualization python data-science machine-learning bigdata tabular-data hdf5 machinelearning dataframe memory-mapped-file pyarrow

UpdatedFeb 5, 2026
Python

apache /hudi

Star6.1k

Upserts, Deletes And Incremental Processing on Big Data.

bigdata stream-processing data-integration datalake apachespark hudi apachehudi incremental-processing apacheflink

UpdatedFeb 19, 2026
Java

volcano-sh /volcano

Star5.3k

A Cloud Native Batch System (Project under CNCF)

training kubernetes golang machine-learning gene ai hpc bigdata serving batch-systems

UpdatedFeb 13, 2026
Go

iGaoWei /BigDataView

Star4.9k

100+套大数据可视化炫酷大屏Html5模板；包含行业：社区、物业、政务、交通、金融银行等，全网最新、最多，最全、最酷、最炫大数据可视化模板。陆续更新中

bigdata html-template viewmodel echarts bigdataviewer

UpdatedOct 24, 2025
JavaScript

DTStack /chunjun

Star4.1k

A data integration framework

java framework bigdata data-integration flink

UpdatedDec 2, 2025
Java

liyupi /sql-generator

Star3.5k

🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~

javascript mysql json typescript sql spark vue hive bigdata monaco-editor ant-design vue3 vite

UpdatedJan 17, 2024
Vue

apache /avro

Star3.2k

Apache Avro is a data serialization system.

ruby python c java php csharp cplusplus avro dotnet perl bigdata

UpdatedFeb 19, 2026
Java

MoRan1607 /BigDataGuide

Star3.1k

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

scala kafka spark hive hadoop bigdata hbase zookeeper flume flink javase

UpdatedJan 20, 2026

douban /dpark

Star2.7k

Python clone of Spark, a MapReduce alike framework in Python

python spark bigdata stream-processing mapreduce dpark

UpdatedDec 25, 2020
Python

griddb /griddb

Star2.5k

GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.

fast iot sql database timeseries time-series nosql bigdata newsql griddb

UpdatedFeb 18, 2026
C++

dotnet /spark

Star2.1k

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

microsoft emr machine-learning streaming spark apache-spark csharp fsharp analytics dotnet azure bigdata spark-streaming dotnet-core dotnet-standard databricks spark-sql tpcds hdinsight tpch

UpdatedFeb 13, 2026
C#

Improve this page

Add a description, image, and links to thebigdata topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thebigdata topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly