big-data

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

python aws data-science machine-learning caffe theano big-data spark deep-learning hadoop tensorflow numpy scikit-learn keras pandas kaggle scipy matplotlib mapreduce

UpdatedMar 20, 2024
Python

apache /flink

Star25.8k

Apache Flink

python java scala sql big-data flink

UpdatedFeb 7, 2026
Java

thingsboard /thingsboard

Star21.1k

Open-source IoT Platform - Device management, data collection, processing and visualization.

visualization http mqtt iot middleware cloud microservices big-data snmp iot-platform dashboards iiot coap-server iot-framework thingsboard iot-analytics lwm2m-server iot-solutions

UpdatedFeb 7, 2026
Java

amark /gun

Sponsor

Star18.9k

An open source cybersecurity protocol for syncing decentralized graph data.

machine-learning cryptography crypto encryption database big-data graph offline-first protocol end-to-end dapp decentralized blockchain realtime p2p artificial-intelligence crdt web3 metaverse dweb

UpdatedFeb 5, 2026
JavaScript

heibaiying /BigData-Notes

Star16.9k

大数据入门指南 ⭐

phoenix scala kafka big-data spark yarn hive hadoop storm bigdata hbase zookeeper hdfs mapreduce flume azkaban sqoop

UpdatedJan 5, 2024
Java

prestodb /presto

Star16.7k

The official home of the Presto distributed SQL query engine for big data

java data query sql big-data presto hive hadoop lakehouse

UpdatedFeb 7, 2026
Java

andkret /Cookbook

Star14.9k

The Data Engineering Cookbook

big-data best-practices cookbook data-engineering data-engineer

UpdatedJan 17, 2026
Python

apache /predictionio

Star12.5k

PredictionIO, a machine learning server for developers and ML engineers.

scala big-data predictionio

UpdatedJan 9, 2021
Scala

trinodb /trino

Star12.5k

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

java distributed-systems data-science sql database big-data presto hive hadoop analytics jdbc databases distributed-database query-engine iceberg datalake prestodb trino delta-lake

UpdatedFeb 7, 2026
Java

vesoft-inc /nebula

Star12k

A distributed, fast open-source graph database featuring horizontal scalability and high availability

distributed-systems database big-data cpp graph raft scalability distributed graph-database graphdb hacktoberfest nebula nebula-graph nebulagraph

UpdatedOct 22, 2025
C++

yahoo /CMAK

Star11.9k

CMAK is a tool for managing Apache Kafka clusters

scala kafka big-data cluster-management

UpdatedAug 2, 2023
Scala

provectus /kafka-ui

Star11.9k

Open-Source Web UI for Apache Kafka Management

opensource kafka big-data web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

UpdatedJul 26, 2024
Java

StarRocks /starrocks

Star11.4k

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

sql database big-data analytics olap join distributed-database realtime-database mpp cloudnative iceberg real-time-analytics datalake vectorized real-time-updates star-schema hudi delta-lake lakehouse lakehouse-platform

UpdatedFeb 7, 2026
Java

quickwit-oss /quickwit

Star10.9k

Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.

rust open-source search-engine big-data logs cloud-storage cloud-native log-management distributed-tracing tantivy

UpdatedFeb 6, 2026
Rust

cython /cython

Star10.6k

The most widely used Python to C compiler

python c performance big-data cpp cython cpython cpython-extensions

UpdatedFeb 7, 2026
Cython

catboost /catboost

Star8.8k

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

python data-science machine-learning data-mining tutorial r big-data gpu cuda kaggle gbdt gbm gpu-computing decision-trees gradient-boosting coreml catboost categorical-features

UpdatedFeb 7, 2026
C++

delta-io /delta

Star8.6k

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

big-data spark analytics acid delta-lake

UpdatedFeb 7, 2026
Scala

Improve this page

Add a description, image, and links to thebig-data topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thebig-data topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly