hadoop

Star

Here are 3,570 public repositories matching this topic...

Language:All

Filter by language

All3,570 Java1,124 Python543 Shell415 Jupyter Notebook350 Scala139 HTML88 Dockerfile81 JavaScript67 Go31 PigLatin19

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

donnemartin /data-science-ipython-notebooks

Star28.4k

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

python aws data-science machine-learning caffe theano big-data spark deep-learning hadoop tensorflow numpy scikit-learn keras pandas kaggle scipy matplotlib mapreduce

UpdatedMar 20, 2024
Python

spotify /luigi

Star18.4k

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

python hadoop scheduling orchestration-framework luigi

UpdatedMay 16, 2025
Python

Tencent /APIJSON

Star18.1k

🏆 实时零代码、全功能、强安全 ORM 库 🚀 后端接口和文档零代码，前端(客户端) 定制返回 JSON 的数据和结构 🏆 Real-Time coding-free, powerful and secure ORM 🚀 providing APIs and Docs without coding by Backend, and the returned JSON of API can be customized by Frontend(Client) users

elasticsearch crud influxdb hive hadoop serverless clickhouse postgresql oracle snowflake baas postgresql-database sqlserver tidb databricks low-code nocode lowcode milvus tdengine

UpdatedJul 8, 2025
Java

heibaiying /BigData-Notes

Star16.5k

大数据入门指南 ⭐

phoenix scala kafka big-data spark yarn hive hadoop storm bigdata hbase zookeeper hdfs mapreduce flume azkaban sqoop

UpdatedJan 5, 2024
Java

prestodb /presto

Star16.4k

The official home of the Presto distributed SQL query engine for big data

java data query sql big-data presto hive hadoop lakehouse

UpdatedJul 18, 2025
Java

apache /hadoop

Star15.2k

Apache Hadoop

hadoop

UpdatedJul 18, 2025
Java

deeplearning4j /deeplearning4j

Star14k

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...

python java clojure scala spark hadoop gpu intellij linear-algebra artificial-intelligence deeplearning neural-nets dl4j matrix-library deeplearning4j

UpdatedJul 18, 2025
Java

trinodb /trino

Star11.6k

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

java distributed-systems data-science sql database big-data presto hive hadoop analytics jdbc databases distributed-database query-engine iceberg datalake prestodb trino delta-lake

UpdatedJul 18, 2025
Java

wangzhiwubigdata /God-Of-BigData

Star10.2k

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

kafka spark hive hadoop bigdata hbase zookeeper hdfs flume flink azkaban

UpdatedAug 7, 2023

linkedin /school-of-sre

Star8k

At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.

mysql python git linux security networking hadoop nosql sre system-design

UpdatedAug 13, 2024
HTML

h2oai /h2o-3

Star7.2k

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

python java data-science machine-learning opensource r big-data spark deep-learning hadoop random-forest gpu naive-bayes h2o distributed pca gbm ensemble-learning automl h2o-automl

UpdatedJul 18, 2025
Jupyter Notebook

Alluxio /alluxio

Star7k

Alluxio, data orchestration for analytics and machine learning in the cloud

spark presto hadoop tensorflow data-analysis alluxio memory-speed data-orchestration virtual-distributed-filesystem

UpdatedApr 29, 2025
Java

HariSekhon /DevOps-Bash-tools

Star6.9k

1000+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..

github mysql git linux api bash docker kubernetes jenkins aws devops kafka hadoop ci terraform perl cloudera postgresql gcp hacktoberfest

UpdatedJul 8, 2025
Shell

apache /hive

Star5.7k

Apache Hive

java sql database big-data hive hadoop apache

UpdatedJul 18, 2025
Java

apache /ignite

Star5k

Apache Ignite

iot cloud sql database big-data hadoop cache osgi ignite network-server in-memory-database data-management-platform network-client distributed-sql-database in-memory-computing

UpdatedJul 18, 2025
Java

apache /calcite

Star4.9k

Apache Calcite

java sql big-data hadoop geospatial calcite

UpdatedJul 16, 2025
Java

tomwhite /hadoop-book

Star3.5k

Example source code accompanying O'Reilly's "Hadoop: The Definitive Guide" by Tom White

hadoop book o-reilly

UpdatedMar 17, 2020
Makefile

WeBankFinTech /DataSphereStudio

Star3.2k

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.