Movatterモバイル変換

Skip to content

#

apache-spark

Apache Spark is an open source distributed general-purpose cluster-computing framework. It provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.

Here are 2,077 public repositories matching this topic...

Language:All

Filter by language

All2,077 Python534 Jupyter Notebook522 Scala372 Java245 HTML53 Shell47 Dockerfile20 JavaScript19 R15 CSS12

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

mlflow

mlflow /mlflow

The open source developer platform to build AI/LLM applications and models with confidence. Enhance your AI applications with end-to-end tracking, observability, and evaluations, all in one integrated platform.

open-source machine-learning ai apache-spark evaluation ml openai agents observability model-management mlops mlflow agentops prompt-engineering ai-governance langchain llmops llm-evaluation

UpdatedNov 6, 2025
Python

SynapseML

microsoft /SynapseML

Simple and Distributed Machine Learning

microsoft http opencv data-science machine-learning scala big-data ai spark apache-spark deep-learning azure ml pyspark lightgbm cognitive-services databricks synapse model-deployment onnx

UpdatedOct 30, 2025
Scala

lakeFS

treeverse /lakeFS

lakeFS - Data version control for your data lake | Git for data

go golang apache-spark aws-s3 google-cloud-storage data-engineering data-lake azure-storage data-version-control object-storage datalake hadoop-filesystem data-quality data-versioning azure-blob-storage apache-sparksql git-for-data lakefs datalakes

UpdatedNov 5, 2025
Go

lw-lin /CoolplaySpark

酷玩 Spark: Spark 源代码解析、Spark 类库等

spark apache-spark sparkcore spark-streaming structured-streaming

UpdatedMay 18, 2022
Scala

spark-notebook /spark-notebook

Interactive and Reactive Data Science using Scala and Spark.

data-science reactive scala spark apache-spark notebook

UpdatedMay 16, 2023
JavaScript

kubeflow /spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

kubernetes spark apache-spark kubernetes-operator kubernetes-controller kubernetes-crd google-cloud-dataproc

UpdatedNov 5, 2025
Go

intel /BigDL

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

python scala apache-spark pytorch keras-tensorflow bigdl distributed-deep-learning deep-neural-network analytics-zoo

UpdatedOct 14, 2025
Jupyter Notebook

dotnet /spark

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

microsoft emr machine-learning streaming spark apache-spark csharp fsharp analytics dotnet azure bigdata spark-streaming dotnet-core dotnet-standard databricks spark-sql tpcds hdinsight tpch

UpdatedSep 24, 2025
C#

big-data-europe /docker-spark

Apache Spark docker image

docker kubernetes apache-spark spark-kubernetes k8s-spark

UpdatedApr 21, 2023
Shell

feathr-ai /feathr

Feathr – A scalable, unified data and AI engineering platform for enterprise

data-science machine-learning apache-spark azure artificial-intelligence data-engineering feature-engineering data-quality mlops feature-store feature-management feature-marketplace feature-governance feature-metadata feature-platform

UpdatedApr 4, 2024
Scala

awesome-spark /awesome-spark

A curated list of awesome Apache Spark packages and resources.

awesome apache-spark pyspark sparkr

UpdatedOct 24, 2024
Shell

OryxProject /oryx

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

java machine-learning kafka apache-spark cloudera apache-kafka lambda-architecture oryx

UpdatedAug 16, 2021
Java

ptyadana /SQL-Data-Analysis-and-Visualization-Projects

SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.

mysql python postgres sql apache-spark sqlite postgresql challenges pyspark mysql-database data-analysis exercises tableau sql-queries pgadmin mysqlworkbench mysql-notes digital-music-store sql-data-analysis

UpdatedJul 18, 2022
Jupyter Notebook

japila-books /apache-spark-internals

The Internals of Apache Spark

spark apache-spark book internals

UpdatedJul 5, 2025

goodreads_etl_pipeline

san089 /goodreads_etl_pipeline

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

python airflow spark apache-spark scheduler s3 data-engineering data-lake warehouse redshift data-migration livy etl-framework apache-airflow emr-cluster etl-pipeline etl-job data-engineering-pipeline airflow-dag goodreads-data-pipeline

UpdatedMar 9, 2020
Python

databricks /LearningSparkV2

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

spark apache-spark mllib structured-streaming spark-sql spark-mllib mlflow delta-lake

UpdatedJan 28, 2025
Scala

lensacom /sparkit-learn

PySpark + Scikit-learn = Sparkit-learn

python machine-learning apache-spark scikit-learn distributed-computing

UpdatedDec 31, 2020
Python

graphframes

graphframes /graphframes

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

big-data spark apache-spark graphs pyspark networks dataframe dataframes network-motifs connected-components network-motif

UpdatedOct 28, 2025
Scala

mahmoudparsian /data-algorithms-book

MapReduce, Spark, Java, and Scala for Data Algorithms Book

python java machine-learning scala apache-spark distributed-computing design-patterns pyspark mapreduce reducers partitioning hadoop-mapreduce distributed-algorithms mappers data-algorithms apache-hadoop

UpdatedOct 14, 2024
Java

databricks /spark-sklearn

(Deprecated) Scikit-learn integration package for Apache Spark

machine-learning apache-spark scikit-learn grid-search parameter-tuning

UpdatedDec 3, 2019
Python

Created by Matei Zaharia

Released May 26, 2014

Followers: 433 followers
Repository: apache/spark
Website: github.com/topics/spark
Wikipedia: Wikipedia

Related topics

[8]ページ先頭

©2009-2025 Movatter.jp