azure-databricks

A cloud-native data pipeline and visualization project analyzing Formula 1 racing data using Azure, Databricks, Delta Lake, Tableau, and Python for insightful EDA and interactive dashboards.

exploratory-data-analysis data-visualization seaborn pyspark matplotlib tableau spark-sql multivariate-analysis azure-data-factory bi-dashboard azure-databricks delta-lake lakehouse unity-catalog medallion-architecture azure-data-lake-storage-gen2 cloud-native-etl batch-and-stream-unification storage-compute-separation

UpdatedJul 23, 2025
Jupyter Notebook

microsoft /A-TALE-OF-THREE-CITIES

Star88

Analyzing the safety (311) dataset published by Azure Open Datasets for Chicago, Boston and New York City using SparkR, SParkSQL, Azure Databricks, visualization using ggplot2 and leaflet. Focus is on descriptive analytics, visualization, clustering, time series forecasting and anomaly detection.

visualization data r azure leaflet geospatial opendata eda sparksql workshop-materials sparkr anomalydiscovery time-series-analysis anomaly-detection databricks-notebooks timeseries-forecasting azure-databricks datascience-machinelearning aiforsocialgood 311-data

UpdatedMay 3, 2021
R

AdamPaternostro /Azure-Databricks-Dev-Ops

Star68

Complete end to end sample of doing DevOps with Azure Databricks

azure-databricks azure-devops github-actions azure-dev-ops databricks-workspace

UpdatedFeb 2, 2022
Shell

Jayvardhan-Reddy /Azure-Certification-DP-200

Star67

Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution

azure data-storage resources data-engineering data-lake azure-storage batch-processing data-engineer azure-data-factory microsoft-azure azure-portal azure-cosmos-db azure-services polybase certification-prep azure-databricks exam-prep azure-certification dp-200 sql-dw

UpdatedAug 5, 2020

tomaztk /Azure-Databricks

Star63

Azure Databricks - Advent of 2020 Blogposts

python machine-learning scala sql spark notebook pyspark data-analytics mllib r-language notebooks sparkr databricks azure-data-factory databricks-notebooks spark-structured-streaming azure-databricks mlflow azure-machine-learnning data-engineerg

UpdatedSep 22, 2022
Jupyter Notebook

bhavink /databricks

Star51

Databricks Platform - Architecture, Security, Automation and much more!!

security databricks azure-databricks deltalake databricks-rest-api databricks-dbconnect databricks-platform databricks-cli

UpdatedDec 11, 2025
Jupyter Notebook

rafaelpierre /pyjaws

Star44

PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows

python workflow airflow spark apache-spark orchestration pyspark data-engineering mage cicd databricks apache-airflow azure-databricks mlops

UpdatedDec 13, 2025
Python

jrlasak /databricks_apparel_streaming

Star29

Databricks DLT Apparel Pipeline Project: Learn medallion architecture, streaming, and data engineering with Delta Live Tables. Includes synthetic data, step-by-step guide, and certification prep.

etl pyspark data-engineering learning-by-doing data-pipelines databricks dlt azure-databricks lakehouse delta-live-tables medallion-architecture

UpdatedNov 4, 2025
Python

airscholar /FootballDataEngineering

Star29

An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.

data-engineering dataengineering azure-data-factory apache-airflow azure-databricks azure-synapse-analytics azure-data-lake-gen2