PySpark-Tutorial provides basic algorithms using PySpark

• UpdatedMay 26, 2025
• Jupyter Notebook

Implementation of Spark code in Jupyter notebook. Topics include: RDDs and DataFrame, exploratory data analysis (EDA), handling multiple DataFrames, visualization, Machine Learning

• UpdatedAug 26, 2020
• Jupyter Notebook

A big data project to develop a real-time data pipeline for analyzing the popularity and sentiments of trending topics on Twitter.

• UpdatedJun 21, 2022
• Scala

Getting started with PySpark for Big data analysis

• UpdatedAug 24, 2022
• Jupyter Notebook

Efficiently tackle large datasets and perform big data analysis with Spark and Python

• UpdatedJan 11, 2019
• Python

MapReduce Job Development, RDDs Programming, Medical Data Management, Sales Analysis, And Efficient Data Integration For Big Data Analysis. Spark: Big Data Processing, SQOOP Integration, And Spark Structured Streaming For Real-Time Data.

• UpdatedJun 7, 2023
• Java

This repository is part of my journey to learn **PySpark**, the Python API for Apache Spark. I explored the fundamentals of distributed data processing using Spark and practiced with real-world data transformation and querying use cases.

• UpdatedJun 28, 2025
• Jupyter Notebook

📚 Master PySpark in 18 days with structured lessons, hands-on tasks, and an end-to-end project, covering essential concepts and ML model training.

• UpdatedFeb 20, 2026

This assignment was part of an IoT motion sensor App running on a watch, predicting actions of the individual wearing the watch based on his arm movements; this IoT Analytics assignments is one of a series of data pipeline coding challenges in the IBM course Scalable Data Science.

• UpdatedJul 30, 2022
• Jupyter Notebook

• UpdatedSep 16, 2025
• Scala

Here I play with the services offered by Apache Spark and try to learn them in more depth.

• UpdatedJun 4, 2021
• Jupyter Notebook

Data Mining using Spark Rdds

• UpdatedJan 15, 2024
• Python

Spark, RDDs and Map Reduce applications related to the BigData@polito course (2019-2020). A set of personal notes are already provided.

• UpdatedSep 9, 2020
• Java

Project on MapReduce for the Μ111 - Big Data Management course, NKUA, Spring 2023.

• UpdatedJul 21, 2023
• TeX

Analysis of Clinical Trial Dataset using PySpark RDD implementation.

• UpdatedMay 22, 2022

📈📊 Big Data Notebooks . ▫️ Análisis masivos de datos con pyspark ▫️ Ingesta de datos. ▫️ Algoritmos de machine learning con datos masivos. ▫️ Procesamiento de mensajes en tiempo real con Kafka.

• UpdatedAug 31, 2024
• Jupyter Notebook

This project illustrates Apache Spark RDD operations, from creation and transformation to actions and results, enhancing users' understanding of distributed data processing.

• UpdatedAug 21, 2023
• Jupyter Notebook