Glue scripts for converting AWS Service Logs for use in Athena

• UpdatedFeb 1, 2024
• Python

The Sensitive Data Protection on AWS solution allows enterprise customers to create data catalogs, discover, protect, and visualize sensitive data across multiple AWS accounts. The solution eliminates the need for manual tagging to track sensitive data such as Personal Identifiable Information (PII) and classified information.

• UpdatedFeb 6, 2025
• TypeScript

Build and deploy a serverless data pipeline on AWS with no effort.

• UpdatedFeb 8, 2023
• Python

AWS ETL Pipleine

• UpdatedMay 16, 2024
• HCL

Extract, transform, and load data for analytic processing using AWS Glue

• UpdatedMay 2, 2021
• Python

AWS streaming pipeline for real-time analysis

• UpdatedMar 11, 2024
• Python

This is a data pipeline built with the purpose of serving a business team.

• UpdatedFeb 28, 2023
• Python

Terraform configuration that creates several AWS services, uploads data in S3 and starts the Glue Crawler and Glue Job.

• UpdatedFeb 10, 2022
• Python

Terraform module which creates Glue Job resources on AWS.

• UpdatedJun 4, 2022
• HCL

A cloud-based ETL pipeline on AWS for automating airline flight data ingestion, transformation, and storage using S3, Glue, Redshift, EventBridge, Step Functions, and SNS.

• UpdatedFeb 17, 2025
• Python

This project outlines the final project requirements for Information Architectures, focusing on group assignments, scoring criteria, topic selection, core requirements, and project components such as design, development, visualization, and executive presentation.

• UpdatedJan 6, 2025
• HTML

• UpdatedOct 28, 2023
• Python

Terraform module to create and manage a AWS Glue job

• UpdatedJan 22, 2025
• HCL

🎈 Learn how to repair inflatables, handle them, where to buy tools / products, and minimize mistakes & risks

• UpdatedFeb 9, 2025

Pipeline ETL na AWS

• UpdatedJul 28, 2024
• Python

Data Engineering project using data streaming produced by python applications, ETL process and availability for ad-hoc SQL queries in the AWS cloud

• UpdatedJul 5, 2024
• Jupyter Notebook

This project creates a serverless data pipeline to extract data from the Colombo Stock Market ASI Index API using AWS Lambda, Kinesis Firehose, and S3. An AWS Glue workflow processes and transforms the data, storing it in an Apache Iceberg table via Athena and Glue ETL jobs.

• UpdatedJul 2, 2024
• Python

IMDB Movie Data ETL Pipeline using S3, Glue, Redshift, EventBridge, SNS

• UpdatedAug 6, 2024
• Python

DeepLearning.AI & AWS Data Engineering Course Exercises

• UpdatedJan 5, 2025
• Jupyter Notebook

This project is an end-to-end, fully automated warehouse management solution designed to tackle real-world inventory challenges in the FMCG sector. From real-time data ingestion and predictive analytics to interactive dashboards, this project combines cutting-edge technologies and an event-driven architecture to simulate a business-ready system.

• UpdatedDec 28, 2024
• Python