tokyo-olympic-azure-data-engineering-project

In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our cloud provider.

This project builds an End-to-End Azure Data Engineering Pipeline, performing ETL and Analytics Reporting on the AdventureWorks2017LT Database.

Developed an end-to-end ML system on Azure to predict loan defaults, leveraging advanced data preprocessing, feature engineering, and machine learning models to optimize accuracy. This project includes a comprehensive suite of tools and techniques for robust financial risk assessment, deployed to enhance decision-making for high-risk exposures.

🔧 Azure Data Engineering Project (On-premise db to the cloud)

No description available

This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study

Real World Project on Formula1 Racing using Azure Databricks, Delta Lake and Azure Data Factory

No description available

End-to-End Azure Data Engineering Project using ADF, Databricks, Synapse, and Power BI.

Azure End To End Data Engineering Project

In this project, we used the Azure cloud services to get done data engineering operations (Ingestion, Transformation, Analysis, Load) on Formula-1 Racing Dataset Available from 'eargst developer API' which includes both 'CSV' and 'JSON' (single and also split) files. we also manage the incremental load and full load approach to dealing with some of that files and the notebook workflow via Azure Data Factory pipelines and  Azure Databricks Itself. we schedule that pipelines with tumbling window trigger to get execute that pipeline in a wise manner.

A complete End to End Data Engineering Project implemented using Azure Cloud provider. Pipeline are created to ingest data from SQL server database , transformed using azure Databricks service and finally data is stored in azure synapase analytics warehouse. Tableau is used to create report over this data.

End-to-end Azure Data Engineering Project with Databricks

In this Udemy course project created by Ramesh Retnasamy, I implemented a data engineering solution using all services available on Azure Synapse Analytics to analyze and report data on Taxi Green trips in New York City from 2020 to 2021.

No description available

No description available

This is a End-to-End Azure Data Engineering Project | Analysis on the entire ETL Pipeline - Azure Factory, Azure Lake Gen 2, Databricks, Azure Synapse Analytics & Dashboards

Project 4: Udacity Nanodegree Program - Data Engineering with Microsoft Azure

Databricks and Azure projects for experimenting, learning and demonstrating knowledge

An end-to-end data pipeline from on-prem SQL Server to Azure Data Lake Gen2 using Azure Data Factory, with layered data processing in Databricks, analytics via Azure Synapse, reporting in Power BI, and security through Azure Active Directory and Key Vault.

No description available

Database migration data engineering project (Azure Cloud)

This repo is for the Linkedin Learning course: Guided Lab: Real-World Data Engineering Project with Azure Fabric

No description available

An end-to-end data engineering project using Azure Synapse Analytics to analyze and transform NYC taxi data.

Formula 1 race data engineering project which utilises azure services and databricks to ingest and analyse the data.

Data Engineering Project on Covid19 Reporting – Using Azure Data Factory, Databricks, HDInsight, Azure Data Factory – An End to End ETL pipeline in addition to a Power BI report dashboard.

This project builds an End-to-End Azure Data Engineering Pipeline, performing ETL and Analytics Reporting on the AdventureWorks2022LT Database.

Open-source repository covering ML algorithms, Generative/Agentic AI, Big Data engineering, and cloud-based MLOps (Azure/AWS/GCP). Includes fundamentals and real-world end-to-end projects.