In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our cloud provider.

Demo repository implementing an end-to-end MLOps workflow on Databricks, using Azure DevOps for CICD orchestration. Project derived from dbx basic python template

End-to-end Azure DE project with Australia Health Expenditure dataset. Services used include Azure Data Factory, DataBricks, Data Lake, Key Vault, and PowerBI.

End-to-End Azure Data Engineering Project using ADF, Databricks, Synapse, and Power BI.

Azure Databricks Free notes Azure Databricks end to end project with Unity Catalog Azure Databricks Mastery: Hands-on project with Unity Catalog, Delta lake, Medallion Architecture

A complete End to End Data Engineering Project implemented using Azure Cloud provider. Pipeline are created to ingest data from SQL server database , transformed using azure Databricks service and finally data is stored in azure synapase analytics warehouse. Tableau is used to create report over this data.

End-to-end Azure Data Engineering Project with Databricks

This is a End-to-End Azure Data Engineering Project | Analysis on the entire ETL Pipeline - Azure Factory, Azure Lake Gen 2, Databricks, Azure Synapse Analytics & Dashboards

Data Engineering Project on Covid19 Reporting – Using Azure Data Factory, Databricks, HDInsight, Azure Data Factory – An End to End ETL pipeline in addition to a Power BI report dashboard.

Azure Databricks: Real-World Healthcare End-to-End Project

End-to-End data engineering project with Azure Databricks as cloud service and Tokyo olympic data

Azure projects - End to End Data Engineering Project with medallion architecture using Azure Data Factory & Azure Databricks. Azure Serverless/Logical DataWarehouse using Azure Synapse Analystics to demo CETAS, Data Modeling, Incremental loading, CDC and Sql Monitoring the data processing connected to Power BI

End-to-end Azure Data Engineering project using Medallion Architecture, Databricks, Synapse, and Power BI.

Azure End To End Data Engineering Project | Azure Data Factory | Azure Databricks | Azure SQL DB | PySpark | Big Data. It is a in depth Data Engineering project using powerful tools like Azure Data Factory, Azure SQL DB, Azure Databricks, Unity Catalog, Delta Live Tables, Spark Streaming, PySpark, Databricks Asset Bundles, GitHub, and more.

This Azure Databricks project uses Spark Structured Streaming for data ingestion and PySpark for large-scale transformations. It automates Slowly Changing Dimensions (SCDs) with Delta Live Tables and finalizes with dynamic dimensional modeling for curated, analytics-ready datasets.

This Azure Data Engineering projects showcase the full capabilities of the Azure ecosystem, integrating Databricks, Azure DevOps, Unity Catalog, Star Schema design, Delta Live Tables, and the implementation of Slowly Changing Dimensions (SCD Type 1 and Type 2).

This project demonstrates the development of a scalable end-to-end data pipeline for processing and reporting retail order data using Azure Cloud services, Delta Lake architecture, and Databricks.

ETL Pipeline using ADF, Delta Lake, Databricks, Spark, Data Governance (UnityCatalog).

End-to-end Azure Data Engineering project using ADF, Synapse, Databricks, and Power BI.

End-To-End-Data-Engineering-Project (Azure Data Factory, Azure SQL Database, Azure Databricks(Scala,Spark,SQL), Azure Synapse, Power BI, ADLS Gen 2)

End to end Azure Data Engineering Project using Medallion Architecture with ADF, ADLS, Databricks, Synapse, and Power BI

End-to-end healthcare data engineering project using Azure Event Hubs, Databricks, Data Factory, Synapse, and Power BI — implementing real-time streaming, Medallion architecture, and automated ETL pipelines.

This project showcases an end-to-end Azure Data Engineering solution, handling data ingestion, transformation, and delivery using modern cloud-native best practices. The architecture combines Azure SQL, Azure Data Factory, Azure Databricks, and Delta Live Tables (DLT) to deliver a scalable, reliable, and production-ready pipeline.

Hello everyone, I'm excited to share my latest End-to-End Bike Sharing Project developed using Azure Databricks, Azure Data Lake Gen 2, and Delta Lake with all of you. The main purpose of this project is to showcase my skills in data engineering and data analysis.

An end-to-end machine learning pipeline on Microsoft Azure to predict flight delays using historical flight and weather data. The project integrated Azure Blob Storage, Data Factory, and Databricks for data ingestion, processing, and analysis, and employed various ML algorithms for model training and evaluation and visualisation on Power BI

This project is an end-to-end data pipeline designed for processing and analyzing E-Commerce data using Azure services like Databricks, Data Lake, and Delta Lake. It enables seamless data ingestion, transformation, and visualization, offering scalability, performance, and real-time insights through tools like Power BI and Tableau.

I built a complete Data Engineering project leveraging powerful tools like Azure Databricks, Delta Live Tables, Spark Streaming, and PySpark. The project dives into real-world use cases, including Dimensional Data Modeling and managing Slowly Changing Dimensions (SCDs) within Databricks.

No description available

No description available

End to End Azure Databricks pipeline for data engineering with Delta Live Tables