This project demonstrates an **end-to-end data engineering pipeline** built on **Microsoft Azure** using Spotify data. The goal is to ingest raw data from the Spotify API, transform it using Spark, and create analytics-ready datasets following data engineering best practices.

No description available

End-to-end Azure Data Engineering project on Spotify data using Azure Data Factory, Azure Databricks (PySpark, Delta Lake, Autoloader) and Azure SQL, implementing medallion architecture, metadata‑driven pipelines, incremental loads and analytics‑ready dimensional models.

This project showcases an end-to-end data engineering pipeline on the Spotify dataset using Azure Databricks, Delta Lake, and Unity Catalog, enabling incremental ingestion, real-time processing, and analytics to reveal insights on listener behavior, top artists, and regional engagement.