Found 121 repositories(showing 30)
airscholar
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
alexanderbean
End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)
prabh8331
No description available
tranhuy25
This project serves as a comprehensive guide to building an end-to-end data engineering pipeline. It covers each stage from data ingestion to processing and finally to storage, utilizing a robust tech stack that includes Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra.
SidiahmedHABIB
This project is an end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using a variety of powerful tools including Apache Airflow, Apache Kafka, Apache Spark and Cassandra. All components are containerized with Docker for easy deployment and scalability.
dogucanelci
No description available
syedhassaanahmed
E2E Spark data pipelines with engineering fundamentals
fatimabintetanveer
End-to-end Azure pipeline for analyzing Tokyo Olympics data, using Azure Data Factory, Data Lake Storage Gen2, Databricks, and Synapse Analytics to provide insights into athlete performance and medal statistics.
himasha0421
real world e2e data engineering pipeline with airflow,kafka,zookeeper,spark ect.
bassamoh32
e2e etl using snowflake & dbt & airflow
sudarshanp1
Crafted an ETL pipeline project with Spotify API on AWS. Extracting data from Spotify, transforming it to the desired format & loading it into an AWS data store, this streamlined process ensures efficient handling, merging Spotify API prowess with AWS scalability.
manojar15
No description available
manojar15
No description available
aadityasharma0912
E2E Data Engineering Project for HCInsurance Claims Adjudication
No description available
This is an Azure Databricks E2E Data Engineering Project.
sreekanthpogula
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
hari255
An ETL pipeline that extarcts data from an API, perfroms transformation, loads into BigQuery, finally enabling users to dig into data, analyze and create visualizations without SQL
gboluwaga
No description available
erickobrinsky
No description available
akashsingla008
Spotify E2E Data Engineering project
AshaRavilla
No description available
0xpradish
No description available
jversolato
No description available
Romit27-eng
No description available
adarshumesh5
Personal project for exploring modern development practices
kanweitech
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
Sumaila-dev
This project demonstrates the development of a complete end-to-end data engineering pipeline. It includes every phase, from data ingestion to processing and storage.
T-Anh-k4
No description available
sripriyareddy20
Modern web application demonstrating clean code principles