A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering solution for processing, storing, and reporting daily transaction data in the online food delivery industry.

End to End Data engineering projects in Google cloud environment

A data engineering project with Airflow, dbt, Terrafrom, GCP and much more!

Data Engineering Project in GCP

Ames Housing Price Predictor is an ML project focused on predicting housing prices in Ames, Iowa. Development spans all stages of the machine learning model lifecycle, from data exploration and cleansing, through feature engineering and model selection, to deploying a working API using FastAPI, Docker, and GCP.

A data engineering project with dbt, Docker, Kestra, Terraform, GCP and Looker.

A complete end to end data engineering project on GCP

Developed a modern data engineering project on the Uber dataset using Google Cloud Platform (GCP). Built a data model in fact and dimension format, transformed the data using Python, deployed the code on a compute instance, loaded the data onto BigQuery, and created a final dashboard for data analysis and visualization.

Data Engineering project - ETL pipeline using Airflow as long as working with dbt, Soda, GCP and Metabase to build a Data Pipeline.

A collection of my hands-on data engineering projects and learning practice, including ETL pipelines, FastAPI, Streamlit, SQL, Python, Airflow, GCP, and automation experiments.

Open-source repository covering ML algorithms, Generative/Agentic AI, Big Data engineering, and cloud-based MLOps (Azure/AWS/GCP). Includes fundamentals and real-world end-to-end projects.

This repository contain my data engineering projects in GCP

This is a near realtime end-to-end data engineering project (GCP) using Aviation data using Aviation Stack API

Data Engineering pet-project covering GCP, Docker, workflow orchestration with Mage, data transforming with dbt, batch processing via Spark and data streaming using Kafka

The goal of this project is to perform comprehensive data analytics on Uber trip data using a modern data engineering stack on Google Cloud Platform (GCP).

Data Engineering an Analysis project using Apache Cassandra as database and Apache Spark for processing with infrastructure deployed in Google Cloud Platform (GCP).

Data engineering project using GCP Dataproc, Apache Hadoop with Hive.

This repo contains Data Engineering Zoomcamp Final Project. The project purpose is to perform Extract, Load and Transform (ELT) approach on GCP to batch process Capital Bike Share Dataset from January 2021 to January 2023.

This is a small data engineering project using python, mysql, mongodb, airflow, docker and gcp services.

"Showcasing expertise in data engineering, AI, and cloud orchestration. Includes projects in Airflow, GCP, Azure, Python, and LLM optimization."

An end to end data engineering project made with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

A Data Engineering Project built using End to End Data Engineering techniques. From analyzing Taxi Ride's data using various tools and technologies, including GCP Storage, Python, Compute Instance, Mage Data Pipeline Tool, BigQuery, and Looker Studio.

Analytics engineering platform for the Olist e-commerce dataset, built with dbt, BigQuery and GCP. Phase 2 of a data warehouse project.

Embark on the Uber Data Engineering Project, advancing from Lucidchart modeling to Jupyter code execution on a GCP instance. Master Python, Pandas, Mage AI, and Google Cloud libraries, leading to BigQuery data storage. Culminate with Looker dashboard creation for end-to-end data engineering, delivering actionable insights.

Data Engineering project leveraging Airflow for weekly ETL of 4.6M+ rows of Chicago crash data, modeled in BigQuery with dbt. Includes an interactive Tableau dashboard for crash trends and geographic distribution, with GCP infrastructure automated via Terraform.

A data engineering project for a real-time ad insertion system leveraging Kafka, Spark Streaming, dbt, machine learning, and Looker, built within a robust data pipeline orchestrated with Airflow and Docker, managed by Terraform on GCP, designed to dynamically enhance viewer engagement and advertising efficacy on OTT platforms.

This repository gives the description of google cloud data engineering projects and its key components

No description available

A data engineering Project using GCP