No description available

Data Pipes for CSV

A cloud-native data pipeline and visualization project analyzing Formula 1 racing data using Azure, Databricks, Delta Lake, Tableau, and Python for insightful EDA and interactive dashboards.

Iterable datapipelines for pytorch training.

The 🌏 data science library you've been waiting for~

Real time stock data pipeline --play with Kafka, Cassandra, Spark, Redis, Node.js, Zookeeper

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Network TCP port forwarding

Send data from your behavioral experiments to the OSF. Born-open data as a service.

Various demos of data pipelines

No description available

Domain-specific language to help build and maintain AWS Data Pipelines

Building Json data pipeline within Snowflake using Streams and Tasks

Example repo to create end to end tests for data pipeline.

dataPipe is a data processing and data analytics library for JavaScript. Inspired by LINQ (C#) and Pandas (Python)

Spark-based pipeline to extract and parse monthly games from the Lichess database.

The most convenient piping syntax for generic data manipulation in Julia.

DataPipeline Examples

Code for the DataPipes article

OCF's DataPipe based dataloader for training and inference

No description available

Tool for sync data from Mysql to Mysql with Rename table and filter columns

The ecosystem of geospatial machine learning tools in the Pangeo world.

Data pipeline project using Data Factory, Databricks and Cosmosdb Graph, deployed using Azure DevOps, secured using firewalls and Azure AD

A data processing framework

This project consist of Datapipeline that collects data from Google Admob, Facebook Ads, Google Analytics and Google Ads into a single csv file.

DataPipeline 是一款批流一体数据融合平台。

data pipeline code of large video generation model

Simple guides to create your dream LLM dataset

No description available