数据流引擎是一款面向数据集成、数据同步、数据交换、数据共享、任务配置、任务调度的底层数据驱动引擎。数据流引擎采用管执分离、多流层、插件库等体系应对大规模数据任务、数据高频上报、数据高频采集、异构数据兼容的实际数据问题。

A reactive query engine built on differential dataflow.

Pydra Dataflow Engine

A lightweight (serverless) native python parallel processing framework based on simple decorators and call graphs.

A Lightweight, Cloud-Native Stateful Distributed Dataflow Engine

A minimal Dataflow programming engine

a decentralized search engine with a decentralized verifiable dataflow

A reactive dataflow engine, a data stream processing framework using Vert.x

Cloud-native Telco BSS hosted in GCP K8s with standalone Diameter to gRPC gateway. Rule Engine using Neo4j graphs. Analytics Events sent to GCP BigData (Dataflow+BigQuery) via PubSub. It's awesome!

Open source dataflow engine in Django

This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering solution for processing, storing, and reporting daily transaction data in the online food delivery industry.

The RStein.AsyncCpp library is a set of types that should be familiar for anyone who knows the Task Parallel Library (TPL) for .NET (C#).

Running Segment Anything Geospatial over Earth Engine exports with Dataflow at scale.

Bitflow's dataflow engine, a lightweight framework for real-time processing of data streams through a graph of operators

MaxJ and C/C++ library and utilities for Maxeler Dataflow Engines

ecommerce GCP Streaming pipeline ― Cloud Storage, Compute Engine, Pub/Sub, Dataflow, Apache Beam, BigQuery and Tableau; GCP Batch pipeline ― Cloud Storage, Dataproc, PySpark, Cloud Spanner and Tableau

A dataflow engine implementation in C# using mono

An experimental Differential Dataflow optimization engine powered by equality saturation

Ruby Dataflow Engine

liquid dataflow engine for Rebol

The dataflow engine for Rhizome

Dataflow library and engine in C++

JavaScript engine for dataflow programming

RTL design and verification for LLM hardware accelerators — systolic arrays, attention engines, dataflow architectures, and FPGA/ASIC implementation

Pure C++20 streaming-first dataflow engine research: single-machine micro-batch runtime, query-local backpressure, stateful windows, and local actor-based scale-up.

Explore and create ML datasets. Sample the dataset and create training, validation, and testing datasets for local development of TensorFlow models. Create a benchmark to evaluate the performance of ML. TensorFlow is used for numerical computations, using directed graphs. Getting started with TensorFlow. Explore the TensorFlow python API, build a graph, run a graph, feed values into a graph. Find areas of a triangle using TensorFlow. Learning from tf.estimator. Read from python’s pandas dataframe into tf.constant, create feature columns for estimator, perform linear regression with tf.Estimator framework. Execute Deep Neural Network regression. Use benchmark dataset. Refactoring to add batching and feature creation. Refactor the input. Refactor the way the features are created. Create and train the model, Evaluate the model. Distributed training and monitoring. Create features out of input data. Train and evaluate. Monitor with Tensorboard. To run TensorFlow at scale, use Cloud ML Engine. Package up the code. Find absolute paths to data. Run the python module from the command line. Run locally using GCloud. Submit training job using GCloud. Deploy model. Make predictions. Train on a 1-million row dataset. Feature Engineering. Working with feature columns. Adding feature crosses in TensorFlow. Reading data from BigQuery. Creating datasets using Dataflow. Using a wide-and-deep model.

The Engine of Robotic DataFlow Automation for IoT

An execution engine for building research dataflow/spreadsheet implementations

End-to-end Azure Data Factory ETL pipeline for COVID-19 reporting. Includes ingestion (HTTP & ADLS), transformations (Filter, Select, Pivot, Lookup, Join), and loading to ADLS. ARM templates included for full deployment.

Tesey DataFlow lets you process dataflows as in batch and as well in streaming modes in any Apache Beam's supported execution engines including Apache Spark, Apache Flink, Apache Samza, etc.