dora/docs/src/getting-started.md

• Create a Rust workspace
• Write your first node
• Write your first operator
• Write your sink node
• Compile everything
• Write a graph definition
• Run it!

Create a Rust workspace

• Initiate the workspace with:

mkdir my_first_dataflow
cd my_first_dataflow

• Create the Cargo.toml file that will configure the entire workspace:

Cargo.toml

[workspace]

members = [
    "rust-dataflow-example-node",
]

Write your first node

Let's write a node which sends the current time periodically. Let's make it after 100 iterations. The other nodes/operators will then exit as well because all sources closed.

• Generate a new Rust binary (application):

cargo new rust-dataflow-example-node

with Cargo.toml:

{{#include ../../examples/rust-dataflow/node/Cargo.toml}}

with src/main.rs:

{{#include ../../examples/rust-dataflow/node/src/main.rs}}

Write your first operator

• Generate a new Rust library:

cargo new rust-dataflow-example-operator --lib

with Cargo.toml:

{{#include ../../examples/rust-dataflow/operator/Cargo.toml}}

with src/lib.rs:

{{#include ../../examples/rust-dataflow/operator/src/lib.rs}}

• And modify the root Cargo.toml:

[workspace]

members = [
    "rust-dataflow-example-node",
    "rust-dataflow-example-operator",
]

Write your sink node

Let's write a logger which will print incoming data.

• Generate a new Rust binary (application):

cargo new sink_logger

with Cargo.toml:

{{#include ../../examples/rust-dataflow/sink/Cargo.toml}}

with src/main.rs:

{{#include ../../examples/rust-dataflow/sink/src/main.rs}}

• And modify the root Cargo.toml:

[workspace]

members = [
    "rust-dataflow-example-node",
    "rust-dataflow-example-operator",
    "rust-dataflow-example-sink"
]

Compile everything

cargo build --all --release

Write a graph definition

Let's write the graph definition so that the nodes know who to communicate with.

dataflow.yml

{{#include ../../examples/rust-dataflow/dataflow.yml}}

Run it!

• Run the dataflow:

dora-coordinator --run-dataflow dataflow.yml dora-runtime