Philipp Oppermann
a6781fd4a9
Canoncicalize node path before launching it
3 years ago
Philipp Oppermann
bad8d30a5f
Split coordinator binary into lib.rs and main.rs
Allows using the `run` function from tests and examples.
3 years ago
Philipp Oppermann
5004aa9d6e
Specify a license for the node-api-c crate
3 years ago
Philipp Oppermann
b849d31496
Rename operator API header file to operator_api.h
3 years ago
Philipp Oppermann
3287761393
Mark the C node API functions as no_mangle
3 years ago
Philipp Oppermann
d1a256a47a
Mark most functions as unsafe since they deref the given pointers
3 years ago
Philipp Oppermann
b2577acf14
Implement API functions for reading inputs and sending outputs
3 years ago
Philipp Oppermann
370cadee6b
Start creating a staticlib with a node API for C
3 years ago
Philipp Oppermann
474c1e2465
Remove unused dependency
3 years ago
Philipp Oppermann
e7eeafa067
Merge pull request #51 from dora-rs/timer-input
Implement runtime-provided periodic timer input for source operators
3 years ago
Philipp Oppermann
047252be6b
Avoid intermediate allocations by implementing Display instead of ToString
3 years ago
Philipp Oppermann
7158ff2d38
Send `InputsStopped` event only once, after all inputs are closed
Ensures that the event is only sent after all input straems are exhausted. This avoids a potential race when there are multiples inputs.
3 years ago
Philipp Oppermann
e677adbd58
Keep operators that use dora timers as inputs running indefinitely
The dora timers produce values forever. We only want to stop operators when all their input streams are closed.
3 years ago
Philipp Oppermann
ed59e69eca
Fix coordinator run instructions in README
3 years ago
Philipp Oppermann
9b711d3964
Fix paths in `mini-dataflow.yml` example
3 years ago
Philipp Oppermann
20501ca461
Send dora timer messages from coordinator
3 years ago
Philipp Oppermann
37c7970e49
Add dora input nodes in visualization
3 years ago
Philipp Oppermann
acd0f6193f
Adjust libraries and binaries for new `InputMapping` format
3 years ago
Philipp Oppermann
3c8ee37795
Implement parsing of new dora timer input keys
3 years ago
Philipp Oppermann
b6a8d5d994
Merge pull request #54 from dora-rs/dora-node-python
Python Node API
3 years ago
haixuanTao
e661324253
change test graph input name `time` to` string`
3 years ago
haixuanTao
4158b1b0de
use `pyproject.toml` for activating `pyo3/extension-module`
Since the merge of https://github.com/PyO3/maturin/pull/605 ,
we can add features flag when maturin build the python project, removing
the need to always enabling `extension-module` flag when building.
The `extension-module` flag creates crash when building and testing outside
of maturin.
Previous fix required to disable default feature flag when building and testing which is not ideal.
3 years ago
Philipp Oppermann
ae9043c961
Fix logo path
3 years ago
haixuanTao
4ff9a59ad7
Use `PyBytes` as it is faster than Vec<u8>
3 years ago
haixuanTao
f2b3bd5a21
adding `abi3-py38` to pyo3 for best target support
3 years ago
haixuanTao
44713538e7
adding `--no-default-features` for pyo3 extension
3 years ago
haixuanTao
9f53dc6904
Adding feature flag to pyo3
3 years ago
haixuanTao
366ec10b1a
Adding `extension-module` for pyo3 extension
3 years ago
haixuanTao
7813615b2c
Remove legacy python binding from `Cargo.toml`
3 years ago
haixuanTao
6b2b082e71
Renaming `PyDoraNode` to `Node`
3 years ago
haixuanTao
79da342b62
Adding test graph for python node
3 years ago
haixuanTao
d69776a85f
Adding `eyre` Error handling derived into `PyErr`
3 years ago
haixuanTao
9000e68111
Adding `__iter__` function
3 years ago
haixuanTao
c6b671b4ee
moving `node` examples into a specific folder
3 years ago
haixuanTao
1903c636cb
Refactoring `python-node-api`
3 years ago
haixuanTao
128bf6ca43
Adding `next`, `send_output` for Python node API
Adding `next` and `send_output` requires an async threadpool as the
communication Layer defined by the Middleware Layer returns an async Future
stream.
I solve this issue by adding a tokio runtime on a separater thread that is connected with two channels.
One for sending data and one for receiving data.
Those channel are then exposed synchronously to Python. This should not be cause for
concern as channel are really fast.
Looking at Zenoh Python client, they are heavily using `pyo3-asyncio` implementation
of futures to pass Rust futures into Python.
This can be a solution as well, but, from previous experiment, I'm concerned about performance on such
solution. I have experienced that putting futures from Rust into the `asyncio` queue to be slow.
I'm concerned also by mixing `async` and `sync` code in Python, as it might be blocking. This might requires 2 threadpool in Python.
This might seem as heavy overhead for some operations.
3 years ago
haixuanTao
515ae91151
making `CommunicationLayer` Send and Sync
3 years ago
haixuanTao
edbd10e410
add `Send` trait to the communicationLayer
The `Send` Trait is required by pyo3 for the python binding
3 years ago
haixuanTao
f7d288ba69
init `dora-node-api-python`
3 years ago
Philipp Oppermann
e98fc0725d
Merge pull request #50 from dora-rs/python-operator-alias
Add support for single operator nodes in YAML dataflow specification
3 years ago
Philipp Oppermann
2750ea59b1
Rename config fields to 'definition'
3 years ago
Xavier Tao
0db7c6a289
Merge pull request #46 from dora-rs/folder_structure
Moving folders in correspondance with #41
3 years ago
Philipp Oppermann
45378870ad
Generalize support for single-operator nodes, independent of their type
3 years ago
Philipp Oppermann
bc56b5947d
Implement alias syntax for runtime nodes with single python operator
3 years ago
haixuanTao
5d35b6c77b
Changing folder name `serialisation` into `message`
3 years ago
haixuanTao
251fe6b20f
Replaced `dora-common` with `dora-core`
3 years ago
haixuanTao
28a902d83a
Moving folders in correspondance with #41
Simply moving crate to mirror our discussion without touching at names
3 years ago
haixuanTao
a2f737bf25
Use `cargo run` in dataflow
Use `cargo run` instead of the path of the binary in the
dataflow as recommended way of using `run:` as it is:
-OS agnostic
-Compile node before being run which avoids the risk of missing development changes
-More readable
3 years ago
haixuanTao
6611578d1a
Injecting `env` variable into node runtimes
Injecting the env variable diclared in the yaml into the node runtime.
To do that we need to re-serialize the parsed env variables into
string.
3 years ago
Philipp Oppermann
cd4089b895
Merge pull request #42 from dora-rs/fix/tracing
Expose name of tracing and fix install batch issue
3 years ago