Chrislearn Young
bcf591deb4
Remove rust--toolchain.toml and use rust-version in Cargo.toml
6 months ago
Philipp Oppermann
9d917bdaa9
Parse dataflow descriptor lazily in node APIs
The dataflow descriptor format still changes often, which led to parse errors. By doing the parsing lazily, this should only affect users of the `dataflow_descriptor` function from now on.
7 months ago
haixuanTao
a9aa556b9c
Adding metrics for node api
10 months ago
yjhmelody
ca614753a1
chore: use workspace edition
1 year ago
Philipp Oppermann
8e8034d103
Set `repository` metadata for all our crates
1 year ago
Philipp Oppermann
6a3dcecf33
Refactor: Move message definitions to `dora-message` crate
First step towards versioning the message definitions indepedently.
1 year ago
haixuanTao
3ad402ce45
Add dynamic node event loop and dynamic node connection
1 year ago
haixuanTao
0568656ef0
Remove unused dependencies for faster compile time
2 years ago
haixuanTao
b1b69693b7
Adding a timeout method to not block indefinetely
2 years ago
Philipp Oppermann
ad3f0482ac
Avoid alignment errors by aligning raw data on deseralization
Ensures that all raw data has at least an alignment of 128.
2 years ago
Philipp Oppermann
8cc5837cfa
Update `arrow` and `pyo3` dependencies to latest versions
2 years ago
Philipp Oppermann
89f980e753
Simplify arrow array construction through new `IntoArrow` trait
2 years ago
haixuanTao
6b910ab619
Adding `arrow-schema` as dependency
2 years ago
haixuanTao
19172523df
Update `pyo3` and `arrow` version to use latest version
2 years ago
Philipp Oppermann
e2899c382e
Merge pull request #311 from dora-rs/external-events
Add convenience functions for merging external event streams with `EventStream`
2 years ago
Haixuan Xavier Tao
53198b861b
Merge pull request #248 from dora-rs/shmem-read-only
Map shared memory region as read-only in receiver
2 years ago
Philipp Oppermann
c4bd9d3edf
Add convenience functions for merging external event streams with `EventStream`
Our upcoming ros2<->dora bridge will allow nodes to subscribe to ROS2 topics, which results in a second stream of events. By providing a merge function, this ROS2 event stream can be merged together with the dora event stream.
2 years ago
Philipp Oppermann
326c650f1f
Implement `Stream` for `EventStream`
2 years ago
Philipp Oppermann
979573689a
Switch to forked `shared_memory_extended` crate
The maintainer of the `shared_memory` crate did not react to our PR, so I decided to create a (temporary) fork.
2 years ago
Philipp Oppermann
1fab082717
Use `arrow` crate to send events from Rust to Python without copying
See #224 for details.
2 years ago
haixuanTao
c4794725aa
Fix API description to use workspace
test new release for github CI/CD
2 years ago
haixuanTao
38e02df62c
Update rust api to publish it to crates.io
2 years ago
haixuanTao
f1cf0864a4
Remove static path from workspace dependency
Removing those path make it easier to move package, reduce path management
reduce complexity and make it easier to export crates to `crates.io`
2 years ago
haixuanTao
70e6d4ce8f
Remove code duplicate for tracing subscriber and use env variable to manage log level.
The intent of this commit is to remove the quantity of log that is being pushed to user.
This commit removes the redeclaration of a set up tracing methods to centralise
the tokio-tracing subscriber within the extension crate. It also add the
feature to filter information based on Environment variable.
This makes it possible to define the log level for tokio tracing like
this:
```
RUST_LOG=debug dora-daemon --run-dataflow dataflow.yml
```
I have also unified the feature flag to make it easier to manage tracing features among the workspace.
I did not change the default behaviour of tracing in our crates and therefore by
using the command above you should get the same tracing log as before.
fix merge conflict generated
2 years ago
Philipp Oppermann
feca83e309
Refactor: Split Rust node API into smaller submodules
2 years ago
Philipp Oppermann
1e07a73a64
Merge branch 'main' into unify-nodes-and-operators
2 years ago
Philipp Oppermann
1553a198ef
Re-export dora-message from dora-core
2 years ago
Philipp Oppermann
0840cf0c0e
Implement support for TCP-based daemon connection in Rust node API
2 years ago
dependabot[bot]
b74a524031
Bump tokio from 1.23.1 to 1.24.2
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.23.1 to 1.24.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/commits )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
Philipp Oppermann
cf95078ca1
Extract shared memory server/client into separate crate
To allow for individual testing and benchmarking. This will also enable reuse of the library in other projects.
3 years ago
dependabot[bot]
ae161e644c
Bump tokio from 1.21.2 to 1.23.1
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.2 to 1.23.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.23.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
Philipp Oppermann
70298d836f
Start migrating the daemon control channel to shared memory
The goal is to avoid the latency of the TCP socket.
3 years ago
Philipp Oppermann
df5dcfb342
Merge branch 'main' into unify-nodes-and-operators
3 years ago
Philipp Oppermann
d3914cca60
Merge pull request #157 from dora-rs/release
Simplify release process
3 years ago
dependabot[bot]
2ba6b1c41c
Bump capnp from 0.14.9 to 0.14.11 ( #158 )
Bumps [capnp](https://github.com/capnproto/capnproto-rust ) from 0.14.9 to 0.14.11.
- [Release notes](https://github.com/capnproto/capnproto-rust/releases )
- [Commits](https://github.com/capnproto/capnproto-rust/compare/capnp-v0.14.9...capnp-v0.14.11 )
---
updated-dependencies:
- dependency-name: capnp
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
Philipp Oppermann
433b7b4ed7
Merge branch 'main' into unify-nodes-and-operators
3 years ago
Philipp Oppermann
d8260d5d2c
Inherit package version from workspace root
This way, we don't need to update the version manually in all sub-crates on every release.
3 years ago
Philipp Oppermann
2e7ef8b630
Implement shared memory mapping in node API
3 years ago
haixuanTao
9a0b303e98
Replace all versions to `v0.1.1`
3 years ago
Philipp Oppermann
201fd228f5
Remove communication layer from dora-node-api
3 years ago
Philipp Oppermann
7d85b92b2a
Connect to dora-daemon from node API and register with node ID
3 years ago
Philipp Oppermann
fc50a8232f
Create a request/reply communication layer abstraction
And a TCP-based implemenation.
3 years ago
Philipp Oppermann
c4da2cbdad
Refactor: Move configuration to `core` crate
3 years ago
Philipp Oppermann
7e9c3050f9
Use `tracing` for logging state of nodes, coordinator, and runtimes
3 years ago
Philipp Oppermann
e0cc48a557
Move `dora-message` crate directly under `libraries` (it's not really an extension)
3 years ago
Philipp Oppermann
0df06fce86
Send metadata in messages encoded with capnproto
Changes the message format from raw data bytes to a higher-level `Message` struct serialized with capnproto. In addition to the raw data, which is sent as a byte array as before, the `Message` struct features `metadata` field. This metadata field can be used to pass open telemetry contexts, deadlines, etc.
3 years ago
Philipp Oppermann
848e136a75
Move communication layer implementation into separate library
To make it usable for other projects.
3 years ago
Philipp Oppermann
6caef3b149
Disable iceoryx support on Windows
It causes lots of linker errors on Windows. The `iceoryx-rs` crate does not seem to support Windows at the moment.
3 years ago
Philipp Oppermann
03348c5be8
Redesign communication layer to be synchronous and add support for iceoryx
3 years ago
Philipp Oppermann
474c1e2465
Remove unused dependency
3 years ago