Philipp Oppermann
b38c112bc8
Merge branch 'main' into rust-typed-input
2 years ago
Philipp Oppermann
afae3b72ce
Merge branch 'main' into rust-typed-input
2 years ago
haixuanTao
be49c748bd
Add roundtrip test for copying data into buffer and back
2 years ago
Philipp Oppermann
13dfda65dd
Fix version nubmer in Python `__init__`
2 years ago
Philipp Oppermann
6154415cbc
Update Rust APIs and rust-dataflow-example to use arrow types
2 years ago
haixuanTao
87399d749e
Bump version to 0.2.5
2 years ago
Philipp Oppermann
ab52c0c03d
Include context when failing to create arrow array
Co-authored-by: Haixuan Xavier Tao <tao.xavier@outlook.com>
2 years ago
Philipp Oppermann
a882104b93
Fix: Count buffer length instead of owned capacity when sending arrow data
Buffer capacity is always 0 if the buffer is allocated externally. Also, we don't want to allocate unneccessary extra space if only parts of a buffer a used (e.g. when slicing).
2 years ago
Philipp Oppermann
19f9cc7c90
Add support for arbitrary Arrow types in Python API
2 years ago
haixuanTao
e77532fb15
Add pyarrow dependency
2 years ago
Philipp Oppermann
b12f5e7620
Adjust python-ros2-dataflow example and move it to top level
2 years ago
Philipp Oppermann
bae02930df
Import ros2_bridge as experimental submodule of python node API
2 years ago
Philipp Oppermann
274a502b6e
Add method for getting inner event object
2 years ago
Philipp Oppermann
b3e5bf90f5
Add new MergeExternalSend trait for usage in Python
The only difference betwenn the two traits is the additional Send bound. We should unify this again using the trait-level impl trait feature once it's stable.
2 years ago
Philipp Oppermann
1cdb717839
Make `start_runtime` public
2 years ago
Philipp Oppermann
ca7ddf83e4
Fix `__getitem__` call for merged events
2 years ago
Philipp Oppermann
bc7bfd55e8
Add a `kind` item to allow differentiating dora and external events
2 years ago
Philipp Oppermann
bd875f6628
Make `dora-python-operator-api` available as Rust library too
2 years ago
Philipp Oppermann
535c2640c3
Fix match statement in `data` method
2 years ago
Philipp Oppermann
dfa71c77c5
Merge branch 'main' into python-ros2-bridge-example
2 years ago
Philipp Oppermann
34fefccbaa
Create `From` constructor for `ExternalEventStream`
2 years ago
haixuanTao
e944b794a3
Adding `data_type` parsing
2 years ago
haixuanTao
0220d1ef65
Adding `data_type` to metadata for typing output
2 years ago
haixuanTao
6b910ab619
Adding `arrow-schema` as dependency
2 years ago
haixuanTao
19172523df
Update `pyo3` and `arrow` version to use latest version
2 years ago
haixuanTao
04b4da1ec7
Update API documentation
2 years ago
Philipp Oppermann
7a21d965a9
Implement event stream merging for Python
2 years ago
haixuanTao
39c2a51177
Bump to 0.2.4
2 years ago
haixuanTao
8a88cb9400
Bump version to 0.2.4-rc3
2 years ago
Philipp Oppermann
482b282d92
Document `dataflow_descriptor` methods
2 years ago
Philipp Oppermann
07689b9404
Make dataflow descriptor available to Python nodes
2 years ago
haixuanTao
33e12a7e27
Bump version to `0.2.3`
2 years ago
haixuanTao
2c52891438
Place documentation as docstring
2 years ago
haixuanTao
2c130da484
Bump to version 0.2.3-rc6
2 years ago
haixuanTao
c5f23a5441
Bump to version `0.2.3-rc5`
2 years ago
haixuanTao
93b7da2b74
Remove pyo3 in runtime and daemon as it generates `libpython` dependencies
Previous implementation of the runtime would link `libpython` even though it
might not be required.
This commit put pyo3 dependency under a feature flag that is only active if
the runtime is called from python for a python operator.
This fix:
- https://github.com/dora-rs/dora/actions/runs/4879188973/jobs/8705515472
- https://github.com/dora-rs/dora/issues/280#issuecomment-1534058207
- and https://github.com/dora-rs/dora-drives/issues/58#event-9162532660
2 years ago
Philipp Oppermann
1948a45e6d
Copy outputs directly into shared memory in dora runtime
Instead of doing an additional copy to send them from the operator thread to the runtime thread.
This commit uses the new `allocate_data_sample` and `send_output_sample` methods introduced in d7cd370 .
2 years ago
Philipp Oppermann
3eef2073ec
Return optional value for `__getitem__`
2 years ago
Philipp Oppermann
c1544f8257
Allow accessing data multiple times on Python event type
2 years ago
Philipp Oppermann
235f6da79d
Support sending arrow data from operators
This is mostly about convenience when working with arrow data types, as the data is still being copied once.
2 years ago
Philipp Oppermann
cc4acfee91
Verify arrow data type in `send_output`
We only support `UInt8` arrays right now.
2 years ago
Philipp Oppermann
a308c48f3e
Refactor Rust node API to ensure proper stopping
2 years ago
Philipp Oppermann
ef37caa3ed
Add arrow support for Python operator inputs
Reuse same event type as for Python node API.
2 years ago
Philipp Oppermann
79fd5e91af
Rename arrow array access field: `data_arrow` -> `value`
2 years ago
Philipp Oppermann
ccdbada2af
Make `send_output` work with both `PyBytes` and arrow arrays
2 years ago
Philipp Oppermann
e413d38a8a
Validate `arrow::ArrayData` creation
2 years ago
Philipp Oppermann
5b76c163fb
Release Python GIL while waiting for events
2 years ago
Philipp Oppermann
d808ace5c3
Add back support for sending `PyBytes` as output for backwards compatibility
Sending bytes using arrow is possible through new `send_output_arrow` method.
2 years ago
Philipp Oppermann
fc719553ae
Name `eyre::Result` type explicitly
2 years ago
Philipp Oppermann
3496073957
Return custom object from event stream to support both arrow and classical array
Some applications might not need the data in arrow format. By creating a custom object, we can provide the data both as a classical `PyBytes` and as a zero-copy arrow array.
2 years ago