haixuanTao
c4263e6c8f
Remove bound from definition of .to_py_dict
1 year ago
haixuanTao
ae9aea24e0
update stubs definition
1 year ago
haixuanTao
b2eace277d
Transform custom PyEvent into standard python dictionary for easier debuggability.
Currently having a custom PyEvent make debugging very hard as fields are hidden within the class PyEvent that is defined within Rust Code.
Python user are getting really confused about this obscure class.
This PR transforms the class into a standard python dictionary.
1 year ago
Haixuan Xavier Tao
f34cbe2ffc
Merge pull request #517 from dora-rs/detached-python-process
Run dynamic node
1 year ago
haixuanTao
7d3a5de1c5
Make rust flexible logic and make sure to always use env variables DORA_NODE_CONFIG if it exists.
This remove the complexity of having to rename the node id when used within a yaml configuration.
1 year ago
haixuanTao
3ad402ce45
Add dynamic node event loop and dynamic node connection
1 year ago
Michael-J-Ward
bf088fc1eb
lint: fix clippy lint
1 year ago
Michael-J-Ward
25884b658a
update pyo3 to new bounds api
1 year ago
haixuantao
0f530906aa
Add dataflow_id getter in python
1 year ago
haixuanTao
597ee41428
Improving typing by adding dora prefix
1 year ago
haixuanTao
e790af385e
Using `__init__.py` as without using it
1 year ago
haixuanTao
4adc0889ba
Adding python IDE typing
1 year ago
haixuanTao
0cd5281325
Use duration instead of f32 secs
2 years ago
haixuanTao
b1b69693b7
Adding a timeout method to not block indefinetely
2 years ago
Philipp Oppermann
6154415cbc
Update Rust APIs and rust-dataflow-example to use arrow types
2 years ago
Philipp Oppermann
a882104b93
Fix: Count buffer length instead of owned capacity when sending arrow data
Buffer capacity is always 0 if the buffer is allocated externally. Also, we don't want to allocate unneccessary extra space if only parts of a buffer a used (e.g. when slicing).
2 years ago
Philipp Oppermann
19f9cc7c90
Add support for arbitrary Arrow types in Python API
2 years ago
Philipp Oppermann
b12f5e7620
Adjust python-ros2-dataflow example and move it to top level
2 years ago
Philipp Oppermann
bae02930df
Import ros2_bridge as experimental submodule of python node API
2 years ago
Philipp Oppermann
b3e5bf90f5
Add new MergeExternalSend trait for usage in Python
The only difference betwenn the two traits is the additional Send bound. We should unify this again using the trait-level impl trait feature once it's stable.
2 years ago
Philipp Oppermann
1cdb717839
Make `start_runtime` public
2 years ago
Philipp Oppermann
dfa71c77c5
Merge branch 'main' into python-ros2-bridge-example
2 years ago
Philipp Oppermann
34fefccbaa
Create `From` constructor for `ExternalEventStream`
2 years ago
haixuanTao
0220d1ef65
Adding `data_type` to metadata for typing output
2 years ago
haixuanTao
04b4da1ec7
Update API documentation
2 years ago
Philipp Oppermann
7a21d965a9
Implement event stream merging for Python
2 years ago
Philipp Oppermann
482b282d92
Document `dataflow_descriptor` methods
2 years ago
Philipp Oppermann
07689b9404
Make dataflow descriptor available to Python nodes
2 years ago
haixuanTao
2c52891438
Place documentation as docstring
2 years ago
Philipp Oppermann
c1544f8257
Allow accessing data multiple times on Python event type
2 years ago
Philipp Oppermann
235f6da79d
Support sending arrow data from operators
This is mostly about convenience when working with arrow data types, as the data is still being copied once.
2 years ago
Philipp Oppermann
cc4acfee91
Verify arrow data type in `send_output`
We only support `UInt8` arrays right now.
2 years ago
Philipp Oppermann
a308c48f3e
Refactor Rust node API to ensure proper stopping
2 years ago
Philipp Oppermann
ef37caa3ed
Add arrow support for Python operator inputs
Reuse same event type as for Python node API.
2 years ago
Philipp Oppermann
79fd5e91af
Rename arrow array access field: `data_arrow` -> `value`
2 years ago
Philipp Oppermann
ccdbada2af
Make `send_output` work with both `PyBytes` and arrow arrays
2 years ago
Philipp Oppermann
e413d38a8a
Validate `arrow::ArrayData` creation
2 years ago
Philipp Oppermann
5b76c163fb
Release Python GIL while waiting for events
2 years ago
Philipp Oppermann
d808ace5c3
Add back support for sending `PyBytes` as output for backwards compatibility
Sending bytes using arrow is possible through new `send_output_arrow` method.
2 years ago
Philipp Oppermann
fc719553ae
Name `eyre::Result` type explicitly
2 years ago
Philipp Oppermann
3496073957
Return custom object from event stream to support both arrow and classical array
Some applications might not need the data in arrow format. By creating a custom object, we can provide the data both as a classical `PyBytes` and as a zero-copy arrow array.
2 years ago
Philipp Oppermann
1fab082717
Use `arrow` crate to send events from Rust to Python without copying
See #224 for details.
2 years ago
Philipp Oppermann
2ca97fdc6b
Remove lifetime from `Event` type
Don't bind the lifetime of the event to the `next` call anymore. This makes it possible to use the original event in Python, which does not support borrowed data or lifetimes. The drawback is that we no longer have the guarantee that an event is freed before the next call to `recv`.
2 years ago
haixuanTao
b766a44a9f
use arrow for rust-python interface
2 years ago
haixuanTao
8ad1a74b7a
Better debugging messages
2 years ago
haixuanTao
2a9c9d4fe3
Remove unnecessary `unwrap` that cause a panic
2 years ago
Philipp Oppermann
d334c6f243
Make `daemon` module of Rust node API private and reexport symbols
2 years ago
Philipp Oppermann
8cbc891251
Add start runtime function to python node api again
3 years ago
Philipp Oppermann
352fb3d757
Change Python event types to uppercase
3 years ago
Philipp Oppermann
27daceafd1
Fix typo
Co-authored-by: Haixuan Xavier Tao <tao.xavier@outlook.com>
3 years ago