haixuanTao
b5de93e5f5
Pretty Print Rust object when called from Python print
1 year ago
haixuanTao
597ee41428
Improving typing by adding dora prefix
1 year ago
haixuanTao
e790af385e
Using `__init__.py` as without using it
1 year ago
haixuanTao
4adc0889ba
Adding python IDE typing
1 year ago
haixuanTao
629a218ddb
`copy_array_into_sample` do not need to return a result
2 years ago
haixuanTao
e45975f65a
Use cleaner `__from_elem` `avec` constructor
2 years ago
haixuanTao
fe065eec6f
replace const cache line with 128 to match arrow
2 years ago
Philipp Oppermann
905e9ee726
Add more context to serialize_deserialize roundrip errors
2 years ago
Philipp Oppermann
ad3f0482ac
Avoid alignment errors by aligning raw data on deseralization
Ensures that all raw data has at least an alignment of 128.
2 years ago
Philipp Oppermann
c6ce45785f
Rework raw data to arrow array conversion
- If input has no data, return an empty `NullArray` instead of an empty byte array.
- If input contains Vec data (instead of shared memory data), use safe `arrow::buffer::Buffer::from_vec` constructor, which is also zero-copy.
2 years ago
Philipp Oppermann
8cc5837cfa
Update `arrow` and `pyo3` dependencies to latest versions
2 years ago
Philipp Oppermann
631019f8d3
Fix some clippy warnings
2 years ago
Philipp Oppermann
df65eab292
Adjust Python operator API tests
2 years ago
Philipp Oppermann
b38c112bc8
Merge branch 'main' into rust-typed-input
2 years ago
haixuanTao
be49c748bd
Add roundtrip test for copying data into buffer and back
2 years ago
Philipp Oppermann
6154415cbc
Update Rust APIs and rust-dataflow-example to use arrow types
2 years ago
Philipp Oppermann
ab52c0c03d
Include context when failing to create arrow array
Co-authored-by: Haixuan Xavier Tao <tao.xavier@outlook.com>
2 years ago
Philipp Oppermann
a882104b93
Fix: Count buffer length instead of owned capacity when sending arrow data
Buffer capacity is always 0 if the buffer is allocated externally. Also, we don't want to allocate unneccessary extra space if only parts of a buffer a used (e.g. when slicing).
2 years ago
Philipp Oppermann
19f9cc7c90
Add support for arbitrary Arrow types in Python API
2 years ago
Philipp Oppermann
274a502b6e
Add method for getting inner event object
2 years ago
Philipp Oppermann
ca7ddf83e4
Fix `__getitem__` call for merged events
2 years ago
Philipp Oppermann
bc7bfd55e8
Add a `kind` item to allow differentiating dora and external events
2 years ago
Philipp Oppermann
535c2640c3
Fix match statement in `data` method
2 years ago
Philipp Oppermann
dfa71c77c5
Merge branch 'main' into python-ros2-bridge-example
2 years ago
haixuanTao
e944b794a3
Adding `data_type` parsing
2 years ago
haixuanTao
0220d1ef65
Adding `data_type` to metadata for typing output
2 years ago
haixuanTao
6b910ab619
Adding `arrow-schema` as dependency
2 years ago
haixuanTao
19172523df
Update `pyo3` and `arrow` version to use latest version
2 years ago
Philipp Oppermann
7a21d965a9
Implement event stream merging for Python
2 years ago
Philipp Oppermann
1948a45e6d
Copy outputs directly into shared memory in dora runtime
Instead of doing an additional copy to send them from the operator thread to the runtime thread.
This commit uses the new `allocate_data_sample` and `send_output_sample` methods introduced in d7cd370 .
2 years ago
Philipp Oppermann
3eef2073ec
Return optional value for `__getitem__`
2 years ago
Philipp Oppermann
c1544f8257
Allow accessing data multiple times on Python event type
2 years ago
Philipp Oppermann
235f6da79d
Support sending arrow data from operators
This is mostly about convenience when working with arrow data types, as the data is still being copied once.
2 years ago
Philipp Oppermann
ef37caa3ed
Add arrow support for Python operator inputs
Reuse same event type as for Python node API.
2 years ago
Philipp Oppermann
4fb519dbc1
Update `pyo3` to `v0.18`
2 years ago
haixuanTao
7a7a20a298
Add missing licenses and centralise license definition
test new crates version release
2 years ago
haixuanTao
101ddd2092
add documentation and description to all crates
test rc 2 version
fix version typo
Test crates.io new dora tag release
2 years ago
haixuanTao
f1cf0864a4
Remove static path from workspace dependency
Removing those path make it easier to move package, reduce path management
reduce complexity and make it easier to export crates to `crates.io`
2 years ago
Philipp Oppermann
d8260d5d2c
Inherit package version from workspace root
This way, we don't need to update the version manually in all sub-crates on every release.
3 years ago
haixuanTao
9a0b303e98
Replace all versions to `v0.1.1`
3 years ago
Philipp Oppermann
1c1891446a
Add `uhlc` timestamp to message metadata
3 years ago
Philipp Oppermann
808b6679ad
Fix clippy warnings
3 years ago
Xavier Tao
44a7c16c21
Add Python Node API metadata ( #101 )
* Add a `dora-python-operator` crate to hold utils functions for dora python
* Remove python serialisation and deserialisation from `dora-runtime`
* Update `python` documentation
3 years ago