Michael-J-Ward
cd8ed19620
deps: upgrade pyo3 and arrow deps
Arrow needs to be upgraded alongside pyo3 because they both link to python.
NOTE: Arrow marked `ffi::from_ffi` unsafe.
https://github.com/apache/arrow-rs/pull/5080
1 year ago
Philipp Oppermann
2eeb40b1ff
Fix operator API: Box optional error string since `Option<String>` is not FFI-safe
1 year ago
Philipp Oppermann
9a10280595
Simplify reading of primitive and string types from arrow arrays
- introduce a new `ArrayData` wrapper struct
- implement `TryFrom` to convert to primitive types and strings
- update examples to use the new conversion functions
2 years ago
Philipp Oppermann
89f980e753
Simplify arrow array construction through new `IntoArrow` trait
2 years ago
Philipp Oppermann
6154415cbc
Update Rust APIs and rust-dataflow-example to use arrow types
2 years ago
haixuanTao
d55df9a326
Adding documentation for the send method
This PR/commit adds documentation for the send_output method.
Follow up discussion: https://github.com/orgs/dora-rs/discussions/317#discussioncomment-6475816
2 years ago
haixuanTao
f9d8fac8e5
Adding Rust docstring documentation
2 years ago
Philipp Oppermann
7232a31b57
Send `InputClosed` events to operators too
2 years ago
Philipp Oppermann
1213d70efb
Mark operator `Event` type as `non_exhaustive`
2 years ago
Philipp Oppermann
cdc0be1206
Rename `FfiEvent` to `RawEvent`
2 years ago
Philipp Oppermann
62319f1ec1
Add support for Rust operators again
2 years ago
Philipp Oppermann
f473c856c1
Fix clippy warnings across whole projects
3 years ago
Philipp Oppermann
c54209ff98
Pass `SendOutput` by reference
3 years ago
Philipp Oppermann
933affaf16
Rename type
3 years ago
Philipp Oppermann
dcd3f8f18d
Rework raw operator API to support the `#[ffi_export]` attribute of `safer-ffi`
Requires removing the lifetime of the `SendOutput` type. We now use a `'static` Arc-owned closure instead.
To make sure that all types show up in the C header files that we'll generate, we make sure that all the top-level types use `#[repr(C)]` instead of `#[repr(transparent)]`.
3 years ago
Philipp Oppermann
b0cf6fa4f5
Rework raw operator API on top of `safer-api` crate
Allows a higher-level interface
3 years ago
Philipp Oppermann
8e21a990d0
Create a Rust dataflow example
The example shows how to use the dora node and operator APIs.
3 years ago
Philipp Oppermann
825e9a8d7b
Allow operators to stop themselves through a special return value
Useful for source operators that take timers as inputs. Without an explicit stop instructions, they would always run forever.
3 years ago
haixuanTao
28a902d83a
Moving folders in correspondance with #41
Simply moving crate to mirror our discussion without touching at names
3 years ago