features

Ensures that all raw data has at least an alignment of 128.

Buffer capacity is always 0 if the buffer is allocated externally. Also, we don't want to allocate unneccessary extra space if only parts of a buffer a used (e.g. when slicing).

This PR/commit adds documentation for the send_output method.

Follow up discussion: https://github.com/orgs/dora-rs/discussions/317#discussioncomment-6475816

In #229, I renamed `eyre::eyre!("{err}{traceback}")`  where it should have `eyre::eyre!("{traceback}\n{err}")` . This PR fix this issue.

This Pull Request also add the name of the node process at the end of the process call to help with debugging.

Instead of doing an additional copy to send them from the operator thread to the runtime thread.

This commit uses the new `allocate_data_sample` and `send_output_sample` methods introduced in d7cd370.

This is mostly about convenience when working with arrow data types, as the data is still being copied once.

Reuse same event type as for Python node API.

The previous approach no longer works on `pyo3` v0.18 because `setattr` now requires a `IntoPy<Py<PyString>>` as key.

This commit adds the ability to use the `opentelemetry` feature flag at
runtime by using the env variable `DORA_TELEMETRY`. Using environment
variable reduce the need to do some argument drilling between our
different process as well as making configurable in the YAML at no cost.

Usage:

```bash
docker run -d -p6831:6831/udp -p6832:6832/udp -p16686:16686 jaegertracing/all-in-one:latest
DORA_TELEMETRY=true dora-daemon --run-dataflow dataflow.yml
```

Current limitation:

- The telemetry currently only work with Jaeger on the local network with
default port: 6831 and 6832.

After adding #236 , error of initialization is shadowed by bailing on
the sender channel.

Bailing used to happen here: ccec196234/binaries/runtime/src/lib.rs (L136)

This makes it hard to debug.

This PR will push the error into the sender channel, and report any error
of initialization making it easier to debug.

Now the channel is used without error, but any error of initialization
will bail the runtime here: ccec196234/binaries/runtime/src/lib.rs (L137)

The `tracing` feature just enables the tracing subscriber, the `telemetry` feature the `opentelemetry` integration.

We now report other event types as well. Right now, the only other event type is `Stop`, but we can extend this in the future.

To alievate the unbounded memory growth, we're replacing variable dereferencing
with scoped `GILPool` as described in the pyo3 documentation recently updated. See:
https://github.com/PyO3/pyo3/pull/2864

This commit fix the memory leak happening in the dora API operator. This
is seemingly due to pyo3 leaking memory on object created in Rust.
Using standard `drop` did not drop the memory on the `PyBytes` included in
the `PyDict`.

See: https://github.com/PyO3/pyo3/issues/1801

Fixes https://github.com/dora-rs/dora/issues/163

By making the stopping loop the first loop, we can avoid using `pyo3/allow_threads`, which
seems buggy.

This commit is an initial draft at fixing #147. The error is due to the
fact that pyo3 has linked the libpython from the compilation and not
trying to use libpython that is available in `LD_LIBRARY_PATH`.

The current only solution to disable the linking is to use the `extension-module` flag.
This requires to make the python `runtime-node` packaged in a python library.
The python `runtime-node` should also be fully compatible with the other operators in case we want hybrid runtime.

The issue that i'm facing is that. Due to the packaging, I have to deal with the `GIL` that is present from the start of
`dora-runtime` node. This however makes the process single threaded wich is impossible.

So, I have to disable it, but when I do, I have a race condition:
```bash
Exception ignored in: <module 'threading' from '/usr/lib/python3.8/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 1373, in _shutdown
    assert tlock.locked()
AssertionError:
```
The issue is the same as https://github.com/PyO3/pyo3/issues/1274

To fix this issue, I'm going to look again at the different step to make this work.

But this is my current investigation.

Allow operators to stop the full dataflow