Instead of doing an additional copy to send them from the operator thread to the runtime thread.
This commit uses the new `allocate_data_sample` and `send_output_sample` methods introduced in d7cd370.
This commit adds the ability to use the `opentelemetry` feature flag at
runtime by using the env variable `DORA_TELEMETRY`. Using environment
variable reduce the need to do some argument drilling between our
different process as well as making configurable in the YAML at no cost.
Usage:
```bash
docker run -d -p6831:6831/udp -p6832:6832/udp -p16686:16686 jaegertracing/all-in-one:latest
DORA_TELEMETRY=true dora-daemon --run-dataflow dataflow.yml
```
Current limitation:
- The telemetry currently only work with Jaeger on the local network with
default port: 6831 and 6832.
After adding #236 , error of initialization is shadowed by bailing on
the sender channel.
Bailing used to happen here: ccec196234/binaries/runtime/src/lib.rs (L136)
This makes it hard to debug.
This PR will push the error into the sender channel, and report any error
of initialization making it easier to debug.
Now the channel is used without error, but any error of initialization
will bail the runtime here: ccec196234/binaries/runtime/src/lib.rs (L137)
The intent of this commit is to remove the quantity of log that is being pushed to user.
This commit removes the redeclaration of a set up tracing methods to centralise
the tokio-tracing subscriber within the extension crate. It also add the
feature to filter information based on Environment variable.
This makes it possible to define the log level for tokio tracing like
this:
```
RUST_LOG=debug dora-daemon --run-dataflow dataflow.yml
```
I have also unified the feature flag to make it easier to manage tracing features among the workspace.
I did not change the default behaviour of tracing in our crates and therefore by
using the command above you should get the same tracing log as before.
fix merge conflict generated
Makes it possible to set an URL as operator source. The `dora-runtime` will then try to download the operator from the given URL when the dataflow is started.
* add opentelemetry feature to python op
* modify `python-dataflow` to test opentelemetry
* change `timer` topic to `tick`
* adding tracing to `shared_libraries`
* example install refactoring
* Rename `input` to `dora_input` for python ex