The intent of this commit is to remove the quantity of log that is being pushed to user.
This commit removes the redeclaration of a set up tracing methods to centralise
the tokio-tracing subscriber within the extension crate. It also add the
feature to filter information based on Environment variable.
This makes it possible to define the log level for tokio tracing like
this:
```
RUST_LOG=debug dora-daemon --run-dataflow dataflow.yml
```
I have also unified the feature flag to make it easier to manage tracing features among the workspace.
I did not change the default behaviour of tracing in our crates and therefore by
using the command above you should get the same tracing log as before.
fix merge conflict generated
We currently need an additional request to prepare zero-copy shared memory messages. For small messages, it might be faster to avoid this extra round-trip and send the message directly over TCP. This commit implements support for this. Messages smaller than a threshold (currently set to 4096 bytes) are sent via TCP, while larger messages still use shared memory.
This step also enables future optimizations such as queueing output messages in order to improve the throughput.
These two messages can be essential for correctness. For example, a node might not finish properly when an `InputClosed` event is lost. So we need to always send them, even if the target node was not subscribed yet when the event occurred.
Fixes a deadlock issue when the `incoming` channel is closed first. The problem was that the `send_out_buf` was set to `Fuse::terminated` without breaking the loop, so on the next loop iteration both futures were already terminated.
Without joining the thread it is killed suddenly by the OS when the executable exits. This causes interrupted connections to the daemon, which leads to some errors in the log messages. This commit fixes this by joining the event stream thread again, with the following improvements:
- Instead of joining the thread directly on drop, we now do the joining in a second thread and report the join result over a channel. This allows us to use a timeout when waiting for the join result in the drop implementation, so that we don't block indefinitely.
- We now share the ownership of the join handle between the node and the event stream. This way, its drop handler is only run after both instances were dropped. This way we avoid the deadlock that happened when joining the event steam thread before the event stream instance was dropped.
It is not guaranteed that the `EventStream` is dropped before the `DoraNode`. If it is dropped later on the same thread, this `join` leads to a deadlock.