This commit migrates the allocation of shared memory samples from the daemon to the individual nodes. This way, the nodes can prepare the memory themselves when sending outputs. Thus, we avoid the extra roundtrip to the daemon that we used before (the prepare request and reply). The other advantage is that we can now queue shared memory outputs on the TCP socket without waiting for replies from the daemon (like we already do for `Vec`-backed messages).
The intent of this commit is to remove the quantity of log that is being pushed to user.
This commit removes the redeclaration of a set up tracing methods to centralise
the tokio-tracing subscriber within the extension crate. It also add the
feature to filter information based on Environment variable.
This makes it possible to define the log level for tokio tracing like
this:
```
RUST_LOG=debug dora-daemon --run-dataflow dataflow.yml
```
I have also unified the feature flag to make it easier to manage tracing features among the workspace.
I did not change the default behaviour of tracing in our crates and therefore by
using the command above you should get the same tracing log as before.
fix merge conflict generated
We currently need an additional request to prepare zero-copy shared memory messages. For small messages, it might be faster to avoid this extra round-trip and send the message directly over TCP. This commit implements support for this. Messages smaller than a threshold (currently set to 4096 bytes) are sent via TCP, while larger messages still use shared memory.
This step also enables future optimizations such as queueing output messages in order to improve the throughput.
This commit enables users to pass environment variable as composition
of env variable such as:
```yaml
- id: yolov5
operator:
outputs:
- bbox
- ready
inputs:
image: oasis_agent/image
check: oasis_agent/yolov5_check
python: ../../operators/yolov5_op.py
env:
YOLOV5_PATH: $HOME/Documents/dependencies/yolov5
YOLOV5_WEIGHT_PATH: $DORA_DEP_HOME/dependencies/yolov5/yolov5n.pt
```
Note that this is only for the env field. Expanding env variable for
source path might conflict with using url as source path.