Allows us to mark certain node errors as cascading, which will deprioritize them when printed.
Also updates the error printing code to only print the error that happened first.
* refuse pass relative path to remote daemon
* add local ip checker in dora start
* delete some additional packages on CI to make enough space available
* add coodinator_is_remote argument
We plan to (soft-)remove operators and simplify the dataflow YAML file by removing the additional nesting caused by the `custom` field. This commit prepares for that. See #474 for context.
We are not interested in the local bind address of the daemon. Instead, we want to use the IP address under which the daemon is available from other machines.
This should also avoids the issue that connecting to 0.0.0.0 is not possible on Windows (we want to use 0.0.0.0 as default bind address).
Tokio background tasks are canceled when the program exits, so we need some additional synchronization to ensure that the `DestroyResult` is actually sent out.
The dataflow might be finished already when `dora stop` is invoked, which caused an error previously. This commit changes this behavior to return the dataflow result instead in this case.
The name generator uses a thread-local random number generator internally, which makes it `!Send`. So we should not keep it across await points because our `multiple_daemons` example needs to spawn it in the tokio runtime, which requires `Send`.
The previous watchdog requests required waiting for a reply, which could slow down the system under load and also lead to false errors on slower systems (e.g. the CI). Because of this, this commit replaces the watchdog requests with asynchronous heartbeat messages, which don't require replies. Instead, we record the last heartbeat timestamp whenever we receive a heartbeat message. We then periodically check the time since the last received heartbeat message and consider the connection closed if too much time has passed.