The coordinator now sends an immediate `DataflowStartTriggered` reply when receiving a `DataflowStart` command. This enables the CLI to directly attach to the dataflow and observe the build output.
To wait until the build/spawning is done, this commit introduces a new `WaitForSpawn` command, to which the coordinator replies with a `DataflowSpawned` message once the node has been started. We use this command for the `dora build` command.
Instead of running all the build commands directly, run them on their intended target machines through the coordinator.
This commit is a breaking change because a coordinator connection is now required for `dora build`.
The previous machine ID is still used, but optional. Users don't need to ensure that the chosen machine IDs are unique anymore because they are augmented with a UUID.
Make `dora-message` a dependency of `dora-core`, instead of the other way around. This way, we can continue to freely bump the version of `dora-core` with the other workspace crates, without introducing errors such as #708.
- introduce a new `ArrayData` wrapper struct
- implement `TryFrom` to convert to primitive types and strings
- update examples to use the new conversion functions
The coordinator is our control plane and should not be involved in data plane operations. This way, the dataflow can continue even if the coordinator fails.
Removes the separate `dora-runtime` binary. The runtime can now be started by passing `--run-dora-runtime` to `dora-daemon`. This change makes setup and deployment easier since it removes one executable that needs to be copied across machines.