The dataflow descriptor format still changes often, which led to parse errors. By doing the parsing lazily, this should only affect users of the `dataflow_descriptor` function from now on.
The coordinator now sends an immediate `DataflowStartTriggered` reply when receiving a `DataflowStart` command. This enables the CLI to directly attach to the dataflow and observe the build output.
To wait until the build/spawning is done, this commit introduces a new `WaitForSpawn` command, to which the coordinator replies with a `DataflowSpawned` message once the node has been started. We use this command for the `dora build` command.
Fixes the tests/examples of this repo. A UNC path as working directory doesn't work because the build commands of our examples invoke `cargo` on the dora repo. The dora repo uses wildcards in the member list in the top-level Cargo.toml, which are ignored in UNC-paths. Thus, `failed to load manifest for workspace member` errors occur.
The previous machine ID is still used, but optional. Users don't need to ensure that the chosen machine IDs are unique anymore because they are augmented with a UUID.
Make `dora-message` a dependency of `dora-core`, instead of the other way around. This way, we can continue to freely bump the version of `dora-core` with the other workspace crates, without introducing errors such as #708.
By versioning the `dora-message` crate individually, we can use the
semver rules to encode which versions are compatible. This way, we can
allow different versions of dora to work together (e.g. CLI version can
be different than node API version), as long as the message formats are
compatible. Breaking message format changes are signaled by a
semver-incompatible release of `dora-message`. For example, 0.4.0 is not
compatible with 0.3.5.
One alternative approach could be to use the main version to signal
compatibility, i.e. the common version that we use for all dora crates.
This has the disadvantage that we might need to bump the minor version
of the main dora crate every time we want to change the message format
in a breaking way. As we still expect semi-regular breaking changes to
the message format in the near future, we want to avoid this churn. Once
we consider the message format more stable, we plan to revisit this
approach.
Fixes https://github.com/dora-rs/dora/issues/504
TODO
- [x] Update release script: We should not try to publish the
`dora-message` crate if there is no new version.
- [x] Relax version checks to only compare major/minor version
(according to semver compatibility rules).