The dataflow descriptor format still changes often, which led to parse errors. By doing the parsing lazily, this should only affect users of the `dataflow_descriptor` function from now on.
This PR makes it possible to run dataflow from python using:
```python
from dora import run
# Make sure to build it first with the CLI.
# Build step is on the todo list.
run("qwen2.5.yaml", uv=True)
```
Making it easier to run dataflow based on our needs
By versioning the `dora-message` crate individually, we can use the
semver rules to encode which versions are compatible. This way, we can
allow different versions of dora to work together (e.g. CLI version can
be different than node API version), as long as the message formats are
compatible. Breaking message format changes are signaled by a
semver-incompatible release of `dora-message`. For example, 0.4.0 is not
compatible with 0.3.5.
One alternative approach could be to use the main version to signal
compatibility, i.e. the common version that we use for all dora crates.
This has the disadvantage that we might need to bump the minor version
of the main dora crate every time we want to change the message format
in a breaking way. As we still expect semi-regular breaking changes to
the message format in the near future, we want to avoid this churn. Once
we consider the message format more stable, we plan to revisit this
approach.
Fixes https://github.com/dora-rs/dora/issues/504
TODO
- [x] Update release script: We should not try to publish the
`dora-message` crate if there is no new version.
- [x] Relax version checks to only compare major/minor version
(according to semver compatibility rules).