The dataflow descriptor format still changes often, which led to parse errors. By doing the parsing lazily, this should only affect users of the `dataflow_descriptor` function from now on.
By versioning the `dora-message` crate individually, we can use the
semver rules to encode which versions are compatible. This way, we can
allow different versions of dora to work together (e.g. CLI version can
be different than node API version), as long as the message formats are
compatible. Breaking message format changes are signaled by a
semver-incompatible release of `dora-message`. For example, 0.4.0 is not
compatible with 0.3.5.
One alternative approach could be to use the main version to signal
compatibility, i.e. the common version that we use for all dora crates.
This has the disadvantage that we might need to bump the minor version
of the main dora crate every time we want to change the message format
in a breaking way. As we still expect semi-regular breaking changes to
the message format in the near future, we want to avoid this churn. Once
we consider the message format more stable, we plan to revisit this
approach.
Fixes https://github.com/dora-rs/dora/issues/504
TODO
- [x] Update release script: We should not try to publish the
`dora-message` crate if there is no new version.
- [x] Relax version checks to only compare major/minor version
(according to semver compatibility rules).
When dropping the `DoraNode`, it waits for the remaining drop tokens. This only works if all the dora events were already dropped before. With the Python GC, this is not guaranteed as some events might still be live on the heap (the user might even use them later). In such cases, we waited until we ran into a timeout, which resulted in very long exit times (see https://github.com/dora-rs/dora/issues/598).
This commit fixes this issue by adding a reference-counted copy of the `DoraNode` and `EventStream` to every event given to Python. This way, we can ensure that the underlying `DoraNode` is only dropped after the last event reference has been freed.
Currently having a custom PyEvent make debugging very hard as fields are hidden within the class PyEvent that is defined within Rust Code.
Python user are getting really confused about this obscure class.
This PR transforms the class into a standard python dictionary.
The deprecation warning states:
> code not using the `GIL Refs` API can safely remove the use of `Py::new_pool`
All GIL Refs usage have been removed, so this should be fine.