The file stores local state and should not be commited to git repositories. By adding the file to a `.gitignore` we can avoid this.
Creates a new local `.gitignore` file if none exists. Otherwise, it appends the file to the existing `.gitignore` file.
The dataflow descriptor format still changes often, which led to parse errors. By doing the parsing lazily, this should only affect users of the `dataflow_descriptor` function from now on.
This PR fixes#900 by checking the current exe of the daemon.
If the daemon is running on python => spawn the runtime on python.
If the daemon is running on a dora binary => use current dora binary.
Alternative to #940
We don't wait for dynamic nodes when the dataflow is done otherwise, so they might stop after the dataflow is done. This commit fixes a daemon error that happened in this case.
The coordinator now sends an immediate `DataflowStartTriggered` reply when receiving a `DataflowStart` command. This enables the CLI to directly attach to the dataflow and observe the build output.
To wait until the build/spawning is done, this commit introduces a new `WaitForSpawn` command, to which the coordinator replies with a `DataflowSpawned` message once the node has been started. We use this command for the `dora build` command.
Instead of running all the build commands directly, run them on their intended target machines through the coordinator.
This commit is a breaking change because a coordinator connection is now required for `dora build`.
Spawn nodes one-by-one instead of doing it concurrently. Subsequent nodes might want to reuse git repos or build artifacts of previous nodes. Spawning them in parallel might lead to conflicts and errors (e.g. because a git repo reuse is handled before a git clone or because some build command is not concurrency-safe).
The actual directory creation and clone call will happen later in an asynchronous task, but we still want subsequent nodes to treat this directory as used. Otherwise, multiple nodes might try to clone into the same directory, which can lead to errors.