haixuantao
31f0b33803
Fix coordinator not using build feature flag
6 months ago
Philipp Oppermann
125d5a0443
Merge branch 'main' into git-source
7 months ago
haixuantao
f72991b09c
Add Cause to stpo
7 months ago
Philipp Oppermann
f030d8d6b5
Fix typo
7 months ago
Philipp Oppermann
bf54c6d706
Improve log output for `dora run`
Send the log messages back to the CLI instead of logging them through tracing at the `daemon` level.
7 months ago
Philipp Oppermann
4392affe16
Tweak heuristic for doing local or distributed builds
7 months ago
Philipp Oppermann
65ded260c9
Convert `log` records to `tracing` events
Ensures that `log::info` etc messages are logged too.
7 months ago
Philipp Oppermann
9eac013637
Fix clone directory copying
7 months ago
Philipp Oppermann
84084b1475
Fix: Don't rename git dir if other nodes of same dataflow still need it
7 months ago
Philipp Oppermann
482a7ecb61
Put session file into `out` dir
7 months ago
Philipp Oppermann
5eeab6f905
Fix: Append session UUID to path, without `SessionId(...)` wrapper
Parentheses in directory names confuse some build tools/scripts (e.g. openssl Makefile).
7 months ago
Philipp Oppermann
5993403d28
Print build_id instead of session_id
7 months ago
Philipp Oppermann
31d7c975fc
Only print daemon and dataflows IDs if they're useful
7 months ago
Philipp Oppermann
950862b916
Add a special log level for `stdout` output
7 months ago
haixuantao
c64d6642af
fix string message
7 months ago
haixuantao
5988a65ea2
Expose all input closed message as a stop message
7 months ago
Philipp Oppermann
4d15cc51bb
Improve log output for local builds
7 months ago
Philipp Oppermann
eb50a368f0
Fix: Store build results to report them on subsequent `WaitForBuild` messages
Quick failures might occur before the CLI even sent the `WaitForBuild` message
7 months ago
Philipp Oppermann
e21520d433
Add `dora-session.yaml` to `.gitignore` to avoid accidental commits
The file stores local state and should not be commited to git repositories. By adding the file to a `.gitignore` we can avoid this.
Creates a new local `.gitignore` file if none exists. Otherwise, it appends the file to the existing `.gitignore` file.
7 months ago
Philipp Oppermann
6071ba71b9
Add pyo3-build-config and macos-deployment-target also for binaries/cli
7 months ago
Philipp Oppermann
003542c0fa
Add `build` feature to `dora-core` to make `git2` dependency optional
We don't want to include it for e.g. the `dora-node-api-c`.
7 months ago
Philipp Oppermann
a4382dedd1
Fix: Check `exit_when_done` after spawn failures too
7 months ago
Philipp Oppermann
14477b8645
Add a `deploy.working_dir` key to dataflow YAML
Allows overriding the working directory per node.
7 months ago
Philipp Oppermann
c3c22d160b
Link libz in C++ cmake template too
7 months ago
Philipp Oppermann
314707bf95
Improve log output
7 months ago
Philipp Oppermann
01168ddb02
Link libz in cmake template
7 months ago
Philipp Oppermann
9d917bdaa9
Parse dataflow descriptor lazily in node APIs
The dataflow descriptor format still changes often, which led to parse errors. By doing the parsing lazily, this should only affect users of the `dataflow_descriptor` function from now on.
7 months ago
Philipp Oppermann
829f483007
Make `dora run` error on distributed dataflows
7 months ago
Philipp Oppermann
187efed38e
Merge branch 'main' into git-source
7 months ago
Philipp Oppermann
51ed0194c7
Rework and refactor for two-step build
7 months ago
Haixuan Xavier Tao
d9a1b508c8
Fix spawning runtime through python when it is installed with pip ( #1011 )
This PR fixes #900 by checking the current exe of the daemon.
If the daemon is running on python => spawn the runtime on python.
If the daemon is running on a dora binary => use current dora binary.
Alternative to #940
7 months ago
haixuantao
2456e934ba
Fix wrong error message and wrong python spawning options
7 months ago
haixuantao
94001c989f
Use current python interpreter instead of python interpreter for non python operators
8 months ago
haixuantao
c197c1475c
Remove extension in filename
8 months ago
haixuantao
42b3509aa9
Fix spawning runtime through python when it is installed with pip
8 months ago
sjfhsjfh
51190788ba
refactor: use builder style
fix: logging thread block
8 months ago
mivik
6ba2d611f6
perf: remove unused sysinfo monitor
8 months ago
Philipp Oppermann
9f7c095998
Don't error if dynamic node stops after dataflow is done
We don't wait for dynamic nodes when the dataflow is done otherwise, so they might stop after the dataflow is done. This commit fixes a daemon error that happened in this case.
8 months ago
Philipp Oppermann
e897f78024
Add extra error cause when node fails to spawn
for better error messages
8 months ago
Philipp Oppermann
4c22c16570
Subscribe to log messages while waiting for dataflow spawning
8 months ago
Philipp Oppermann
95709b44b4
Refactor spawn result recording to fix `WaitForSpawnResult` handling
9 months ago
Philipp Oppermann
c9a720d7ae
Error if coordinator doesn't reply properly
Instead of sending `CoordinatorStopped`.
This should make it easier to debug when a reply is not sent as expected.
9 months ago
Philipp Oppermann
90733aabeb
Send build output to CLI
The coordinator now sends an immediate `DataflowStartTriggered` reply when receiving a `DataflowStart` command. This enables the CLI to directly attach to the dataflow and observe the build output.
To wait until the build/spawning is done, this commit introduces a new `WaitForSpawn` command, to which the coordinator replies with a `DataflowSpawned` message once the node has been started. We use this command for the `dora build` command.
9 months ago
Philipp Oppermann
a58936fa09
Refactor: Move git operations to submodule
9 months ago
Philipp Oppermann
80436901ce
Rename module from `spawn.rs` to `spawn/mod.rs` to prepare for submodules
9 months ago
Philipp Oppermann
f71ad7922e
Refactor reuse logic and improve documentation
9 months ago
Philipp Oppermann
17f75c04d5
Wait until all nodes on daemon have been built before spawning
9 months ago
Philipp Oppermann
80c7231fb9
Tweak logging
9 months ago
Philipp Oppermann
a13a297758
Fix: Remove dataflow from daemon running list if it was `build_only`
9 months ago
Philipp Oppermann
3b2ccec5ec
Fix error messages (actually include node ID instead of printing `'{node_id}'`
9 months ago