Philipp Oppermann
d610140e68
Refactor dora library: Add `DoraConnection` struct
1 year ago
Philipp Oppermann
4e2d85e3b0
Fix or silence clippy warnings
1 year ago
Philipp Oppermann
a29189a040
Close log subscribers that cannot keep up
We need to avoid blocking the coordinator for too long.
1 year ago
Philipp Oppermann
1f32b29afe
Close log subscriber connection if send fails (instead of erroring)
1 year ago
Philipp Oppermann
2d0f0dcf5e
Merge branch 'main' into error-logging
1 year ago
Philipp Oppermann
2680e61404
Change default log level to info and consider `RUST_LOG`
Allows to override the log level through the `RUST_LOG` env variable.
1 year ago
Philipp Oppermann
e90ffd0b7f
Merge branch 'main' into list-failed-dataflows
1 year ago
Philipp Oppermann
2e0419b100
Add basic log forwarding from daemon to CLI
Forwarded by coordinator
1 year ago
Philipp Oppermann
262f39fb98
Refactor: Report dataflow list as one `Vec`
1 year ago
Philipp Oppermann
39164b9634
Remove old formatting methods
1 year ago
Philipp Oppermann
72bc9cd74e
Print error causes and include node that caused error
1 year ago
Philipp Oppermann
9425e4a540
Merge branch 'main' into better-errors
1 year ago
Philipp Oppermann
8056c23db7
List failed and finished dataflows in `dora list`
1 year ago
Haixuan Xavier Tao
f34cbe2ffc
Merge pull request #517 from dora-rs/detached-python-process
Run dynamic node
1 year ago
Philipp Oppermann
3019eba371
Print only first node error and report more metadata in dataflow results
Allows us to mark certain node errors as cascading, which will deprioritize them when printed.
Also updates the error printing code to only print the error that happened first.
1 year ago
chang xu
799a3a6363
Refuse relative path for remote in coordinator ( #538 )
* refuse pass relative path to remote daemon
* add local ip checker in dora start
* delete some additional packages on CI to make enough space available
* add coodinator_is_remote argument
1 year ago
haixuanTao
d180aa7345
Adding configuration for dynamic nodes port
1 year ago
EricLBuehler
6c62246e53
Fix typos
1 year ago
Philipp Oppermann
2ccba2b953
Allow setting custom control port in coordinator
1 year ago
Philipp Oppermann
22ba6e02a5
Adjust dataflow node parsing to be more flexible
We plan to (soft-)remove operators and simplify the dataflow YAML file by removing the additional nesting caused by the `custom` field. This commit prepares for that. See #474 for context.
1 year ago
haixuanTao
a631cafe52
Reply to stop msg immediately to coordinator
1 year ago
haixuanTao
b9f67a2855
Add grace duration and kill process
1 year ago
Michael-J-Ward
ae9e19e57f
refactor: format error messages without allocating a string for each line
I added a test to confirm the output matches the old.
Fixes clippy warning clippy::format_collect
https://rust-lang.github.io/rust-clippy/master/index.html#/format_collect
1 year ago
Philipp Oppermann
01707fc14d
Fix: Use `peer_addr` of incoming daemon register request for listen socket
We are not interested in the local bind address of the daemon. Instead, we want to use the IP address under which the daemon is available from other machines.
This should also avoids the issue that connecting to 0.0.0.0 is not possible on Windows (we want to use 0.0.0.0 as default bind address).
1 year ago
Philipp Oppermann
6115ddbb4d
Revert "update dora_coordinator::start to return bound addr instead of just the port"
This reverts commit 1ba763cd67 .
1 year ago
Michael-J-Ward
1ba763cd67
update dora_coordinator::start to return bound addr instead of just the port
1 year ago
Philipp Oppermann
279530e21d
Delay watchdog disconnect to 30 seconds
1 year ago
Philipp Oppermann
58eac29024
Increase log level of 'disconnecting daemons' message to `error`
1 year ago
Michael-J-Ward
3a6323ff0c
feat: add bind configuration for coordination daemon
Ref: #459
1 year ago
haixuanTao
f0db125499
Use `Uuid::v7` to have chronological Uuid
1 year ago
Philipp Oppermann
cb69a033f0
Wait until `DestroyResult` is sent before exiting dora-daemon
Tokio background tasks are canceled when the program exits, so we need some additional synchronization to ensure that the `DestroyResult` is actually sent out.
2 years ago
haixuanTao
054ae5afcc
Fix multiple daemon example
2 years ago
haixuanTao
ddf94f7094
Adding ctrlc back to their individual repo
2 years ago
haixuanTao
0568656ef0
Remove unused dependencies for faster compile time
2 years ago
haixuanTao
4ec81cc743
Removing old main files
2 years ago
haixuanTao
6dfd4b17b3
adding daemon, coordinator and runtime command in the cli
2 years ago
haixuanTao
fd7c0939df
Run `cargo fmt --all`
2 years ago
haixuanTao
ea1259da20
Bump which version
2 years ago
Philipp Oppermann
bbb9b02a0d
Update clap to v0.4.3
2 years ago
Philipp Oppermann
4da34d82d9
Merge branch 'main' into event-timestamps
2 years ago
Philipp Oppermann
5139c85a87
Use timestamps for coordinator messages and inter daemon events
2 years ago
Haixuan Xavier Tao
2e9108e894
Merge pull request #301 from dora-rs/pass-dataflow-to-nodes
Make dataflow descriptor available to Python nodes and operators
2 years ago
Philipp Oppermann
8a439c3411
Pass dataflow descriptor to Rust nodes via API
2 years ago
Philipp Oppermann
295614fac0
Return dataflow result when trying to stop stopped dataflow
The dataflow might be finished already when `dora stop` is invoked, which caused an error previously. This commit changes this behavior to return the dataflow result instead in this case.
2 years ago
Philipp Oppermann
868bee5897
Wait for dataflow finish on `dora stop`
2 years ago
Philipp Oppermann
3bb665ae00
Pass dataflow result to CLI
2 years ago
Philipp Oppermann
e46ea2f6f6
Make coordinator's `start_inner` function `Send` again
The name generator uses a thread-local random number generator internally, which makes it `!Send`. So we should not keep it across await points because our `multiple_daemons` example needs to spawn it in the tokio runtime, which requires `Send`.
2 years ago
Philipp Oppermann
baebc49396
Autogenerate a name for started dataflows if none is given
2 years ago
Philipp Oppermann
97bc9d0f92
Merge branch 'main' into heartbeat
2 years ago
Philipp Oppermann
5edfc394a6
Use monotonic `Instant` instead of `SystemTime`
2 years ago