Philipp Oppermann
d610140e68
Refactor dora library: Add `DoraConnection` struct
1 year ago
Philipp Oppermann
ce18beb232
Pass dora CLI path as argument instead of using `current_exe`
1 year ago
Philipp Oppermann
987ccde1e7
Split CLI into library and binary part
Enables other executables to invoke dora CLI commands directly without going through the argument parsing.
1 year ago
haixuanTao
7bcb132075
Retrieve `open_telemetry_context` from metadata parameters.
1 year ago
Haixuan Xavier Tao
16ef29ab80
Merge pull request #601 from dora-rs/delay-node-drop
Delay dropping of `DoraNode` in Python until all event data is freed
1 year ago
Philipp Oppermann
9cec8df1cc
Delay dropping of `DoraNode` in Python until all event data is freed
When dropping the `DoraNode`, it waits for the remaining drop tokens. This only works if all the dora events were already dropped before. With the Python GC, this is not guaranteed as some events might still be live on the heap (the user might even use them later). In such cases, we waited until we ran into a timeout, which resulted in very long exit times (see https://github.com/dora-rs/dora/issues/598 ).
This commit fixes this issue by adding a reference-counted copy of the `DoraNode` and `EventStream` to every event given to Python. This way, we can ensure that the underlying `DoraNode` is only dropped after the last event reference has been freed.
1 year ago
chang xu
bccb1ae27d
Add domain unix socket supports ( #594 )
1 year ago
Philipp Oppermann
508311ed9b
Don't wait for non-started dynamic nodes on stop
1 year ago
haixuanTao
0a61082476
Increasing grace duration to 2 seconds so that drop token get well returned or ignored
1 year ago
XxChang
1f23470d8b
minor
1 year ago
Philipp Oppermann
4e2d85e3b0
Fix or silence clippy warnings
1 year ago
Philipp Oppermann
a29189a040
Close log subscribers that cannot keep up
We need to avoid blocking the coordinator for too long.
1 year ago
Philipp Oppermann
1f32b29afe
Close log subscriber connection if send fails (instead of erroring)
1 year ago
Philipp Oppermann
2d0f0dcf5e
Merge branch 'main' into error-logging
1 year ago
Philipp Oppermann
1aaf5ef1a2
Improve log output
1 year ago
Philipp Oppermann
2680e61404
Change default log level to info and consider `RUST_LOG`
Allows to override the log level through the `RUST_LOG` env variable.
1 year ago
Philipp Oppermann
8aecbb249c
Log when node finishes or fails
1 year ago
Philipp Oppermann
e90ffd0b7f
Merge branch 'main' into list-failed-dataflows
1 year ago
Philipp Oppermann
8d2d88c69d
Merge pull request #561 from dora-rs/attach-as-default
Make `dora start` attach by default, add `--detach` to opt-out
1 year ago
Philipp Oppermann
5bda3338af
Only add `[...]` omission marker if stderr buffer is full
1 year ago
Philipp Oppermann
f91c9e44b2
Make `dora start` attach by default, add `--detach` to opt-out
1 year ago
Philipp Oppermann
ee394361e9
Add error cause for grace duration kills
1 year ago
Philipp Oppermann
2e0419b100
Add basic log forwarding from daemon to CLI
Forwarded by coordinator
1 year ago
Philipp Oppermann
361ea27485
Slightly tweak error printing
1 year ago
Philipp Oppermann
4fb5a96390
Use `handle_dataflow_result for attach too
1 year ago
Philipp Oppermann
0423155b54
Refactor: move dataflow result handling to a separate function
1 year ago
Philipp Oppermann
a0e8fff411
Include dataflow UUID in error message
Co-authored-by: Haixuan Xavier Tao <tao.xavier@outlook.com>
1 year ago
Philipp Oppermann
d2e2eef99e
Slightly adjust how status is printed
1 year ago
Philipp Oppermann
262f39fb98
Refactor: Report dataflow list as one `Vec`
1 year ago
Philipp Oppermann
cdb3123fa8
Format dora list output as table
1 year ago
Philipp Oppermann
5a210b688f
Merge branch 'main' into better-errors
1 year ago
Philipp Oppermann
912338d806
Shorten node_exited_before_subscribe error message
1 year ago
Philipp Oppermann
450002ed92
Fix number of consequential errors and improve message
1 year ago
Philipp Oppermann
94fecd61c5
Print all non-cascading errors (instead of only first one)
1 year ago
Haixuan Xavier Tao
42255f5a70
Merge pull request #557 from dora-rs/convert-pyevent-into-dictionary
Transform custom PyEvent into standard python dictionary for easier d…
1 year ago
Philipp Oppermann
b231df1b82
Report last 10 stderr lines on node failure
1 year ago
haixuanTao
c4263e6c8f
Remove bound from definition of .to_py_dict
1 year ago
haixuanTao
b2eace277d
Transform custom PyEvent into standard python dictionary for easier debuggability.
Currently having a custom PyEvent make debugging very hard as fields are hidden within the class PyEvent that is defined within Rust Code.
Python user are getting really confused about this obscure class.
This PR transforms the class into a standard python dictionary.
1 year ago
Philipp Oppermann
39164b9634
Remove old formatting methods
1 year ago
Philipp Oppermann
72bc9cd74e
Print error causes and include node that caused error
1 year ago
XxChang
2488d0916c
fix event_as_input bug
1 year ago
XxChang
7cca18f726
minor
1 year ago
XxChang
7e9c589e56
resolve suggestion
1 year ago
Philipp Oppermann
4e69495012
Fix: Pass `cascading_errors` as arg to ensure that it is always applied
Before this commit, there were some cases where the returned `DataflowStatus` was ignored and the reported `cascading_errors` were never applied.
1 year ago
Philipp Oppermann
9425e4a540
Merge branch 'main' into better-errors
1 year ago
Philipp Oppermann
71ea44ad01
Mark node failures as cascading on init errors caused by other nodes
The init function returns an error if another node exited before initialization. In this case, we consider the subsequent error as a cascading error and skip it when printing the error message to the user.
1 year ago
XxChang
60b86fdfad
resolve suggestion
1 year ago
XxChang
02c5228756
resolve suggestion
1 year ago
XxChang
f6816e60c7
fix review
1 year ago
Philipp Oppermann
8056c23db7
List failed and finished dataflows in `dora list`
1 year ago