dora-rs/dora - dora - 开源协同云脑生态支撑系统

Commit Graph

Author	SHA1	Message	Date
Philipp Oppermann	6a3dcecf33	Refactor: Move message definitions to `dora-message` crate First step towards versioning the message definitions indepedently.	2 years ago
chang xu	bccb1ae27d	Add domain unix socket supports (#594 )	2 years ago
haixuanTao	3ad402ce45	Add dynamic node event loop and dynamic node connection	2 years ago
Philipp Oppermann	984ee364e4	Fill in timestamps when sending events from node to daemon	3 years ago
Philipp Oppermann	a308c48f3e	Refactor Rust node API to ensure proper stopping	3 years ago
Philipp Oppermann	abd850c81e	Send drop tokens over separate channel	3 years ago
Philipp Oppermann	ef37caa3ed	Add arrow support for Python operator inputs Reuse same event type as for Python node API.	3 years ago
Philipp Oppermann	a19a746d5a	Report remaining drop tokens before event stream thread finishes	3 years ago
Philipp Oppermann	59f4dde8e4	Improve tracing output	3 years ago
Philipp Oppermann	bb822a5ff4	Move pending drop token handling to separate function	3 years ago
Philipp Oppermann	0ed4dcf0f6	Handle drop tokens asynchronously instead of waiting for them The Python garbage collection will drop them non-deterministically in the background. Also, the dropping cannot happen while the GIL is held by our wait code.	3 years ago
Philipp Oppermann	446fa7f0dc	Wait for remaining drop tokens before breaking from event stream thread There might still be some pending drop tokens after the receiving end of the event stream was closed. So we don't want to break from the receiver thread directly. Instead we keep it running until the control channel signals that it expects no more drop tokens by closing the `finished_drop_tokens` channel. This happens either when all required drop tokens were received, or because of a timeout.	3 years ago
Philipp Oppermann	9030e729fe	Extract event stream loop into separate function	3 years ago
Philipp Oppermann	0d0d6630a1	Add TODO	3 years ago
Philipp Oppermann	1fab082717	Use `arrow` crate to send events from Rust to Python without copying See #224 for details.	3 years ago
Philipp Oppermann	06e020472b	Check that dora daemon and node API versions match	3 years ago
haixuanTao	8f05ec79a1	Add attach logic and hot-reloading logic to the cli	3 years ago
haixuanTao	2d76cd5cc8	Add `Reload` event containing `dataflow_id, node_id, op_id` that needs to be reloaded	3 years ago
haixuanTao	c565071140	Change `instrument` level to not pollute user logs	3 years ago
Philipp Oppermann	b4ebbff074	Allocate shared memory in nodes to improve throughput This commit migrates the allocation of shared memory samples from the daemon to the individual nodes. This way, the nodes can prepare the memory themselves when sending outputs. Thus, we avoid the extra roundtrip to the daemon that we used before (the prepare request and reply). The other advantage is that we can now queue shared memory outputs on the TCP socket without waiting for replies from the daemon (like we already do for `Vec`-backed messages).	3 years ago
Philipp Oppermann	feca83e309	Refactor: Split Rust node API into smaller submodules	3 years ago
Philipp Oppermann	c21b70ae65	Rename submodule	3 years ago

22 Commits (2cef9eb6261d862dfe02a07f78f64451e71ca43c)