dora-rs/dora - dora - 开源协同云脑生态支撑系统

Commit Graph

Author	SHA1	Message	Date
Philipp Oppermann	738507fa04	Batch dropping of messages by introducing new queue overshoot parameter	2 years ago
Philipp Oppermann	0b53324cea	Increase default queue size to 50	2 years ago
Philipp Oppermann	9bb57ee531	Don't send replies for `SendMessage` requests when using TCP Allows queueing of multiple messages on the TCP socket, which considerably improves throughput.	2 years ago
Philipp Oppermann	2ccc170649	Send all queued incoming events at once on `NextEvent` request Sends a list of all queued events in the reply instead of sending them one-by-one. This way, we avoid some communication overhead.	2 years ago
Philipp Oppermann	00421e2bcd	Send small messages directly without shared memory We currently need an additional request to prepare zero-copy shared memory messages. For small messages, it might be faster to avoid this extra round-trip and send the message directly over TCP. This commit implements support for this. Messages smaller than a threshold (currently set to 4096 bytes) are sent via TCP, while larger messages still use shared memory. This step also enables future optimizations such as queueing output messages in order to improve the throughput.	2 years ago
Philipp Oppermann	9439b08f16	Fix: Keep track of `InputClosed` and `Stop` messages and send them on subscribe These two messages can be essential for correctness. For example, a node might not finish properly when an `InputClosed` event is lost. So we need to always send them, even if the target node was not subscribed yet when the event occurred.	2 years ago
Philipp Oppermann	d5ceb4428a	Fix: Update `open_inputs` even if there is no subscriber for closed input	2 years ago
Philipp Oppermann	7232a31b57	Send `InputClosed` events to operators too	2 years ago
Philipp Oppermann	b994ab60a7	Add support for `shell`-based nodes again	2 years ago
Philipp Oppermann	9e4f01318a	Fix: Move `InputBuffer` break condition to end of loop Fixes a deadlock issue when the `incoming` channel is closed first. The problem was that the `send_out_buf` was set to `Fuse::terminated` without breaking the loop, so on the next loop iteration both futures were already terminated.	2 years ago
Philipp Oppermann	aae4a66705	Wait for control connections to finish too	2 years ago
Philipp Oppermann	97dab13a93	Wait for spawned listener tasks before exiting coordinator Ensures that we don't exit while some listener task is still sending out a reply.	2 years ago
Philipp Oppermann	4bbb724876	Flush TCP connections on sending in coordinator and daemon too	2 years ago
Philipp Oppermann	4d13a29dea	Improve error messages and fix clippy warnings in daemon	2 years ago
Philipp Oppermann	598ece765e	Don't panic in runtime when operator panics	2 years ago
Philipp Oppermann	ea537d45f6	Remove resolved TODO	2 years ago
Philipp Oppermann	bcbd9f016c	Fix typo in log message	2 years ago
Philipp Oppermann	a84f91c572	Fix: Don't keep on polling incoming event channel after it's closed This causes the `send_out_buf` future to completely starve.	2 years ago
Philipp Oppermann	2c80b6bcc2	Log when dropping inputs in daemon	2 years ago
Philipp Oppermann	a35850cb07	Drop oldest operator inputs when queue becomes too full	2 years ago
Philipp Oppermann	0f29d26fe8	Drop oldest operator inputs when queue becomes too full	2 years ago
Philipp Oppermann	a9203e0274	Add intermediate operator channel buffer to avoid blocking runtime	2 years ago
Philipp Oppermann	a9e9976b6b	Remove old unneeded code from runtime	2 years ago
Philipp Oppermann	871dd712f0	Make control channel listener async, handle coordinator exit cleanly, update reply type	2 years ago
Philipp Oppermann	019e411eef	Change CLI control request reply type	2 years ago
Philipp Oppermann	cdcb19fe28	Wait a bit before exiting coordinator to ensure that destroy confirmation is sent out	2 years ago
Philipp Oppermann	2ffae835d5	Fix: send `StopReply` instead of `SpawnReply` after receiving stop message in daemon	2 years ago
Philipp Oppermann	821aec2236	Improve 'unexpected reply' error messages in coordinator	2 years ago
Philipp Oppermann	de786ef101	Fix some warnings in runtime	2 years ago
Philipp Oppermann	8226805eb4	Fix import in Rust operator template	2 years ago
Philipp Oppermann	11bb9e446a	Update Rust operator template	2 years ago
Philipp Oppermann	cdc0be1206	Rename `FfiEvent` to `RawEvent`	2 years ago
Philipp Oppermann	e77804b293	Check for node exit errors when running single dataflow	2 years ago
Philipp Oppermann	62319f1ec1	Add support for Rust operators again	2 years ago
Philipp Oppermann	3658b5356d	Fix import in Rust node template	2 years ago
Philipp Oppermann	d334c6f243	Make `daemon` module of Rust node API private and reexport symbols	2 years ago
Philipp Oppermann	8c8f56a368	Update Python operator template for new `on_event` method	2 years ago
Philipp Oppermann	83c8ecf3e9	Change Python operator API: `on_event` instead of `on_input` We now report other event types as well. Right now, the only other event type is `Stop`, but we can extend this in the future.	2 years ago
Philipp Oppermann	1e07a73a64	Merge branch 'main' into unify-nodes-and-operators	2 years ago
Philipp Oppermann	8c2587e24d	Fix: abort watchdog timer events on destroy	2 years ago
Philipp Oppermann	f0242299b7	Simplify abortion of event streams on destroy	2 years ago
Philipp Oppermann	e521a2bc50	Add ctrlc handler to dora-coordinator	2 years ago
Philipp Oppermann	1553a198ef	Re-export dora-message from dora-core	2 years ago
Philipp Oppermann	2a86213e70	Cache shared memory regions in daemon and reuse them if size matches	2 years ago
Philipp Oppermann	8dc184a7e8	Improve a log message	2 years ago
Philipp Oppermann	5e4f1b2519	Clean up	2 years ago
Philipp Oppermann	f00e27bcf0	Limit enum size by boxing large fields	2 years ago
Philipp Oppermann	f43809f993	Reduce log output	2 years ago
Philipp Oppermann	9e2dca0e82	Don't error if listener exits because of `ConnectionReset` error	2 years ago
Philipp Oppermann	4ca7cc6666	Improve handling of stopped nodes - don't fail daemon when a node errors - detects when a node exited because of a signal - improve log messages	2 years ago

1 2 3 4 5 ...

333 Commits (tweak-queue-sizes)