dora-rs/dora - dora - 开源协同云脑生态支撑系统

Commit Graph

Author	SHA1	Message	Date
Philipp Oppermann	cf95078ca1	Extract shared memory server/client into separate crate To allow for individual testing and benchmarking. This will also enable reuse of the library in other projects.	3 years ago
Philipp Oppermann	0e95b0868a	Fix: Report correct length in `SharedMemSample`	3 years ago
Philipp Oppermann	a24d7828e9	Fix: The drop tokens might reach the `SharedMemHandler` before the `SentOut` message	3 years ago
Philipp Oppermann	68b46f7ba7	Move shared memory handling to separate task To reduce load on the main task.	3 years ago
Philipp Oppermann	7b6f24c24b	Confirm output sending a bit earlier	3 years ago
Philipp Oppermann	b73c872043	Use blocks for initializing shmem severs	3 years ago
Philipp Oppermann	c6d864b20f	Make shmem server/client typed and report event stream closure	3 years ago
Philipp Oppermann	503a9497fd	Use shared memory for event stream too	3 years ago
Philipp Oppermann	670c154e5e	Create a safer server/client API on top of `ShmemChannel`	3 years ago
Philipp Oppermann	8d1c3742a1	Implement disconnect signal for `ShmemChannel` and set it on drop	3 years ago
Philipp Oppermann	be6c459c6c	Set log level to DEBUG for daemon	3 years ago
Philipp Oppermann	70298d836f	Start migrating the daemon control channel to shared memory The goal is to avoid the latency of the TCP socket.	3 years ago
Philipp Oppermann	5666895a7c	Close event channel when last input is closed or when stop is received	3 years ago
Philipp Oppermann	de4b048853	Fix performance: Set `nodelay` on listener side too Without this, the TCP connection uses Nagle's algorithm, which delays sending of packets until ACKs are received. This is problematic because most OSs deliberately delay ACKs to reduce overhead. For example, the default ACK delay on Linux is 40ms, which leads to 80ms latency when sending an output.	3 years ago
Philipp Oppermann	218106721d	Add watchdog for checking that coordinator is still reachable Throw an error if a daemon cannot reach the coordinator anymore. This is only an interim solution. In the future, we want to make the daemon more robust and ideally even allow restarts of the coordinator.	3 years ago
Philipp Oppermann	2b7d2508f4	Implement watchdog messages for detecting sudden disconnects of daemon	3 years ago
Philipp Oppermann	35e86684d0	Update `up` and `destroy` commands for dora-daemon and ensure proper coordinator exit	3 years ago
Philipp Oppermann	778d9d7eb7	Fix: Use correct length for received data The shared memory region size might be larger because of padding.	3 years ago
Philipp Oppermann	07311fd611	Check exit status of all nodes when using `exit_when_done` Instead of finishing already when all nodes sent their `Stopped` message, we now check the exit code of the node executables too.	3 years ago
Philipp Oppermann	1516ee6559	Don't allocate shared memory when `data_len=0` when preparing output	3 years ago
Philipp Oppermann	2e13ef5486	Set `stdin` to `null` when spawning nodes	3 years ago
Philipp Oppermann	5db784ff27	Move descriptor reading to dora daemon + add cli arg to run dataflow	3 years ago
Philipp Oppermann	611103b211	Add more logging to `rust-dataflow` example	3 years ago
Philipp Oppermann	550062917b	Create test/examples entry point for daemon and use it to run rust-dataflow example	3 years ago
Philipp Oppermann	824da6a00c	Make the coordinator connection optional	3 years ago
Philipp Oppermann	34e4e62188	Split off library from daemon binary To allow usage for tests and examples.	3 years ago
Philipp Oppermann	2ed7db1b0e	Make `dora stop` command working with new daemon design	3 years ago
Philipp Oppermann	377703c325	Report finished dataflows from daemon to coordinator Makes the `dora list` command work.	3 years ago
Philipp Oppermann	2527ca0e85	Report spawn result from daemon to coordinator	3 years ago
Philipp Oppermann	5982815968	Don't block daemon when receivers cannot keep up This can easily cause deadlocks.	3 years ago
Philipp Oppermann	a2cc06ca50	Use drop tokens and reference counting to free shared memory again after usage	3 years ago
Philipp Oppermann	8bfb665201	Set `TCP_NODELAY` for all TCP connections to reduce latency	3 years ago
Philipp Oppermann	3c4469d4d3	Cancel timer tasks once a dataflow is finished	3 years ago
Philipp Oppermann	f0e31e2cc6	Report results of spawned nodes as events	3 years ago
Philipp Oppermann	3cd1b4de65	Notify downstream nodes about closed inputs	3 years ago
Philipp Oppermann	a10fd0b1cb	Implement timer messages	3 years ago
Philipp Oppermann	672fb4d0b4	Close connections on errors	3 years ago
Philipp Oppermann	ce4e5a1616	Send metadata through TCP for now	3 years ago
Philipp Oppermann	1724c86af8	Also treat `BrokenPipe` error as stopped connection	3 years ago
Philipp Oppermann	a0d0b21c1b	Map outputs to inputs of local nodes	3 years ago
Philipp Oppermann	0ae5fc2ffa	Add dataflow ID to daemon messages	3 years ago
Philipp Oppermann	cc1dc971ed	Update dora-coordinator to start dataflows through dora-daemon	3 years ago
Philipp Oppermann	5ae18eae4c	Include dataflow ID in daemon to avoid node ID conflicts	3 years ago
Philipp Oppermann	e5d7ac55e2	Bundle all nodes of a single machine in a single daemon spawn command Avoids synchronization issues, e.g. if one node sends outputs before the receiver was launched and registered.	3 years ago
Philipp Oppermann	3312af1941	Connect to dora-coordinator and implement command to spawn nodes	3 years ago
Philipp Oppermann	2e7ef8b630	Implement shared memory mapping in node API	3 years ago
Philipp Oppermann	e36f15e369	Add support to subscribe to messages in daemon	3 years ago
Philipp Oppermann	c2e4948dd5	Update Rust node API to communicate through dora-daemon	3 years ago
Philipp Oppermann	7d85b92b2a	Connect to dora-daemon from node API and register with node ID	3 years ago
Philipp Oppermann	f7c6cca943	Move port number constant to `dora-core`	3 years ago

1 2

51 Commits (cf95078ca12f39dcfb00beea78ca8d450c8eebe9)