dora-rs/dora - dora - 开源协同云脑生态支撑系统

Commit Graph

Author	SHA1	Message	Date
haixuanTao	ca182bb131	Provide a way to access logs through the CLI	3 years ago
Haixuan Xavier Tao	5c699cc16c	Merge pull request #256 from dora-rs/multiple-daemons Add support for distributed deployments with multiple daemons	3 years ago
Philipp Oppermann	e13415d532	Send cross-machine outputs directly to target daemon without involving coordinator The coordinator is our control plane and should not be involved in data plane operations. This way, the dataflow can continue even if the coordinator fails.	3 years ago
haixuanTao	ddd460b826	Make level filtering be `WARN` or below	3 years ago
Philipp Oppermann	8bf124a014	Merge branch 'main' into multiple-daemons	3 years ago
Philipp Oppermann	504e98e052	Fix: `deny_unknown_fields` is not supported in combination with `flatten`	3 years ago
Philipp Oppermann	a72f17b71b	Deny unknown fields in operator and deploy config	3 years ago
Philipp Oppermann	c28427d8da	Pass parsed dataflow descriptor instead of path We still need to pass the path through a new `working_dir` field as we haven't figured out deployment of compiled nodes/operators yet.	3 years ago
Philipp Oppermann	d535dfee4b	Prefix new YAML keys with `_unstable_`	3 years ago
haixuanTao	d4b3468d45	Filter default log level at `warn` for `tokio::tracing`	3 years ago
Philipp Oppermann	aac5e6515a	Integrate `dora-runtime` into `dora-daemon` Removes the separate `dora-runtime` binary. The runtime can now be started by passing `--run-dora-runtime` to `dora-daemon`. This change makes setup and deployment easier since it removes one executable that needs to be copied across machines.	3 years ago
Philipp Oppermann	d6571bf305	Rework communication config	3 years ago
Philipp Oppermann	17c52e3482	Use control messages in `multiple-daemons` example instead of sleeping	3 years ago
Philipp Oppermann	c523b3b9d4	Report closed inputs to remote daemons	3 years ago
Philipp Oppermann	188813cd57	Make coordinator usable in examples Allow passing runtime path via coordinator	3 years ago
Philipp Oppermann	852fc2c363	Synchronize node startup with coordinator	3 years ago
Philipp Oppermann	7397bcc4c3	Forward outputs through coordinator to target machine and nodes	3 years ago
Philipp Oppermann	b1d9b58d08	Create new `ResolvedDeploy` type with non-`Option` fields	3 years ago
Philipp Oppermann	7b8de70f88	Set deploy defaults when resolving aliases Ensures that we set them also when using the daemon's `run_dataflow` function for examples.	3 years ago
Philipp Oppermann	6ce8b7a26f	Allow configuring deploy machine	3 years ago
Philipp Oppermann	a453adf217	Fix: `ReportDropTokens` message expects no reply when using TCP	3 years ago
Philipp Oppermann	8dff241c2d	Signal daemon when event stream is dropped This allows the daemon to close the event stream directly.	3 years ago
Philipp Oppermann	a308c48f3e	Refactor Rust node API to ensure proper stopping	3 years ago
Philipp Oppermann	abd850c81e	Send drop tokens over separate channel	3 years ago
Philipp Oppermann	a19a746d5a	Report remaining drop tokens before event stream thread finishes	3 years ago
Philipp Oppermann	06e020472b	Check that dora daemon and node API versions match	3 years ago
Philipp Oppermann	ad0d585758	Check that dora daemon and coordinator versions match	3 years ago
haixuanTao	fea136d6dc	Add `Default` to serde descriptor to remove the need of the `communication` keyword Bump version to `v0.2.2-rc-2`	3 years ago
haixuanTao	813fba9516	Validate shell command as source and url source. In #237, I have grouped the validation of yaml as a method of descriptor. This validation was copied from the `cli check` method. However, we did not add the validation of shell command and accept url as valid source in the original `cli check`. This Pull Request validate both sources.	3 years ago
haixuanTao	bd9c370dbf	Add Reload and Check enum	3 years ago
haixuanTao	2d76cd5cc8	Add `Reload` event containing `dataflow_id, node_id, op_id` that needs to be reloaded	3 years ago
haixuanTao	d3b612faf2	Use opentelemetry if env variable `DORA_JAEGER_TRACING` is set	3 years ago
haixuanTao	976a632c24	Remove cargo.lock update and clippy import warning	3 years ago
haixuanTao	b4fda251a1	Use pretty-print for trace span logs	3 years ago
haixuanTao	eed7041dab	Use `tracing_opentelemetry` to unify tracing	3 years ago
haixuanTao	1e965452d5	Replace opentelemetry with tokio::tracing	3 years ago
Philipp Oppermann	f53fe65846	Merge pull request #237 from dora-rs/validate-yaml Validate yaml when reading	3 years ago
haixuanTao	6ad1cabc89	Option communication config As we are not currently using zenoh communication, it would be preferable to not mention it in the datalflow graph as some people might: - A. confuse it with our shared memory. - B. Question why it is there. - C. Question what is zenoh. I think that we can support dora-rs without external communication config, as I can see many use-case in simulation.	3 years ago
haixuanTao	480bafb3a8	Fix clippy	3 years ago
haixuanTao	2b1ae1ad93	Add `runtime_path` arg into descriptor check typography	3 years ago
haixuanTao	d28fb3f651	validate `dora-runtime` when checking dataflow. Both Python and shared library. And make sure that `dora-runtime` version match the `dora-core` version	3 years ago
haixuanTao	af4a79e1e0	Remove `base_path` from dataflow description	3 years ago
haixuanTao	a15cf4a64d	add `is_valid` method to make validation optional	3 years ago
haixuanTao	704d8db565	Use validation logic on every yaml description to avoid undefined behaviour	3 years ago
haixuanTao	db92865c3f	Remove duplicated `read_descriptor` function	3 years ago
Philipp Oppermann	2d9fcc95b4	Add custom parsing to be backwards compatible Inputs can now either be a string, which results in a mapping with default values, or an object with additional configuration.	3 years ago
Philipp Oppermann	7f3041ed6c	Allow configuring queue size per input	3 years ago
Philipp Oppermann	c9ecc87d3b	Rename `max_queue_len` to `queue_size`	3 years ago
Philipp Oppermann	8c55397811	Make event queue length configurable	3 years ago
haixuanTao	69bfd1e8c5	Replace git `raw_sync` with cargo `raw_sync_2` to publish our own crate on `crates.io`	3 years ago

1 2 3 4

195 Commits (cb39cb00a1e239affc6a1fb4075d8e414ac453a9)