Philipp Oppermann
8a4ddf4f4d
Use UUID v7 instead of v4 for build and session IDs
V7 UUIDs include a timestamp and are sortable. This is useful to see which ID is newer.
1 year ago
Philipp Oppermann
003542c0fa
Add `build` feature to `dora-core` to make `git2` dependency optional
We don't want to include it for e.g. the `dora-node-api-c`.
1 year ago
Philipp Oppermann
0a60e3dbb6
Log a warning that git working dir is still unstable and might change
1 year ago
Philipp Oppermann
2a05324692
Fix some warnings
1 year ago
Philipp Oppermann
634f5cb583
Bump dora-message version to 0.5.0-alpha
1 year ago
Philipp Oppermann
14477b8645
Add a `deploy.working_dir` key to dataflow YAML
Allows overriding the working directory per node.
1 year ago
Philipp Oppermann
314707bf95
Improve log output
1 year ago
Philipp Oppermann
9d917bdaa9
Parse dataflow descriptor lazily in node APIs
The dataflow descriptor format still changes often, which led to parse errors. By doing the parsing lazily, this should only affect users of the `dataflow_descriptor` function from now on.
1 year ago
Philipp Oppermann
829f483007
Make `dora run` error on distributed dataflows
1 year ago
Philipp Oppermann
a7af34d004
Don't include uuid when printing DaemonId
1 year ago
Philipp Oppermann
b82b1059dc
Remove empty module
1 year ago
Philipp Oppermann
187efed38e
Merge branch 'main' into git-source
1 year ago
Philipp Oppermann
51ed0194c7
Rework and refactor for two-step build
1 year ago
sjfhsjfh
51190788ba
refactor: use builder style
fix: logging thread block
1 year ago
haixuantao
edb1766ffe
fix terminal
1 year ago
Philipp Oppermann
e897f78024
Add extra error cause when node fails to spawn
for better error messages
1 year ago
Philipp Oppermann
c9a720d7ae
Error if coordinator doesn't reply properly
Instead of sending `CoordinatorStopped`.
This should make it easier to debug when a reply is not sent as expected.
1 year ago
Philipp Oppermann
90733aabeb
Send build output to CLI
The coordinator now sends an immediate `DataflowStartTriggered` reply when receiving a `DataflowStart` command. This enables the CLI to directly attach to the dataflow and observe the build output.
To wait until the build/spawning is done, this commit introduces a new `WaitForSpawn` command, to which the coordinator replies with a `DataflowSpawned` message once the node has been started. We use this command for the `dora build` command.
1 year ago
Philipp Oppermann
3b00466e3b
Skip path check for operators with build commands
1 year ago
Philipp Oppermann
ead79013c8
Skip path checks for nodes with build command
1 year ago
Philipp Oppermann
bcb861932c
Make `dora build` command behave like `dora start` without spawning
Instead of running all the build commands directly, run them on their intended target machines through the coordinator.
This commit is a breaking change because a coordinator connection is now required for `dora build`.
1 year ago
Philipp Oppermann
49b87daeae
Handle dataflow spawn asynchronously in coordinator too
1 year ago
Philipp Oppermann
c0ac912fc2
Simplify UNC working directory paths on Windows
Fixes the tests/examples of this repo. A UNC path as working directory doesn't work because the build commands of our examples invoke `cargo` on the dora repo. The dora repo uses wildcards in the member list in the top-level Cargo.toml, which are ignored in UNC-paths. Thus, `failed to load manifest for workspace member` errors occur.
1 year ago
Philipp Oppermann
e31b2a3489
Remove needless cloning of node `env` field for building
1 year ago
Philipp Oppermann
6998964cda
Merge branch 'main' into git-source
1 year ago
Shar-jeel-Sajid
4647ee23c8
using context in error handling
1 year ago
Shar-jeel-Sajid
c655d1f103
formatting
1 year ago
Shar-jeel-Sajid
755c5b452a
Made conversion generic
1 year ago
haixuanTao
26c3917470
Fix GPU Memory not displayed
1 year ago
haixuanTao
48cb5ae709
Use current tokio threadpool and update otlp to latest version
1 year ago
Philipp Oppermann
e8d7bb02fb
Add support for git repo sources for nodes
1 year ago
Shar-jeel-Sajid
a88196e331
linting and renaming
1 year ago
Shar-jeel-Sajid
0bb576008f
cast_values for f64
1 year ago
Shar-jeel-Sajid
0e4bcddb24
Added f16 type conversion
1 year ago
Shar-jeel-Sajid
524eaa0dec
Very small typo
Co-authored-by: Haixuan Xavier Tao <tao.xavier@outlook.com>
1 year ago
Shar-jeel-Sajid
ee5426565b
Combining macros
1 year ago
Shar-jeel-Sajid
77532746ff
procedural macros in from_impls file
1 year ago
Shar-jeel-Sajid
f89c2638ef
procedural macros in into_impls file
1 year ago
Haroon Tahir
7ceffa6d5c
Changed String conversion in from to simplify the code and handle edge cases removed arrow-convert crate from cargo.toml as well since it is not being used
1 year ago
Haroon Tahir
7872ef2483
Removed conversion for &String since &str covers that use case
1 year ago
Shar-jeel-Sajid
a25be9df0b
Now support conversion for chrono::NaiveDateTime
1 year ago
Shar-jeel-Sajid
a2ce18ae2b
Now support Date32,Date64,Tim32second,Time32millisecond,Time64microsecond,Time64nanosecond
1 year ago
Shar-jeel-Sajid
39a7bd25f7
Resolved issues
1 year ago
Shar-jeel-Sajid
fbba2ecd93
Added Datatype NaiveDate and NaiveTime
1 year ago
Haroon Tahir
9530603944
fixed string conversion to use reference
1 year ago
Haroon Tahir
90ab326504
Added missing types in from_impl.rs that were previously present in to_impl.rs and added in mutable string type conversion in both
1 year ago
Shar-jeel-Sajid
fef99c4c4e
Adding String DataType
1 year ago
Shar-jeel-Sajid
1e585b0fc1
Added TestCases
1 year ago
Shar-jeel-Sajid
48c3bb5b4a
Removing Deprecation
1 year ago
Mati-ur-rehman-017
3332c0a1ce
Fixed D100 issues
1 year ago