Browse Source

Improve `README` and create 2 dataflow, one for dev and one for full remote version

tags/v0.3.9-rc1
haixuantao 1 year ago
parent
commit
d3ac0a3d33
6 changed files with 2024 additions and 43 deletions
  1. +23
    -2
      examples/speech-to-text/README.md
  2. +0
    -0
      examples/speech-to-text/whisper-dev.yml
  3. +33
    -0
      examples/speech-to-text/whisper.yml
  4. +0
    -0
      examples/vlm/qwenvl-dev.yml
  5. +1968
    -0
      examples/vlm/qwenvl.yml
  6. +0
    -41
      examples/vlm/vision_only.yml

+ 23
- 2
examples/speech-to-text/README.md View File

@@ -3,8 +3,29 @@
Make sure to have, dora, pip and cargo installed.

```bash
dora build dataflow.yml
dora run dataflow.yml
dora build whisper.yml
dora run whisper.yml

# Wait for the whisper model to download which can takes a bit of time.
```

## Graph Visualization

```mermaid

flowchart TB
dora-microphone
dora-vad
dora-distil-whisper
dora-rerun[/dora-rerun\]
subgraph ___dora___ [dora]
subgraph ___timer_timer___ [timer]
dora/timer/secs/2[\secs/2/]
end
end
dora/timer/secs/2 -- tick --> dora-microphone
dora-microphone -- audio --> dora-vad
dora-vad -- audio as input --> dora-distil-whisper
dora-distil-whisper -- text as original_text --> dora-rerun

```

examples/speech-to-text/dataflow.yml → examples/speech-to-text/whisper-dev.yml View File


+ 33
- 0
examples/speech-to-text/whisper.yml View File

@@ -0,0 +1,33 @@
nodes:
- id: dora-microphone
description: Microphone
build: pip install dora-microphone
path: dora-microphone
inputs:
tick: dora/timer/millis/2000
outputs:
- audio

- id: dora-vad
build: pip install dora-vad
path: dora-vad
inputs:
audio: dora-microphone/audio
outputs:
- audio

- id: dora-whisper
build: pip install dora-whisper
path: dora-whisper
inputs:
input: dora-vad/audio
outputs:
- text
env:
TARGET_LANGUAGE: english

- id: dora-rerun
build: pip install dora-rerun
path: dora-rerun
inputs:
original_text: dora-whisper/text

examples/vlm/dataflow.yml → examples/vlm/qwenvl-dev.yml View File


+ 1968
- 0
examples/vlm/qwenvl.yml
File diff suppressed because it is too large
View File


+ 0
- 41
examples/vlm/vision_only.yml View File

@@ -1,41 +0,0 @@
nodes:
- id: camera
build: pip install -e ../../node-hub/opencv-video-capture
path: opencv-video-capture
inputs:
tick: dora/timer/millis/50
outputs:
- image
env:
CAPTURE_PATH: 0
IMAGE_WIDTH: 640
IMAGE_HEIGHT: 480

- id: dora-qwenvl
build: pip install -e ../../node-hub/dora-qwenvl
path: dora-qwenvl
inputs:
image:
source: camera/image
queue_size: 1
tick: dora/timer/millis/400
outputs:
- text
- tick
env:
DEFAULT_QUESTION: Describe the image in a very short sentence.
# USE_MODELSCOPE_HUB: true

- id: plot
build: cargo build -p dora-rerun --release
path: dora-rerun
inputs:
image:
source: camera/image
queue_size: 1
text: dora-qwenvl/tick
env:
IMAGE_WIDTH: 640
IMAGE_HEIGHT: 480
README: |
# Visualization of QwenVL2

Loading…
Cancel
Save