beimingwu
/
learnware
Not watched
1
0
Fork 0
Code
Releases 1 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
1114 Commits
1 Branch
20 MB
62 Download
Tree: e33db2e8fd
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'e33db2e8fd'
${ noResults }
learnware/examples/dataset_text_workflow/get_data.py

get_data.py 401 B
Raw Normal View History

[ENH] Add text specification and test
2 years ago
​
[MNT] format code by black
2 years ago
[ENH] Add text specification and test
2 years ago
[MNT] format code by black
2 years ago
​
[ENH] Add text specification and test
2 years ago
​
​
[MNT] format code by black
2 years ago
[ENH] Add text specification and test
2 years ago
​
​
​
 1234567891011121314 
  1. from torchtext.datasets import SST2

  2. 

  3. 

  4. def get_sst2(data_root="./data"):

  5.     train_datapipe = SST2(root="./data", split="train")

  6. 

  7.     X_train = [x[0] for x in train_datapipe]

  8.     y_train = [x[1] for x in train_datapipe]

  9. 

  10.     dev_datapipe = SST2(root="./data", split="dev")

  11. 

  12.     X_test = [x[0] for x in dev_datapipe]

  13.     y_test = [x[1] for x in dev_datapipe]

  14.     return X_train, y_train, X_test, y_test

基于学件范式，全流程地支持学件上传、检测、组织、查搜、部署和复用等功能。同时，该仓库作为北冥坞系统的引擎，支撑北冥坞系统的核心功能。