hummingbird
/
fastNLP
Not watched
2
0
Fork 0
Code
Releases 13 Wiki Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
2110 Commits
3 Branches
25 MB
221 Download
Tree: 3f00642ba1
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '3f00642ba1'
${ noResults }
fastNLP/tests/core/dataloaders/test_prepare_dataloader.py

test_prepare_dataloader.py 355 B
Raw Normal View History

1.修改部分注释; 2.增加prepare_dataloader函数;3.paddle的paddler修改为先集中使用numpy pad再一把to_tensor
3 years ago
​
​
​
​
​
​
​
​
​
​
​
​
 12345678910111213 
  1. import pytest

  2. 

  3. from fastNLP import prepare_dataloader

  4. from fastNLP import DataSet

  5. 

  6. 

  7. @pytest.mark.torch

  8. def test_torch():

  9.     import torch

  10.     ds = DataSet({"x": [[1, 2], [2, 3, 4], [4, 5, 6, 7]] * 10, "y": [1, 0, 1] * 10})

  11.     dl = prepare_dataloader(ds, batch_size=2, shuffle=True)

  12.     for batch in dl:

  13.         assert isinstance(batch['x'], torch.Tensor)

一款轻量级的自然语言处理（NLP）工具包，目标是减少用户项目中的工程型代码，例如数据处理循环、训练循环、多卡运行等