anzhengqi
dd942e1807
fix memory ascend while multi model.eval
5 years ago
heleiwang
4870abc848
1. fix generator_dataset hangs
2. fix test_graphdata_distributed.py failing randomly
5 years ago
jonyguo
5b4b539751
fix: concat with none sample dataset
5 years ago
Zirui Wu
78c1aa1d96
Implemented Callback for Dataset
implment pause in MapOp, added more to callback
add ds_callback
- Initial drop of Python DSCallback
- Pybind DSCallback
- Pybind DSCallback
added callback to mapOp
- de_pipeline DSCallback
- de_pipeline DSCallback
add test case, segfault for now
fix seg fault
- de_pipeline DSCallback
remove 1 line
update callback test case, now works
use builder class for mapOp callback
- de_pipeline DSCallback
- de_pipeline DSCallback
- de_pipeline DSCallback
better test case
minor fix
add comments and minor clean ups
get rid of nullptr in MapOp, use other flag instead
fix a bug ParseMapOp only takes 1 callback
- Added WaitedDSCalabck
refactor callback param
fix text case incorrect number
- added testing
fix cpp test case
- added testing
- revert back lenet changes
- cleanup test_callbacks.py
- cleanup test_callbacks.py
fix CI stage I
fix CI stage II
fix CI and update epoch counter
- add validation
- add more testing test_callbacks.py
use random data op to do tests
adjust when to call EpochBegin/End
- add repeat with callback
- addressing reviewers' comments
- docstring and CI fixes
- docstring and CI fixes
- docstring and CI fixes
- rebase with upstream/master
fix cpp test case
fix review comments
addr review cmts, add test case
5 years ago
mindspore-ci-bot
256dccc651
!4498 Gnn data processing supports distributed scenarios
Merge pull request !4498 from heleiwang/gnn_distributed
5 years ago
heleiwang
8ee4d8e92d
Gnn data processing supports distributed scenarios
5 years ago
jonyguo
b0b370336f
fix: concat get_dataset_size error
5 years ago
mindspore-ci-bot
8ed3b423e0
!4389 fix document api error
Merge pull request !4389 from yanghaitao/yht_fix_doc2
5 years ago
mindspore-ci-bot
ae8440dfec
!4421 dataset: modify some error in api comment
Merge pull request !4421 from ms_yan/api_comment
5 years ago
ms_yan
501f549bc9
modify comment for api
5 years ago
mindspore-ci-bot
7a10e07fc6
!4383 Add distributedSampler to concatOP
Merge pull request !4383 from genglishuai/genglshOK
5 years ago
genglishuai
4d4df68d1e
add DistributedSampler for Concat op
5 years ago
mindspore-ci-bot
ed1244558c
!4366 Fixing param type in c++ api and fixing python validator for Repeat Op
Merge pull request !4366 from TinaMengtingZhang/cpp-api-repeat-count
5 years ago
yanghaitao
46ae915cb3
fix doc error
5 years ago
tinazhang
0868720eb7
fix parameter type for repeat op in c++ api and added c++/python ut.
5 years ago
anzhengqi
3e31ac6d62
all Dataset support get_dataset_size
5 years ago
Cathy Wong
7f39b5cfd7
C++ API Support for TextFile Dataset and Unit Tests
5 years ago
yanghaitao
9401e7d96e
fix doc
5 years ago
dengyutao
388e43f7cb
delete parameter block_reader for MindDataset
5 years ago
xiefangqi
6017b050ac
add generatordataset support split
5 years ago
mindspore-ci-bot
387dac5832
!3651 change num_samples definition
Merge pull request !3651 from jiangzhiwen/dataset/change_num_samples
5 years ago
mindspore-ci-bot
fe514bd1cc
!3644 [MD] fix minddataset core dump when file list size ia greater than 1000.
Merge pull request !3644 from liyong126/fix_mindrecord_bug
5 years ago
jiangzhiwen
1eda0ef071
change num_samples definition
5 years ago
mindspore-ci-bot
9ccc6889eb
!3624 fix GeneratorDataset time out
Merge pull request !3624 from yanghaitao/yht_generator_timeout
5 years ago
liyong
ed70de8070
fix coredump when number of file list more than 1000.
5 years ago
yanghaitao
5c9b0657df
fix GeneratorDataset time out
5 years ago
xiefangqi
cb9c6fad86
fix numpyslice issue
5 years ago
mindspore-ci-bot
0975df3093
!3540 [MD]fix error message when file exists.
Merge pull request !3540 from liyong126/fix_mindrecord_log
5 years ago
mindspore-ci-bot
eeb8d72ac9
!3532 fix python api doc for mindspore .dataset
Merge pull request !3532 from guansongsong/gss_fix_api
5 years ago
guansongsong
1906ed1be8
fix cache description again
5 years ago
guansongsong
b6529b9096
fix cache description
5 years ago
liyong
7dd8c0e31c
fix mindrecord log
5 years ago
guansongsong
e377ffcebc
fix python api doc for mindspore.dataset second
5 years ago
guansongsong
0bf6ae913a
fix python api doc for mindspore.dataset
5 years ago
panfengfeng
939e612906
fix get daataset size error
5 years ago
mindspore-ci-bot
d874150fb3
!3334 dataset: add param check for device_que and to_device
Merge pull request !3334 from ms_yan/device_que_param
5 years ago
mindspore-ci-bot
7b54fd8304
!3318 [MD] add pydoc for save ops
Merge pull request !3318 from liyong126/fix_save_doc
5 years ago
mindspore-ci-bot
21a5f06e93
!3328 Add testcases
Merge pull request !3328 from shenwei41/sw_master
5 years ago
ms_yan
27190cf2fc
add param check for to_device and device_que
5 years ago
liyong
43e5db3d42
fix save op pydoc
5 years ago
Cathy Wong
8921a6099c
C++ API Support for Skip Dataset Op and UTs
5 years ago
shenwei41
4eaa396ca7
Add testcases
5 years ago
mindspore-ci-bot
0e27dccbcf
!3093 VOCDataset output change to multi-columns
Merge pull request !3093 from xiefangqi/md_voc_multi_columns
5 years ago
mindspore-ci-bot
3f7488f3e5
!3241 fix some log error of dataset
Merge pull request !3241 from panfengfeng/fix_some_log_error
5 years ago
panfengfeng
cbb159de08
fix some log errors
5 years ago
jiangzhiwen
2f506b7985
CSV dataset loader
5 years ago
xulei2020
18b519ae0f
add sentence piece
5 years ago
xiefangqi
fbd9e3d402
vocdataset output multi-columns
5 years ago
anzhengqi
008b91b2a1
inject epoch ctrl op in the execution tree and send eos at the end of epoch
5 years ago
liyong
bc676fe250
save op in minddataset
5 years ago