mindspore-ci-bot
b37db1edf5
!603 [MD] update pk sampler in minddataset
Merge pull request !603 from liyong126/update_pk_sampler
5 years ago
mindspore-ci-bot
f82e63fecc
!671 Added testcase for sync_wait
Merge pull request !671 from EricZ/master
5 years ago
liyong
bfba630aa2
update pK_sampler
5 years ago
mindspore-ci-bot
c47ef8ee4e
!711 Address first epoch always shuffle with same seed init of random sampler
Merge pull request !711 from ZiruiWu/mindspore_master_only
5 years ago
mindspore-ci-bot
e6473f0765
!736 Skip blank line for TextFileDataset
Merge pull request !736 from yanghaitao/yht_tftextfile_black_line
5 years ago
yanghaitao
1cb3bead32
skip black line for TextFileDataset
5 years ago
Zirui Wu
8c3931cf1d
fix first epoch always shuffle with default seed in random sampler
fix ut
5 years ago
zhoufeng
b681cec8f2
cmake refactor
5 years ago
eric
2d115cd04e
Added example for multiple iterator
Added new testcase for multi iterator
Addressing review
Fixed typo
5 years ago
Adel Shafiei
f29eacbb34
fixed an input validation error for uniform augment op
5 years ago
mindspore-ci-bot
fe67db75a3
!580 fix dataset & mindrecord log print
Merge pull request !580 from panfengfeng/panff/fix_log_print_in_ge
5 years ago
Alexey Shevlyakov
6acae622dc
fix random seed behaviour
6 years ago
xiefangqi
abd4239da9
fix random stuck problem
5 years ago
panfengfeng
6328edf455
modify dataset & mindrecord log compile
5 years ago
xulei2020
b5b506c4c3
add code
5 years ago
mindspore-ci-bot
dc0491caf9
!508 [Dataset] Adding sync_wait operator for dataset
Merge pull request !508 from EricZ/master
5 years ago
mindspore-ci-bot
b0f4b36f48
!511 Uniform Augmentation C++ OP
Merge pull request !511 from AdelShafiei/uniform_augmentation
5 years ago
eric
cd94518769
X# This is a combination of 2 commits.
Initial commit for dataset op python
Added signature to barrier
Adde compiling barrier code
Rebasing, fixed new compile errors
Final fix for make_unique
Added pybind API for barrier
Fixed pyfunc invocation
python interface - sync_wait
!1 sync_wait python interface
* python interface - sync_wait
fix test
update test
update test
Added new test case
add test case
test for shuffle + batch
Added two-sync test case
Restrited that no shuffle after sync
Added sync to pipeline info
block first databuffer as well
Intelligently get batch size
Fix default case
Lock Pair shares among all iterators
Added fix for empty character
Fixed up test case formatting
Fix end of epoch in sync_wait
Fixing CI
6 years ago
Adel Shafiei
5fcd3f01a6
Added C++ UniformAugOp support
6 years ago
mindspore-ci-bot
3c307cf486
!552 Update clang format rule
Merge pull request !552 from ZhouFeng/clang-rule-update
5 years ago
zhoufeng
c2b3360d69
update clang format rule
5 years ago
mindspore-ci-bot
6e6b3eeab0
!547 delete unsafe option
Merge pull request !547 from xiefangqi/del_unsafe_option
5 years ago
xiefangqi
a931fbbae2
delete unsafe compile option
6 years ago
qianlong
db80f4ff92
The num_samples and numRows in schema for TFRecordDataset are conflict
6 years ago
mindspore-ci-bot
d9e4dcc33b
!483 Optimize skip dataset op
Merge pull request !483 from jiangzhiwen/dataset/skip_opt
6 years ago
mindspore-ci-bot
d307c264b9
!517 [Data]Delete sort logic for filter operation
Merge pull request !517 from xulei/filter_master
6 years ago
xulei2020
eb053a6233
add filterOp code
6 years ago
liyong
f1542a90a3
add pk sampler
6 years ago
jiangzhiwen
e1b109e8b8
optimize skip dataset op
6 years ago
mindspore-ci-bot
6369cf27bd
!406 added first row crc check for when reading tfrecord files
Merge pull request !406 from Peilin/first-row-crc-check
6 years ago
mindspore-ci-bot
98fbd30a5b
!460 [Data]Add filter operation
Merge pull request !460 from xulei/filter_master
6 years ago
mindspore-ci-bot
822a3160e4
!404 [Dataset] Add Python Sampler support for CPP dataset
Merge pull request !404 from JunhanHu/python_sampler_for_cpp_dataset
6 years ago
xulei2020
c705ea5e5b
add filterOp code
6 years ago
Peilin Wang
9bc2134cb7
added checking of first row crc to find invalid tfrecord files
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file
fixed clang format
fixed cppcheck
fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before
fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments
more clang whitespace fixes...
changed print to logger.info
6 years ago
mindspore-ci-bot
d8176a77f4
!425 New api "TextFileDataset"
Merge pull request !425 from yanghaitao/yht_textfiledatasetv2
6 years ago
ms_yan
c5cfb09e66
Repair some MS_LOG problem
6 years ago
yanghaitao
2795e492ff
TextFileDataset
6 years ago
Junhan Hu
d0ae610832
Review
6 years ago
Junhan Hu
43a2e99833
Add python sampler support for CPP dataset
6 years ago
ms_yan
f0c07c3fa6
Realize take op and add ut
6 years ago
xiefangqi
d4822137f3
fix segment bug on windows
6 years ago
jzw
3f7054dccb
add skip dataset op
6 years ago
mindspore-ci-bot
c0c0b0985e
!378 Multiple Iterators can cause performance problem
Merge pull request !378 from h.farahat/multi_itr
6 years ago
hesham
fb6c7ba2e1
Fix two problem when we create multiple instances of the same dataset (2 for-loops)
-- Iterator list is keeping all created iterators wihtout cleaning them up
-- alter tree modifies the original.
6 years ago
mindspore-ci-bot
7874e96b9b
!377 Tensor end() iterator performance problem
Merge pull request !377 from h.farahat/end_iterator
6 years ago
hesham
c91a58dc55
end() iterator is causing performance problem
6 years ago
mindspore-ci-bot
cf026096a6
!183 Mindspore.dataset CPP sampler for GeneratorDataset
Merge pull request !183 from JunhanHu/cpp_sampler
6 years ago
mindspore-ci-bot
b2b3e24a8e
!329 [MS]support building on windows 10
Merge pull request !329 from chenjianping/building-on-windows-2
6 years ago
chenjianping
1286767d0e
support building on windows
6 years ago
Junhan Hu
9739d3b048
Add CPP sampler support for GeneratorDataset
6 years ago