mindspore-ci-bot
f82e63fecc
!671 Added testcase for sync_wait
Merge pull request !671 from EricZ/master
5 years ago
mindspore-ci-bot
c47ef8ee4e
!711 Address first epoch always shuffle with same seed init of random sampler
Merge pull request !711 from ZiruiWu/mindspore_master_only
5 years ago
mindspore-ci-bot
e6473f0765
!736 Skip blank line for TextFileDataset
Merge pull request !736 from yanghaitao/yht_tftextfile_black_line
5 years ago
yanghaitao
1cb3bead32
skip black line for TextFileDataset
5 years ago
Zirui Wu
8c3931cf1d
fix first epoch always shuffle with default seed in random sampler
fix ut
5 years ago
zhoufeng
b681cec8f2
cmake refactor
5 years ago
eric
2d115cd04e
Added example for multiple iterator
Added new testcase for multi iterator
Addressing review
Fixed typo
5 years ago
xiefangqi
abd4239da9
fix random stuck problem
5 years ago
xulei2020
b5b506c4c3
add code
5 years ago
eric
cd94518769
X# This is a combination of 2 commits.
Initial commit for dataset op python
Added signature to barrier
Adde compiling barrier code
Rebasing, fixed new compile errors
Final fix for make_unique
Added pybind API for barrier
Fixed pyfunc invocation
python interface - sync_wait
!1 sync_wait python interface
* python interface - sync_wait
fix test
update test
update test
Added new test case
add test case
test for shuffle + batch
Added two-sync test case
Restrited that no shuffle after sync
Added sync to pipeline info
block first databuffer as well
Intelligently get batch size
Fix default case
Lock Pair shares among all iterators
Added fix for empty character
Fixed up test case formatting
Fix end of epoch in sync_wait
Fixing CI
6 years ago
qianlong
db80f4ff92
The num_samples and numRows in schema for TFRecordDataset are conflict
6 years ago
mindspore-ci-bot
d9e4dcc33b
!483 Optimize skip dataset op
Merge pull request !483 from jiangzhiwen/dataset/skip_opt
6 years ago
mindspore-ci-bot
d307c264b9
!517 [Data]Delete sort logic for filter operation
Merge pull request !517 from xulei/filter_master
6 years ago
xulei2020
eb053a6233
add filterOp code
6 years ago
liyong
f1542a90a3
add pk sampler
6 years ago
jiangzhiwen
e1b109e8b8
optimize skip dataset op
6 years ago
mindspore-ci-bot
6369cf27bd
!406 added first row crc check for when reading tfrecord files
Merge pull request !406 from Peilin/first-row-crc-check
6 years ago
mindspore-ci-bot
98fbd30a5b
!460 [Data]Add filter operation
Merge pull request !460 from xulei/filter_master
6 years ago
mindspore-ci-bot
822a3160e4
!404 [Dataset] Add Python Sampler support for CPP dataset
Merge pull request !404 from JunhanHu/python_sampler_for_cpp_dataset
6 years ago
xulei2020
c705ea5e5b
add filterOp code
6 years ago
Peilin Wang
9bc2134cb7
added checking of first row crc to find invalid tfrecord files
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file
fixed clang format
fixed cppcheck
fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before
fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments
more clang whitespace fixes...
changed print to logger.info
6 years ago
mindspore-ci-bot
d8176a77f4
!425 New api "TextFileDataset"
Merge pull request !425 from yanghaitao/yht_textfiledatasetv2
6 years ago
ms_yan
c5cfb09e66
Repair some MS_LOG problem
6 years ago
yanghaitao
2795e492ff
TextFileDataset
6 years ago
Junhan Hu
d0ae610832
Review
6 years ago
Junhan Hu
43a2e99833
Add python sampler support for CPP dataset
6 years ago
ms_yan
f0c07c3fa6
Realize take op and add ut
6 years ago
xiefangqi
d4822137f3
fix segment bug on windows
6 years ago
jzw
3f7054dccb
add skip dataset op
6 years ago
mindspore-ci-bot
cf026096a6
!183 Mindspore.dataset CPP sampler for GeneratorDataset
Merge pull request !183 from JunhanHu/cpp_sampler
6 years ago
mindspore-ci-bot
b2b3e24a8e
!329 [MS]support building on windows 10
Merge pull request !329 from chenjianping/building-on-windows-2
6 years ago
chenjianping
1286767d0e
support building on windows
6 years ago
Junhan Hu
9739d3b048
Add CPP sampler support for GeneratorDataset
6 years ago
mindspore-ci-bot
30de261c3c
!243 Support nested repeat
Merge pull request !243 from h.farahat/nested_repeat
6 years ago
hesham
0fc23eee0f
Support nested repeat
6 years ago
mindspore-ci-bot
d9cd681c30
!207 Fix potential overflow in batch by changing int32 to int64
Merge pull request !207 from ZiruiWu/batch_info_overflow
6 years ago
dengwentao
b5ca2a3e30
:modify protobuf cmake
6 years ago
Zirui Wu
19f2ea6d41
change int32 to int64 to avoid overflow in batch
fix clang
pybind fix
6 years ago
mindspore-ci-bot
cc75cb357c
!168 remove mindspore::make_unique and make_unique.h
Merge pull request !168 from Alexey_Shevlyakov/remove_mindspore_make_unique
6 years ago
jonyguo
20d1b64443
fix: error info is not exactly when column list invalid
6 years ago
Jonathan Yan
9d0fde29f4
remove ENABLE_MINDRECORD flag
6 years ago
Alexey Shevlyakov
84d780c1a4
remove make_unique.h
6 years ago
Peilin Wang
0ae77bb0db
TFReaderOp fix, threads will exit after reading necessary amount of rows
changes from yanpanhui 524009: added set_dataset_size and changed get_dataest_size according to ME requirements
CI fixes
6 years ago
zhunaipan
930a1fb0a8
initial version
Signed-off-by: leonwanghui <leon.wanghui@huawei.com>
6 years ago