Jamie Nisbet
ff5f2521fc
remove unused variable
6 years ago
fary86
3f323e48e2
Add submodule id for log interface
6 years ago
xulei2020
5b89fa78b0
fix thread hang bug
6 years ago
Jamie Nisbet
7b7b2fa7a0
remove column name id map from data buffer
clang format updates
self-review cleanups
cpplint fix
review updates
ci fixes
6 years ago
jonyguo
cf352d190f
format func name for mindrecord
6 years ago
mindspore-ci-bot
2e80be73b4
!978 [Data]BUG replace Post() after LaunchWorkers
Merge pull request !978 from xulei/filter_op
6 years ago
mindspore-ci-bot
6c79c00a3f
!979 dataset: repair parameter check in rename Op
Merge pull request !979 from ms_yan/rename_columns
6 years ago
mindspore-ci-bot
078dd86cfe
!507 Implemented padded_batch
Merge pull request !507 from ZiruiWu/batch_with_padding
6 years ago
Zirui Wu
c2d364a573
batch with padding implemented
support for 1 specific dimension to be None, added validator
fix various CI complains
another round of CI fixes
ci
refactor parts of the code
code refactor
ci fix
comments added, fix bugs
address review comments
address review comments
review cmts
added simple perf test script
update pad code
perf imprv
6 years ago
ms_yan
d95a2e9ccf
repair num of input columns and output columns check
6 years ago
jonyguo
d4d236bcce
fix: use MindDataset by column_names get data error in some situation
6 years ago
xulei2020
e6e0d62775
replace FindMe after LaunchWorker
6 years ago
panfengfeng
d8a2847bbe
after error run the post
6 years ago
Nat Sutyanyong
f19f394bb8
Refactor duplicate code on random device and seed
6 years ago
ms_yan
c56fe3aa2d
modify take op with an operator
6 years ago
caifubi
cb70daa355
format code
6 years ago
mindspore-ci-bot
2303453753
!869 Random data op
Merge pull request !869 from JesseKLee/random_data_op
6 years ago
Jesse Lee
270bf831a9
Random Data Op
6 years ago
mindspore-ci-bot
323ea9d869
!830 Update the print methods for displaying execution tree info diagnostics
Merge pull request !830 from Jamie/operator_print
6 years ago
mindspore-ci-bot
98939d839c
!856 Fix skip op bug
Merge pull request !856 from jiangzhiwen/dataset/skip_thread
6 years ago
jiangzhiwen
34bfa2f7c9
fix skip
6 years ago
Jamie Nisbet
2facd80c8d
updated print functions for info diagnostics
fix some ci failures
more ci fixes
more ci fails
yet another ci fail
more ci stuff
still ci
more ci
more ci
more ci
6 years ago
Jesse Lee
332398d20a
Check return code from WaitPost::Register
6 years ago
mindspore-ci-bot
3ec19ca9d3
!793 More understandable error msg when number of rows is zero
Merge pull request !793 from yanghaitao/yht_err_msg_num_rows_zero
6 years ago
yanghaitao
eb60ef00d1
More understandable err msg when num rows is zero
6 years ago
mindspore-ci-bot
f82e63fecc
!671 Added testcase for sync_wait
Merge pull request !671 from EricZ/master
6 years ago
mindspore-ci-bot
c47ef8ee4e
!711 Address first epoch always shuffle with same seed init of random sampler
Merge pull request !711 from ZiruiWu/mindspore_master_only
6 years ago
mindspore-ci-bot
e6473f0765
!736 Skip blank line for TextFileDataset
Merge pull request !736 from yanghaitao/yht_tftextfile_black_line
6 years ago
yanghaitao
1cb3bead32
skip black line for TextFileDataset
6 years ago
Zirui Wu
8c3931cf1d
fix first epoch always shuffle with default seed in random sampler
fix ut
6 years ago
zhoufeng
b681cec8f2
cmake refactor
6 years ago
eric
2d115cd04e
Added example for multiple iterator
Added new testcase for multi iterator
Addressing review
Fixed typo
6 years ago
xiefangqi
abd4239da9
fix random stuck problem
6 years ago
xulei2020
b5b506c4c3
add code
6 years ago
eric
cd94518769
X# This is a combination of 2 commits.
Initial commit for dataset op python
Added signature to barrier
Adde compiling barrier code
Rebasing, fixed new compile errors
Final fix for make_unique
Added pybind API for barrier
Fixed pyfunc invocation
python interface - sync_wait
!1 sync_wait python interface
* python interface - sync_wait
fix test
update test
update test
Added new test case
add test case
test for shuffle + batch
Added two-sync test case
Restrited that no shuffle after sync
Added sync to pipeline info
block first databuffer as well
Intelligently get batch size
Fix default case
Lock Pair shares among all iterators
Added fix for empty character
Fixed up test case formatting
Fix end of epoch in sync_wait
Fixing CI
6 years ago
qianlong
db80f4ff92
The num_samples and numRows in schema for TFRecordDataset are conflict
6 years ago
mindspore-ci-bot
d9e4dcc33b
!483 Optimize skip dataset op
Merge pull request !483 from jiangzhiwen/dataset/skip_opt
6 years ago
mindspore-ci-bot
d307c264b9
!517 [Data]Delete sort logic for filter operation
Merge pull request !517 from xulei/filter_master
6 years ago
xulei2020
eb053a6233
add filterOp code
6 years ago
liyong
f1542a90a3
add pk sampler
6 years ago
jiangzhiwen
e1b109e8b8
optimize skip dataset op
6 years ago
mindspore-ci-bot
6369cf27bd
!406 added first row crc check for when reading tfrecord files
Merge pull request !406 from Peilin/first-row-crc-check
6 years ago
mindspore-ci-bot
98fbd30a5b
!460 [Data]Add filter operation
Merge pull request !460 from xulei/filter_master
6 years ago
mindspore-ci-bot
822a3160e4
!404 [Dataset] Add Python Sampler support for CPP dataset
Merge pull request !404 from JunhanHu/python_sampler_for_cpp_dataset
6 years ago
xulei2020
c705ea5e5b
add filterOp code
6 years ago
Peilin Wang
9bc2134cb7
added checking of first row crc to find invalid tfrecord files
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file
fixed clang format
fixed cppcheck
fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before
fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments
more clang whitespace fixes...
changed print to logger.info
6 years ago
mindspore-ci-bot
d8176a77f4
!425 New api "TextFileDataset"
Merge pull request !425 from yanghaitao/yht_textfiledatasetv2
6 years ago
ms_yan
c5cfb09e66
Repair some MS_LOG problem
6 years ago
yanghaitao
2795e492ff
TextFileDataset
6 years ago
Junhan Hu
d0ae610832
Review
6 years ago