Junhan Hu
1e904ddcee
Fix CI warning of samplers.py
5 years ago
mindspore-ci-bot
b37db1edf5
!603 [MD] update pk sampler in minddataset
Merge pull request !603 from liyong126/update_pk_sampler
5 years ago
mindspore-ci-bot
f82e63fecc
!671 Added testcase for sync_wait
Merge pull request !671 from EricZ/master
5 years ago
mindspore-ci-bot
0e3054d527
!466 Deepcopy problem when pyfunc cannot be pickled
Merge pull request !466 from h.farahat/deepcopy
5 years ago
liyong
bfba630aa2
update pK_sampler
5 years ago
mindspore-ci-bot
31a1f9dac1
!739 Check python_multiprocess type for map
Merge pull request !739 from yanghaitao/yht_map_python_multiprocessing_check
5 years ago
xiefangqi
3bacea7bba
fix gpu hung problem
5 years ago
yanghaitao
c87e174661
checkout python_multiprocess type for map
5 years ago
yanghaitao
60f2663326
Check TfFileDataset num_shards and shard_id
5 years ago
eric
2d115cd04e
Added example for multiple iterator
Added new testcase for multi iterator
Addressing review
Fixed typo
5 years ago
mindspore-ci-bot
78bbacdaac
!601 [Data]Fix the nonstandard comments (add space, add function type)
Merge pull request !601 from xulei/filter_master
5 years ago
mindspore-ci-bot
c65a57775b
!631 Fixed the bug on uniform augment input validation
Merge pull request !631 from AdelShafiei/uniform_augment_ut
5 years ago
Adel Shafiei
f29eacbb34
fixed an input validation error for uniform augment op
5 years ago
mindspore-ci-bot
1cc9f65815
!630 UniformAugment: Fixed empty list error
Merge pull request !630 from alashkari/ua-ops
5 years ago
Amir Lashkari
a43b60f99f
Fixed not in list error
5 years ago
mindspore-ci-bot
eefb6edde2
!606 Fix dataset api doc problem
Merge pull request !606 from pengyanjun/fix_dataset_api_doc_change_line
5 years ago
Yanjun Peng
adeed4fe67
fix dataset api doc
5 years ago
yanghaitao
6a6a3cf077
GeneratorDataset column_names support string type
5 years ago
xulei2020
5baee37978
add code
5 years ago
hesham
a9e9266149
Deepcopy problem when pyfunc cannot be pickled
6 years ago
ms_yan
763aa1067e
Add Error catch for giving too many input parameters
5 years ago
mindspore-ci-bot
eda63a559a
!477 Fix VOC dataset test cases
Merge pull request !477 from xiefangqi/xfq_fix_voc
5 years ago
mindspore-ci-bot
aad5771a62
!524 Added support for UA augmentation ops with tests
Merge pull request !524 from alashkari/ua-ops
5 years ago
Amir Lashkari
56e7a7deb5
Added UniformAugment + Python Augmentation Ops
5 years ago
mindspore-ci-bot
dc0491caf9
!508 [Dataset] Adding sync_wait operator for dataset
Merge pull request !508 from EricZ/master
5 years ago
mindspore-ci-bot
b0f4b36f48
!511 Uniform Augmentation C++ OP
Merge pull request !511 from AdelShafiei/uniform_augmentation
5 years ago
eric
cd94518769
X# This is a combination of 2 commits.
Initial commit for dataset op python
Added signature to barrier
Adde compiling barrier code
Rebasing, fixed new compile errors
Final fix for make_unique
Added pybind API for barrier
Fixed pyfunc invocation
python interface - sync_wait
!1 sync_wait python interface
* python interface - sync_wait
fix test
update test
update test
Added new test case
add test case
test for shuffle + batch
Added two-sync test case
Restrited that no shuffle after sync
Added sync to pipeline info
block first databuffer as well
Intelligently get batch size
Fix default case
Lock Pair shares among all iterators
Added fix for empty character
Fixed up test case formatting
Fix end of epoch in sync_wait
Fixing CI
6 years ago
Adel Shafiei
5fcd3f01a6
Added C++ UniformAugOp support
6 years ago
Junhan Hu
78001ac9e6
Add multiprocessing support for Mindspore.Dataset.GeneratorDataset
6 years ago
mindspore-ci-bot
fb18671b28
!506 [Dataset] Multiprocessing support for Pyfunc
Merge pull request !506 from JunhanHu/multiprocess_pyfunc
5 years ago
Junhan Hu
b13e7bc31a
Add python multiprocessing support for Mindspore.dataset
6 years ago
xiefangqi
108eeb8e3d
fix voc test cases
6 years ago
qianlong
db80f4ff92
The num_samples and numRows in schema for TFRecordDataset are conflict
6 years ago
liyong
f1542a90a3
add pk sampler
6 years ago
Cathy Wong
60df369100
Fixup py Normalize doc: takes input CHW
6 years ago
mindspore-ci-bot
6369cf27bd
!406 added first row crc check for when reading tfrecord files
Merge pull request !406 from Peilin/first-row-crc-check
6 years ago
mindspore-ci-bot
98fbd30a5b
!460 [Data]Add filter operation
Merge pull request !460 from xulei/filter_master
6 years ago
mindspore-ci-bot
822a3160e4
!404 [Dataset] Add Python Sampler support for CPP dataset
Merge pull request !404 from JunhanHu/python_sampler_for_cpp_dataset
6 years ago
xulei2020
c705ea5e5b
add filterOp code
6 years ago
Peilin Wang
9bc2134cb7
added checking of first row crc to find invalid tfrecord files
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file
fixed clang format
fixed cppcheck
fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before
fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments
more clang whitespace fixes...
changed print to logger.info
6 years ago
yanghaitao
2795e492ff
TextFileDataset
6 years ago
Junhan Hu
43a2e99833
Add python sampler support for CPP dataset
6 years ago
mindspore-ci-bot
3ad73b7d71
!463 !330 dataset: add take operation
Merge pull request !463 from ms_yan/take_op_merge
6 years ago
ms_yan
f0c07c3fa6
Realize take op and add ut
6 years ago
caojian05
852039b726
fix dataset deepcopy failed while depipline referenced in the iterator
6 years ago
mindspore-ci-bot
6c59093fe8
!427 Fix dataset api description
Merge pull request !427 from pengyanjun/fix_dataset_api_description
6 years ago
Yanjun Peng
c69f3be758
fix dataset api description
6 years ago
mindspore-ci-bot
80333e9f55
!435 Fix dataset serialize and deserialize for MindDataset
Merge pull request !435 from anthonyaje/fix_serdes_dataset
6 years ago
mindspore-ci-bot
40f0a4a4f4
!333 Add skip op to Dataset
Merge pull request !333 from jiangzhiwen/dataset/skip
6 years ago
mindspore-ci-bot
9e1b5efd1d
!434 Bug in cleaning dataset iterators
Merge pull request !434 from h.farahat/multi_itr_bug
6 years ago