heleiwang
4f946bc54b
Modify the processing method of multiple processes in the GeneratorDataset:
1. Start the child process in the init phase
2. At the beginning of each epoch, the child process is not recreated, but the child process created at the beginning is used
5 years ago
jonyguo
f258697901
add per_batch_map usage in api comment
5 years ago
anzhengqi
5cf079595e
fix sink_size error
5 years ago
luoyang
2dc8e5f421
[MD] C++ api add MindDataset
5 years ago
mindspore-ci-bot
6f77ec45f1
!6905 [MD] Enable cache for more leaf datasets
Merge pull request !6905 from lixiachen/CacheOp_dev
5 years ago
xiefangqi
82bc278e26
minddata use new ps api
5 years ago
Lixia Chen
477808e622
Enable cache for other leaf ops
5 years ago
YangLuo
977c41db01
Fix timeout of GeneratorDataset multiprocessing
5 years ago
mindspore-ci-bot
4c0b3c1bd3
!6017 per_batch_map needs to support x input columns and y output columns
Merge pull request !6017 from ZiruiWu/per_batch_map_multi_col
5 years ago
Zirui Wu
0cab373223
per_batch_map supports x num_columns to y num_columns
decided to use project after batch as well
per_batch_map 1->x colunm works
add more test case
add test cases, project after map
col order hahaah
fix ci
add test cases
addr review cmts
ci
fix build err
fix test case failure
ci
5 years ago
mindspore-ci-bot
2f3add4acd
!6301 [MD] Combine c++ and python ops in map
Merge pull request !6301 from nhussain/c_py_compose
5 years ago
nhussain
fda9462682
embed python compose op
Leaf ops do self-reset.
embed python compose op
add implicit compose op
more tests
5 years ago
YangLuo
2fa3e2ea2f
Add comment for "relax timeout constraint" & "try to call terminate"
5 years ago
YangLuo
291cf767ee
relax time constraint for GeneratorWorker
5 years ago
liyong
0e40596bce
fix concat log
5 years ago
mindspore-ci-bot
e2fd5704d6
!6588 check if pyfunc timeout valid
Merge pull request !6588 from yanghaitao/yht_protect_pyfunc_timeout
5 years ago
yanghaitao1
8b4591c482
protect pyfunc timeout
5 years ago
mindspore-ci-bot
ea9870bf05
!6576 TFRecordDataset add notes
Merge pull request !6576 from heleiwang/mod_tfrecord_api
5 years ago
Eric
4e2d530adc
Added timeout for sync_wait
5 years ago
heleiwang
13ed9f99b3
TFRecordDataset add notes
5 years ago
mindspore-ci-bot
53a82fa6e0
!6472 fix get_dataset_size error in CelebaDataset when usage is not 'all'
Merge pull request !6472 from yanghaitao/yht_celeba_get_dataset_size
5 years ago
yanghaitao1
4ff4c17632
fix get_dataset_size in CelebADataset when usage is not all
5 years ago
xiefangqi
a6360cb2e4
add output_numpy validation to iterator
5 years ago
mindspore-ci-bot
1cfad93704
!6479 fix bucket_batch_by_length show pyfunc timeout
Merge pull request !6479 from yanghaitao/yht_bucket_batch_ci
5 years ago
yanghaitao1
da15495d7b
fix bucket_batch ci error
5 years ago
mindspore-ci-bot
d09025acab
!6451 check if partition file exists when usage is not all in CelebADataset
Merge pull request !6451 from yanghaitao/yht_celeba_log
5 years ago
yanghaitao1
f55f274bb4
check list_eval_partition.txt file
5 years ago
Cathy Wong
e971a3aed9
dataset API docstring: Update datasets.py and config.py
5 years ago
mindspore-ci-bot
e60e4920f9
!6346 add timeout value to pyfunc
Merge pull request !6346 from yanghaitao/yht_pyfunc_timeout
5 years ago
yanghaitao1
95f54de5d6
add timeout for pyfunc
5 years ago
mindspore-ci-bot
f6ac30ef29
!6221 stop send data to device after end of sequence
Merge pull request !6221 from anzhengqi/stop-send-at-eos
5 years ago
anzhengqi
8e1a2ef5ae
stop send data to device after end of sequence
5 years ago
Cathy Wong
f7adf648e9
dataset API docstring: Update datasets, samplers, graphdata and text
5 years ago
liyong
16147669a6
fix num_samples in concatDataset
5 years ago
xiefangqi
9b3c33e157
minddata iterator output ms_tensor
5 years ago
Zirui Wu
1bb93580dd
Add usage flag to MNIST/CIFAR
ci fix 1
fix ci 2
fix ci 3
fix ci 4
fix ci 5
fix ci 6
fix ci 7
change var name in CelebA from dataset_type to usage
address review cmts
fix cpp ut fail
change mode to usage in VOCDataset
fix cmts
fix ci test case fail
5 years ago
xiefangqi
d0410d6191
md delete set_dataset_size interface
5 years ago
nhussain
3bac9d3713
switch input columns and operation
change ImagefolderDV2 name
change ds.transforms.vision to ds.vision
change batch api to match map api more closely
compose op changes
test_pylint
remove compose op from vision, move to transform module, refactor map and batch to use column_order
5 years ago
mindspore-ci-bot
2f3684d4fa
!4872 Add C++ API support for TFRecordDataset
Merge pull request !4872 from TinaMengtingZhang/cpp-api-tfrecord-dataset
5 years ago
mindspore-ci-bot
d76ac7c6e8
!5648 GetColumnNames for Python
Merge pull request !5648 from MahdiRahmaniHanzaki/get-col-name
5 years ago
jiangzhiwen
1a1a8893a4
Align num_samples of CSV with other dataset
5 years ago
mindspore-ci-bot
0118930c6b
!5671 modify voc doc & fix buildvocab bug
Merge pull request !5671 from luoyang/c-api-pyfunc
5 years ago
Mahdi
6e7aab8c4f
Added GetColumnNames for Datasets
5 years ago
Cathy Wong
4d4c11b133
dataset API docstring cleanup: Standard product terms NumPy, Python
5 years ago
tinazhang
e430b4056a
add tfrecord dataset to cpp api
fix to support schema=nullptr
5 years ago
luoyang
8aba39a71b
Modify VOCDataset doc & Fix BuildVocab bug
5 years ago
Lixia Chen
27d5281641
Change epoch count to 1 for python testcases
5 years ago
anzhengqi
305434786a
add num_epochs api
5 years ago
mindspore-ci-bot
b23b957228
!4964 memory ascend while multi model.eval
Merge pull request !4964 from anzhengqi/I1QWT0-memory-ascend
5 years ago
mindspore-ci-bot
ac81886328
!4916 fix generator_dataset hangs and test_graphdata_distributed.py failing randomly
Merge pull request !4916 from heleiwang/gnn_fix_bug
5 years ago