yanghaitao
f30928f084
fix tfreaderop hang
5 years ago
mindspore-ci-bot
af1fde399b
!1509 dataset: PR1457 fix 3 bug reports for split
Merge pull request !1509 from ms_yan/r0.3_split
5 years ago
panfengfeng
d0d7864ccc
fix compilation order
5 years ago
Peilin Wang
9228384304
fixed bug for split, RandomSampler and some other cleanup
Cleanup dataset UT: restore config support
5 years ago
mindspore-ci-bot
5157063cbb
!1470 Fix log and comment errors in graphdata
Merge pull request !1470 from heleiwang/r0.3_fix_log
5 years ago
heleiwang
b0a354830b
fix input check
5 years ago
heleiwang
0ca8daa1a2
fix log error
5 years ago
Peilin Wang
0cbcc7200b
made shuffle determinisitc for subsequent epochs
5 years ago
mindspore-ci-bot
6f733ec113
!1308 Stage 2 of adding support for string Tensor
Merge pull request !1308 from h.farahat/string_tensor2
5 years ago
heleiwang
599a449e0b
Support processing GNN data
6 years ago
hesham
df361d1d26
Change mem layout of string tensor
add support for MindRecord and TFRecord
----
optimize tensorshape
optimize tensorshape and FlatIndex
TFRecord and MindRecord support for string tensor
Modify mem layout
Add new constructor
Add method Allocate
Change some GetMutableBuffer usages to AllocateBuffer
6 years ago
jonwe
bb51bb88d7
add compress in mindrecord
6 years ago
mindspore-ci-bot
2e3d55ed87
!1281 Implementation of SplitOp
Merge pull request !1281 from Peilin/splitOp
5 years ago
Peilin Wang
71e8bb1960
general split case done, chaining sampler (basic case) is working
implementation 99% complete
everything and tested except for repeatable shuffling
tested most basic/typical split usecases
cleanup
some more cleanup
fix CI
more ci fix
more ci fixes
more ci fix
more ci fix
more ci fix
added more tests, fixed some bugs
some more clean up and test cases
added shard/shuffle before split warning/error
addressed code review comments and ci
fixed ci
6 years ago
mindspore-ci-bot
661715bad9
!1318 Fix batch error log too simple problem
Merge pull request !1318 from xiefangqi/xfq_fix_batch_log
5 years ago
xiefangqi
25bb3960d4
add detail batch error to log
5 years ago
xiefangqi
394aeb0360
fix too much MS_LOG(INFO) in MD
5 years ago
Junhan Hu
8f774d6149
Fix CI
6 years ago
Junhan Hu
f44d213503
MindData optimizer infrastructure.
6 years ago
ms_yan
e6c25859d4
check wether input column exist in TFRecord files
6 years ago
mindspore-ci-bot
9b8b699eb3
!1235 Auto remove CRLF of VOCDataset txt file
Merge pull request !1235 from xiefangqi/xfq_del_CRLF
6 years ago
xiefangqi
9e48e3daaa
del CR in VOC txt
6 years ago
hesham
e8ca243364
-Add DE_STRING
-replace switch'case by indexing
- Add test case
- Add constructors
- Add getItem string
- Fix bugs
- Add more tests
- Tensor iterator
- asNumpy
- TextFileDataset
- Tensor(Numpy)
- Super > 2D
- Add more test cases for GeneratorDataset
- Change StartAddr to GetBuffer and GetMutableNuffer
- Raise an error if batch is used with strings
Clean-up work
6 years ago
ms_yan
30739001c8
make sure validate column ok first
6 years ago
mindspore-ci-bot
c680cfbf27
!1157 dataset: add concat operation for dataset
Merge pull request !1157 from ms_yan/concat_dataset
6 years ago
mindspore-ci-bot
ab031ee9ea
!1126 VOCDataset support object detection function
Merge pull request !1126 from xiefangqi/voc_support_detection
6 years ago
xiefangqi
c937bad53f
minddata support voc
6 years ago
ms_yan
c0fa7b4b19
init commit of concat dataset
change to use __add__ operation instead ds.concat
6 years ago
mindspore-ci-bot
8f40f36c6c
!924 gpu queue support multi-inputs
Merge pull request !924 from chenweifeng/dataset
6 years ago
liyong
aa3f89e74f
mindrecord support read file list
6 years ago
Jamie Nisbet
ff5f2521fc
remove unused variable
6 years ago
wilfChen
ccf6dabe13
gpu queue support multi-inputs
6 years ago
fary86
3f323e48e2
Add submodule id for log interface
6 years ago
xulei2020
5b89fa78b0
fix thread hang bug
6 years ago
Jamie Nisbet
7b7b2fa7a0
remove column name id map from data buffer
clang format updates
self-review cleanups
cpplint fix
review updates
ci fixes
6 years ago
jonyguo
cf352d190f
format func name for mindrecord
6 years ago
mindspore-ci-bot
2e80be73b4
!978 [Data]BUG replace Post() after LaunchWorkers
Merge pull request !978 from xulei/filter_op
6 years ago
mindspore-ci-bot
6c79c00a3f
!979 dataset: repair parameter check in rename Op
Merge pull request !979 from ms_yan/rename_columns
6 years ago
mindspore-ci-bot
078dd86cfe
!507 Implemented padded_batch
Merge pull request !507 from ZiruiWu/batch_with_padding
6 years ago
Zirui Wu
c2d364a573
batch with padding implemented
support for 1 specific dimension to be None, added validator
fix various CI complains
another round of CI fixes
ci
refactor parts of the code
code refactor
ci fix
comments added, fix bugs
address review comments
address review comments
review cmts
added simple perf test script
update pad code
perf imprv
6 years ago
ms_yan
d95a2e9ccf
repair num of input columns and output columns check
6 years ago
jonyguo
d4d236bcce
fix: use MindDataset by column_names get data error in some situation
6 years ago
xulei2020
e6e0d62775
replace FindMe after LaunchWorker
6 years ago
panfengfeng
d8a2847bbe
after error run the post
6 years ago
Nat Sutyanyong
f19f394bb8
Refactor duplicate code on random device and seed
6 years ago
ms_yan
c56fe3aa2d
modify take op with an operator
6 years ago
caifubi
cb70daa355
format code
6 years ago
mindspore-ci-bot
2303453753
!869 Random data op
Merge pull request !869 from JesseKLee/random_data_op
6 years ago
Jesse Lee
b1ea933345
Fix CI
6 years ago
Jesse Lee
270bf831a9
Random Data Op
6 years ago