YangLuo
4e3bfcf4c9
!2306 [Dataset] Code review & improve quality
5 years ago
liyong
2412ee09ce
fix distributedSampler reshuffle and fix random_device failed
5 years ago
mindspore-ci-bot
d0c5071cab
!1993 [Dataset] Fix codedex.
Merge pull request !1993 from luoyang/pylint
5 years ago
jonyguo
fb73c9c215
fix: db close fail
5 years ago
Yang
dee8471d23
!1993 [Dataset] Fix codedex.
5 years ago
liyong
d4f8f57c7e
add split in minddataset
5 years ago
jonyguo
bb2082daec
fix: MindDataset padded log error
5 years ago
jonyguo
1de7271afc
add floatxx test case
5 years ago
jonyguo
96859f44b6
fix: MindDataset distribute shuffle bug
5 years ago
Yang
525d063abf
[Dataset] Fix codedex.
5 years ago
liyong
feff8899ac
support padding samples
5 years ago
jonyguo
2aaf3a37d4
fix: mindrecord seekg failed when shift raw page
5 years ago
mindspore-ci-bot
6f733ec113
!1308 Stage 2 of adding support for string Tensor
Merge pull request !1308 from h.farahat/string_tensor2
5 years ago
mindspore-ci-bot
3363d4e834
!1249 Add GNN dataset processing API
Merge pull request !1249 from heleiwang/hlw_gnn_data
5 years ago
heleiwang
599a449e0b
Support processing GNN data
5 years ago
hesham
df361d1d26
Change mem layout of string tensor
add support for MindRecord and TFRecord
----
optimize tensorshape
optimize tensorshape and FlatIndex
TFRecord and MindRecord support for string tensor
Modify mem layout
Add new constructor
Add method Allocate
Change some GetMutableBuffer usages to AllocateBuffer
5 years ago
liyong
447420eb9a
fix bug when append data by absolute path
5 years ago
jonwe
bb51bb88d7
add compress in mindrecord
5 years ago
mindspore-ci-bot
cf6a4fe687
!1155 fix bug of log interface
Merge pull request !1155 from fary86/fix_bug_of_log_interface
5 years ago
fary86
057dcdbad5
Fix bug of log interface
1.GLOG_v can not control log level of MindData
2.Sub module ids of some source files under directory `kernel` and `device` are incorrect
3.GLOG_v can not control log level when using dlog
5 years ago
liyong
dbfa34a3f5
remove dead code
5 years ago
liyong
aa3f89e74f
mindrecord support read file list
5 years ago
fary86
3f323e48e2
Add submodule id for log interface
5 years ago
jonyguo
cf352d190f
format func name for mindrecord
5 years ago
mindspore-ci-bot
de7625777f
!951 fix: MindDataset with columns_name parameter cause errors in some scenes
Merge pull request !951 from guozhijian/fix_read_by_columns
5 years ago
jonyguo
d4d236bcce
fix: use MindDataset by column_names get data error in some situation
5 years ago
liyong
b520ca9087
fix pk sampler in mindrecord
5 years ago
Adel Shafiei
3322e65da9
added ut for uniform augment C++ op
5 years ago
liyong
bfba630aa2
update pK_sampler
5 years ago
liyong
1f222ddb9e
fix mindrecord c ut
5 years ago
mindspore-ci-bot
fe67db75a3
!580 fix dataset & mindrecord log print
Merge pull request !580 from panfengfeng/panff/fix_log_print_in_ge
5 years ago
jonwe
353b3b7944
optimize mindrecord writer performance
5 years ago
panfengfeng
b31946750b
回退 'Pull Request !182 : Tuning mindrecord writer performance'
5 years ago
mindspore-ci-bot
c73301bad9
!182 Tuning mindrecord writer performance
Merge pull request !182 from jonathan_yan/write_a
5 years ago
panfengfeng
6328edf455
modify dataset & mindrecord log compile
5 years ago
jonwe
f4bae5f364
optimize mindrecord writer performance
5 years ago
liyong
f1542a90a3
add pk sampler
6 years ago
xiefangqi
d4822137f3
fix segment bug on windows
6 years ago
chenjianping
1286767d0e
support building on windows
6 years ago
jonyguo
ea5da25d10
fix: use exactly read option
6 years ago
liyong
0ce83e39e1
fix TestShardSampleWrongNumber
adjust internal api
6 years ago
liyong
11403492ae
add mindrecord subset random sampler
6 years ago
mindspore-ci-bot
7cec28526a
!95 Modify cmake of protobuf
Merge pull request !95 from dengwentao/protobuf
6 years ago
dengwentao
b5ca2a3e30
:modify protobuf cmake
6 years ago
jonyguo
a9443635b7
fix: mindpage enhance parameter check and search by filename failed
6 years ago
mindspore-ci-bot
bd7ac259af
!147 Parallelize mindrecord index writer via std::thread
Merge pull request !147 from ZiruiWu/master
6 years ago
Zirui Wu
5637f80692
implemented multi-thread index writer for mindrecord
num threads cannot be more than num shards
minor fix
clang style fix
address review comments
6 years ago
jonyguo
20d1b64443
fix: error info is not exactly when column list invalid
6 years ago
jonyguo
c688265671
fix: when use MindDataset block_reade=True hung
6 years ago
zhunaipan
930a1fb0a8
initial version
Signed-off-by: leonwanghui <leon.wanghui@huawei.com>
6 years ago