Yang
abca62f407
10:00 26/5 clean pylint
5 years ago
mindspore-ci-bot
0a8ef2fe72
!1455 Cleanup dataset UT: restore config support
Merge pull request !1455 from cathwong/ckw_dataset_config_util
5 years ago
Cathy Wong
f891e1755c
Cleanup dataset UT: restore config support
5 years ago
mindspore-ci-bot
aeb4c52f2d
!1379 Creating new UTs to three vision transformation ops
Merge pull request !1379 from Tinazhang/linearTrans
5 years ago
mindspore-ci-bot
718a96430f
!1389 Security issues in recent string tensor PR
Merge pull request !1389 from h.farahat/fix_security_issues
5 years ago
mindspore-ci-bot
a6b8451a33
!1380 make ShuffleOp have deterministic behavior for subsequent epochs
Merge pull request !1380 from Peilin/shuffle-subsequent-epoch-deterministic
5 years ago
heleiwang
f28f883cac
fix comments error and modify parameter check
5 years ago
hesham
c0f0e154f5
Clean up work for text python package
5 years ago
mindspore-ci-bot
41456ac824
!1369 dataset: delete StorageDataset
Merge pull request !1369 from ms_yan/del_storage
5 years ago
Tinazhang
7322839b04
add UTs for LinearTransformation, ToPIL, ToType
5 years ago
Peilin Wang
0cbcc7200b
made shuffle determinisitc for subsequent epochs
5 years ago
mindspore-ci-bot
61639be1e0
!1375 Cleanup dataset UT: remove num_parallel_workers=1 in test_exception
Merge pull request !1375 from cathwong/ckw_dataset_ut_cleanup5
5 years ago
mindspore-ci-bot
a528797253
!1377 Code Fix for Uniform Augmentation
Merge pull request !1377 from Tinazhang/cc
5 years ago
mindspore-ci-bot
458436186c
!1365 Clean up work for text python sub-package
Merge pull request !1365 from h.farahat/text_namespace
5 years ago
Tinazhang
b390883c6a
Bug fix
5 years ago
Cathy Wong
702005d403
Cleanup dataset UT: remove num_parallel_workers=1 in test_exception
5 years ago
ms_yan
d5e896b51c
delete storageDataset Op API and its test case
5 years ago
mindspore-ci-bot
6f733ec113
!1308 Stage 2 of adding support for string Tensor
Merge pull request !1308 from h.farahat/string_tensor2
5 years ago
xiefangqi
34236ce1f1
fix pylint
5 years ago
mindspore-ci-bot
3363d4e834
!1249 Add GNN dataset processing API
Merge pull request !1249 from heleiwang/hlw_gnn_data
5 years ago
hesham
6c21e556c4
Clean up work for text python package
5 years ago
heleiwang
599a449e0b
Support processing GNN data
5 years ago
jinyaohui
fbdba6e4da
clean pylint
5 years ago
hesham
df361d1d26
Change mem layout of string tensor
add support for MindRecord and TFRecord
----
optimize tensorshape
optimize tensorshape and FlatIndex
TFRecord and MindRecord support for string tensor
Modify mem layout
Add new constructor
Add method Allocate
Change some GetMutableBuffer usages to AllocateBuffer
5 years ago
mindspore-ci-bot
58e6d7d950
!1341 Added lookup and vocab to mindspore.dataset.text
Merge pull request !1341 from ZiruiWu/vocab_and_lookup
5 years ago
jonwe
bb51bb88d7
add compress in mindrecord
5 years ago
mindspore-ci-bot
2e3d55ed87
!1281 Implementation of SplitOp
Merge pull request !1281 from Peilin/splitOp
5 years ago
mindspore-ci-bot
39b9aedf68
!1342 Bug fix on issue Core dump on GPU when train with lenet with AU
Merge pull request !1342 from Tinazhang/cc
5 years ago
Peilin Wang
71e8bb1960
general split case done, chaining sampler (basic case) is working
implementation 99% complete
everything and tested except for repeatable shuffling
tested most basic/typical split usecases
cleanup
some more cleanup
fix CI
more ci fix
more ci fixes
more ci fix
more ci fix
more ci fix
added more tests, fixed some bugs
some more clean up and test cases
added shard/shuffle before split warning/error
addressed code review comments and ci
fixed ci
5 years ago
Tinazhang
e9e40b688b
Bug fix
5 years ago
Zirui Wu
25ab2ef303
Implemented lookup and vocab
5 years ago
mindspore-ci-bot
46949fc327
!1307 Cleanup dataset UT: unskip and enhance TFRecord sharding tests
Merge pull request !1307 from cathwong/ckw_dataset_ut_unskip2
5 years ago
qianlong
451c20a6f5
Add UnicodeCharTokenizer for nlp
5 years ago
mindspore-ci-bot
93e7c97a96
!1272 [Dataset] MindData Tree Optimizer Infrastructure
Merge pull request !1272 from JunhanHu/minddata_opt
5 years ago
Cathy Wong
b78894e02b
Cleanup dataset UT: unskip and enhance TFRecord sharding tests
5 years ago
Junhan Hu
f44d213503
MindData optimizer infrastructure.
5 years ago
xulei2020
163b6b7ea7
add jieba c++ code
5 years ago
Tinazhang
17cecf2cf5
Added TCs to RandomCrop and RandomCropAndResize and modified visalize() calling
5 years ago
jinyaohui
5a914994ba
clean pylint
5 years ago
jinyaohui
bcfaff97f9
clean pylint
5 years ago
hesham
e8ca243364
-Add DE_STRING
-replace switch'case by indexing
- Add test case
- Add constructors
- Add getItem string
- Fix bugs
- Add more tests
- Tensor iterator
- asNumpy
- TextFileDataset
- Tensor(Numpy)
- Super > 2D
- Add more test cases for GeneratorDataset
- Change StartAddr to GetBuffer and GetMutableNuffer
- Raise an error if batch is used with strings
Clean-up work
5 years ago
jiangzhiwen
cb2814b498
flat_map first commit
5 years ago
mindspore-ci-bot
c680cfbf27
!1157 dataset: add concat operation for dataset
Merge pull request !1157 from ms_yan/concat_dataset
5 years ago
mindspore-ci-bot
ab031ee9ea
!1126 VOCDataset support object detection function
Merge pull request !1126 from xiefangqi/voc_support_detection
5 years ago
xiefangqi
c937bad53f
minddata support voc
5 years ago
ms_yan
c0fa7b4b19
init commit of concat dataset
change to use __add__ operation instead ds.concat
5 years ago
jonyguo
be2e7531ca
fix: MindDataset parameter shard_id & num_shards check
5 years ago
Cathy Wong
913074e656
Cleanup dataset UT: resolve skipped test units
5 years ago
liyong
aa3f89e74f
mindrecord support read file list
5 years ago
Cathy Wong
49ef53f164
Cleanup dataset UT: util.py internals
5 years ago