Peilin Wang
71e8bb1960
general split case done, chaining sampler (basic case) is working
implementation 99% complete
everything and tested except for repeatable shuffling
tested most basic/typical split usecases
cleanup
some more cleanup
fix CI
more ci fix
more ci fixes
more ci fix
more ci fix
more ci fix
added more tests, fixed some bugs
some more clean up and test cases
added shard/shuffle before split warning/error
addressed code review comments and ci
fixed ci
6 年前
Tinazhang
e9e40b688b
Bug fix
6 年前
Zirui Wu
25ab2ef303
Implemented lookup and vocab
6 年前
mindspore-ci-bot
46949fc327
!1307 Cleanup dataset UT: unskip and enhance TFRecord sharding tests
Merge pull request !1307 from cathwong/ckw_dataset_ut_unskip2
6 年前
qianlong
451c20a6f5
Add UnicodeCharTokenizer for nlp
6 年前
mindspore-ci-bot
93e7c97a96
!1272 [Dataset] MindData Tree Optimizer Infrastructure
Merge pull request !1272 from JunhanHu/minddata_opt
6 年前
Cathy Wong
b78894e02b
Cleanup dataset UT: unskip and enhance TFRecord sharding tests
6 年前
Junhan Hu
f44d213503
MindData optimizer infrastructure.
6 年前
xulei2020
163b6b7ea7
add jieba c++ code
6 年前
Tinazhang
17cecf2cf5
Added TCs to RandomCrop and RandomCropAndResize and modified visalize() calling
6 年前
jinyaohui
5a914994ba
clean pylint
6 年前
jinyaohui
bcfaff97f9
clean pylint
6 年前
hesham
e8ca243364
-Add DE_STRING
-replace switch'case by indexing
- Add test case
- Add constructors
- Add getItem string
- Fix bugs
- Add more tests
- Tensor iterator
- asNumpy
- TextFileDataset
- Tensor(Numpy)
- Super > 2D
- Add more test cases for GeneratorDataset
- Change StartAddr to GetBuffer and GetMutableNuffer
- Raise an error if batch is used with strings
Clean-up work
6 年前
jiangzhiwen
cb2814b498
flat_map first commit
6 年前
mindspore-ci-bot
c680cfbf27
!1157 dataset: add concat operation for dataset
Merge pull request !1157 from ms_yan/concat_dataset
6 年前
mindspore-ci-bot
ab031ee9ea
!1126 VOCDataset support object detection function
Merge pull request !1126 from xiefangqi/voc_support_detection
6 年前
xiefangqi
c937bad53f
minddata support voc
6 年前
ms_yan
c0fa7b4b19
init commit of concat dataset
change to use __add__ operation instead ds.concat
6 年前
jonyguo
be2e7531ca
fix: MindDataset parameter shard_id & num_shards check
6 年前
Cathy Wong
913074e656
Cleanup dataset UT: resolve skipped test units
6 年前
liyong
aa3f89e74f
mindrecord support read file list
6 年前
Cathy Wong
49ef53f164
Cleanup dataset UT: util.py internals
6 年前
mindspore-ci-bot
2860fd9338
!984 Add unit test case for HWC2CHW.
Merge pull request !984 from Tinazhang/hwc2chw
6 年前
Tinazhang
c8b5586c7f
add unit test for HWC2CHWC
6 年前
Cathy Wong
58226addd6
Cleanup dataset UT: use md5 npz in test_zip for images
6 年前
mindspore-ci-bot
47f5abceb4
!960 Adding example for grayscale
Merge pull request !960 from EricZ/grayscale_fix
6 年前
mindspore-ci-bot
078dd86cfe
!507 Implemented padded_batch
Merge pull request !507 from ZiruiWu/batch_with_padding
6 年前
mindspore-ci-bot
de7625777f
!951 fix: MindDataset with columns_name parameter cause errors in some scenes
Merge pull request !951 from guozhijian/fix_read_by_columns
6 年前
eric
0f0548f21b
Added test case for grayscale support
6 年前
Zirui Wu
c2d364a573
batch with padding implemented
support for 1 specific dimension to be None, added validator
fix various CI complains
another round of CI fixes
ci
refactor parts of the code
code refactor
ci fix
comments added, fix bugs
address review comments
address review comments
review cmts
added simple perf test script
update pad code
perf imprv
6 年前
jonyguo
d4d236bcce
fix: use MindDataset by column_names get data error in some situation
6 年前
liyong
b520ca9087
fix pk sampler in mindrecord
6 年前
Cathy Wong
772e6c1461
Cleanup dataset UT: test_batch, save_and_check support
6 年前
eric
36fffb7706
Added example md5 generation
Comparison example
Added md5 and comparison example for py_transforms
Added md5 check for images
6 年前
Junhan Hu
83c68ca2ef
Skip pyfunc test case
6 年前
eric
26cb3e8a5f
Added test function to show that seed doesn't work.
Added testcase to show that c image aug don't use seed properly
Added passing test cases
Added working testcases for using seed
Added additional test cases to show seed use
Added test case for seed
6 年前
ms_yan
c56fe3aa2d
modify take op with an operator
6 年前
mindspore-ci-bot
8af10eb51e
!875 Reject python OP in operations argument for C++ uniform augmentation OP
Merge pull request !875 from AdelShafiei/ua_py
6 年前
Adel Shafiei
d15bd04bfe
added input validation to reject python op in C++ uniform augmentation operations list
6 年前
mindspore-ci-bot
a606c2e4da
!872 [Dataset] Add schema support for GeneratorDataset
Merge pull request !872 from JunhanHu/generator_schema
6 年前
mindspore-ci-bot
2303453753
!869 Random data op
Merge pull request !869 from JesseKLee/random_data_op
6 年前
Junhan Hu
c5a8ffe4f4
Add schema support for GeneratorDataset
6 年前
Jesse Lee
5236d0c3c0
Replace print with logger.info
6 年前
mindspore-ci-bot
8d3695f666
!672 Added UT for uniform augmentation C++ OP
Merge pull request !672 from AdelShafiei/ua_ut
6 年前
Jesse Lee
270bf831a9
Random Data Op
6 年前
jiangzhiwen
34bfa2f7c9
fix skip
6 年前
Adel Shafiei
3322e65da9
added ut for uniform augment C++ op
6 年前
mindspore-ci-bot
b37db1edf5
!603 [MD] update pk sampler in minddataset
Merge pull request !603 from liyong126/update_pk_sampler
6 年前
mindspore-ci-bot
f82e63fecc
!671 Added testcase for sync_wait
Merge pull request !671 from EricZ/master
6 年前
mindspore-ci-bot
0e3054d527
!466 Deepcopy problem when pyfunc cannot be pickled
Merge pull request !466 from h.farahat/deepcopy
6 年前