Alexey Shevlyakov
4e333a2f22
MindData profiler infrastructure
5 years ago
mindspore-ci-bot
b3da41bd7a
!1913 CocoDataset implementation
Merge pull request !1913 from xiefangqi/xfq_support_coco
5 years ago
xiefangqi
097cbcc0cb
CocoDataset implementation
5 years ago
heleiwang
3ece8dd090
1. support get_all_edges, get_nodes_from_edge, get_sampled_neighbors, get_neg_sampled_neighbors and graph_info API
2. mod cora and citeseer conversion script
5 years ago
liyong
feff8899ac
support padding samples
5 years ago
mindspore-ci-bot
5c21616293
!1807 Implemented Ngram TensorOp for dataset
Merge pull request !1807 from ZiruiWu/ngram_dev
5 years ago
Zirui Wu
dbf9936ec4
Implemented n-gram for dataset TensorOp
5 years ago
Jamie Nisbet
51bc0c0460
consistent design for num_samples
updates
more update
more work
more fixin
post rebase updates
clang formatting
code review recovery
ci fixes
updates
update
update
update
5 years ago
xiefangqi
5e4728c50f
add voc support split
5 years ago
anthony
ba2eaea7e1
Changed uniform_aug_op py::list to std::vector with some cleanup and enhancement.
5 years ago
mindspore-ci-bot
458436186c
!1365 Clean up work for text python sub-package
Merge pull request !1365 from h.farahat/text_namespace
5 years ago
hesham
6c21e556c4
Clean up work for text python package
5 years ago
heleiwang
599a449e0b
Support processing GNN data
5 years ago
mindspore-ci-bot
58e6d7d950
!1341 Added lookup and vocab to mindspore.dataset.text
Merge pull request !1341 from ZiruiWu/vocab_and_lookup
5 years ago
mindspore-ci-bot
2e3d55ed87
!1281 Implementation of SplitOp
Merge pull request !1281 from Peilin/splitOp
5 years ago
Peilin Wang
71e8bb1960
general split case done, chaining sampler (basic case) is working
implementation 99% complete
everything and tested except for repeatable shuffling
tested most basic/typical split usecases
cleanup
some more cleanup
fix CI
more ci fix
more ci fixes
more ci fix
more ci fix
more ci fix
added more tests, fixed some bugs
some more clean up and test cases
added shard/shuffle before split warning/error
addressed code review comments and ci
fixed ci
5 years ago
Zirui Wu
25ab2ef303
Implemented lookup and vocab
5 years ago
qianlong
451c20a6f5
Add UnicodeCharTokenizer for nlp
5 years ago
xulei2020
163b6b7ea7
add jieba c++ code
5 years ago
hesham
e8ca243364
-Add DE_STRING
-replace switch'case by indexing
- Add test case
- Add constructors
- Add getItem string
- Fix bugs
- Add more tests
- Tensor iterator
- asNumpy
- TextFileDataset
- Tensor(Numpy)
- Super > 2D
- Add more test cases for GeneratorDataset
- Change StartAddr to GetBuffer and GetMutableNuffer
- Raise an error if batch is used with strings
Clean-up work
5 years ago
mindspore-ci-bot
c680cfbf27
!1157 dataset: add concat operation for dataset
Merge pull request !1157 from ms_yan/concat_dataset
5 years ago
mindspore-ci-bot
ab031ee9ea
!1126 VOCDataset support object detection function
Merge pull request !1126 from xiefangqi/voc_support_detection
5 years ago
xiefangqi
c937bad53f
minddata support voc
5 years ago
ms_yan
c0fa7b4b19
init commit of concat dataset
change to use __add__ operation instead ds.concat
5 years ago
liyong
aa3f89e74f
mindrecord support read file list
5 years ago
xiefangqi
9a2daae84c
del unused change_mode_op
5 years ago
Jesse Lee
270bf831a9
Random Data Op
5 years ago
liyong
bfba630aa2
update pK_sampler
5 years ago
Alexey Shevlyakov
6acae622dc
fix random seed behaviour
5 years ago
mindspore-ci-bot
dc0491caf9
!508 [Dataset] Adding sync_wait operator for dataset
Merge pull request !508 from EricZ/master
5 years ago
mindspore-ci-bot
b0f4b36f48
!511 Uniform Augmentation C++ OP
Merge pull request !511 from AdelShafiei/uniform_augmentation
5 years ago
eric
cd94518769
X# This is a combination of 2 commits.
Initial commit for dataset op python
Added signature to barrier
Adde compiling barrier code
Rebasing, fixed new compile errors
Final fix for make_unique
Added pybind API for barrier
Fixed pyfunc invocation
python interface - sync_wait
!1 sync_wait python interface
* python interface - sync_wait
fix test
update test
update test
Added new test case
add test case
test for shuffle + batch
Added two-sync test case
Restrited that no shuffle after sync
Added sync to pipeline info
block first databuffer as well
Intelligently get batch size
Fix default case
Lock Pair shares among all iterators
Added fix for empty character
Fixed up test case formatting
Fix end of epoch in sync_wait
Fixing CI
6 years ago
Adel Shafiei
5fcd3f01a6
Added C++ UniformAugOp support
5 years ago
liyong
f1542a90a3
add pk sampler
5 years ago
mindspore-ci-bot
822a3160e4
!404 [Dataset] Add Python Sampler support for CPP dataset
Merge pull request !404 from JunhanHu/python_sampler_for_cpp_dataset
5 years ago
yanghaitao
2795e492ff
TextFileDataset
5 years ago
Junhan Hu
43a2e99833
Add python sampler support for CPP dataset
5 years ago
jzw
3f7054dccb
add skip dataset op
6 years ago
mindspore-ci-bot
c0c0b0985e
!378 Multiple Iterators can cause performance problem
Merge pull request !378 from h.farahat/multi_itr
5 years ago
hesham
fb6c7ba2e1
Fix two problem when we create multiple instances of the same dataset (2 for-loops)
-- Iterator list is keeping all created iterators wihtout cleaning them up
-- alter tree modifies the original.
5 years ago
mindspore-ci-bot
cf026096a6
!183 Mindspore.dataset CPP sampler for GeneratorDataset
Merge pull request !183 from JunhanHu/cpp_sampler
5 years ago
chenjianping
1286767d0e
support building on windows
5 years ago
Junhan Hu
9739d3b048
Add CPP sampler support for GeneratorDataset
6 years ago
mindspore-ci-bot
f1fa2a9941
!273 [MD] update subset random sampler in minddataset
Merge pull request !273 from liyong126/mindrecord_subset_sampler_python
5 years ago
liyong
0ce83e39e1
fix TestShardSampleWrongNumber
adjust internal api
5 years ago
Zirui Wu
19f2ea6d41
change int32 to int64 to avoid overflow in batch
fix clang
pybind fix
6 years ago
Jonathan Yan
9d0fde29f4
remove ENABLE_MINDRECORD flag
6 years ago
zhunaipan
930a1fb0a8
initial version
Signed-off-by: leonwanghui <leon.wanghui@huawei.com>
6 years ago