updates

fix

fix cpplint

fix

init remove storage op test case c++

remove source c++ files

Merge pull request !2317 from ZiruiWu/vocab_rework

phase II vocab rework

added more test cases

fix api doc string

address review cmts and fix CI

address ci complains

fix review cmts

ci

c implementation done, just need to call batch/pad

added python api and validator

added pybind/de_pipeline stuff, fixed some compile errors, figure out how null py::function works

added tiny bit of doc

integrated with static batch methods

fixed some bugs

some more bug fixes and cleanup

ci fix

fix ci

ci fix

fix ci

added test_cases and debugged

addressed code review comments

addressed code review comments

ci fix

ci fix

addressed code review comments

addressed code review comments

fix complie error

more tests

address CI complains

fix ci

adress review comments

address review cmts

Merge pull request !1913 from xiefangqi/xfq_support_coco

Merge pull request !1874 from h.farahat/string_tensor_updates

- Support accepting Numpy of str as input
- Support batching strings
- Core logic of batch&pad is static
- Make Pad a utility function

updates

more update

more work

more fixin

post rebase updates

clang formatting

code review recovery

ci fixes

updates

update

update

update

Merge pull request !1281 from Peilin/splitOp

implementation 99% complete

everything and tested except for repeatable shuffling

tested most basic/typical split usecases

cleanup

some more cleanup

fix CI

more ci fix

more ci fixes

more ci fix

more ci fix

more ci fix

added more tests, fixed some bugs

some more clean up and test cases

added shard/shuffle before split warning/error

addressed code review comments and ci

fixed ci

Merge pull request !1157 from ms_yan/concat_dataset

Merge pull request !1126 from xiefangqi/voc_support_detection

change to use __add__ operation instead ds.concat

support for 1 specific dimension to be None, added validator

fix various CI complains

another round of CI fixes

ci

refactor parts of the code

code refactor

ci fix

comments added, fix bugs

address review comments

address review comments

review cmts

added simple perf test script

update pad code

perf imprv

Initial commit for dataset op python

Added signature to barrier

Adde compiling barrier code

Rebasing, fixed new compile errors

Final fix for make_unique

Added pybind API for barrier

Fixed pyfunc invocation

python interface - sync_wait

!1 sync_wait python interface
* python interface - sync_wait

fix test

update test

update test

Added new test case

add test case

test for shuffle + batch

Added two-sync test case

Restrited that no shuffle after sync

Added sync to pipeline info

block first databuffer as well

Intelligently get batch size

Fix default case

Lock Pair shares among all iterators

Added fix for empty character

Fixed up test case formatting

Fix end of epoch in sync_wait

Fixing CI

adjust internal api

Merge pull request !168 from Alexey_Shevlyakov/remove_mindspore_make_unique

Signed-off-by: leonwanghui <leon.wanghui@huawei.com>