Merge pull request !2211 from h.farahat/pair_truncate

Merge pull request !2198 from h.farahat/mask_op

add CaseFold, NormalizeUTF8

add RegexReplace

add RegexTokenizer

add BasicTokenizer

add WordpieceTokenizer

add BertTokenizer

Merge pull request !1977 from JonathanY/randomwalk

Merge pull request !2162 from h.farahat/slice_op

Merge pull request !2028 from nhussain/fillOp_learning

c implementation done, just need to call batch/pad

added python api and validator

added pybind/de_pipeline stuff, fixed some compile errors, figure out how null py::function works

added tiny bit of doc

integrated with static batch methods

fixed some bugs

some more bug fixes and cleanup

ci fix

fix ci

ci fix

fix ci

added test_cases and debugged

addressed code review comments

addressed code review comments

ci fix

ci fix

addressed code review comments

addressed code review comments

Merge pull request !1932 from jiangzhiwen/dataset/clue

Merge pull request !2029 from lixiachen/lixia/tensorrow

Merge pull request !1981 from ZiruiWu/from_dataset_dev

fix complie error

more tests

address CI complains

fix ci

adress review comments

address review cmts

Merge pull request !1913 from xiefangqi/xfq_support_coco

2. mod cora and citeseer conversion script

Merge pull request !1807 from ZiruiWu/ngram_dev

updates

more update

more work

more fixin

post rebase updates

clang formatting

code review recovery

ci fixes

updates

update

update

update

Merge pull request !1365 from h.farahat/text_namespace

Merge pull request !1341 from ZiruiWu/vocab_and_lookup

Merge pull request !1281 from Peilin/splitOp

implementation 99% complete

everything and tested except for repeatable shuffling

tested most basic/typical split usecases

cleanup

some more cleanup

fix CI

more ci fix

more ci fixes

more ci fix

more ci fix

more ci fix

added more tests, fixed some bugs

some more clean up and test cases

added shard/shuffle before split warning/error

addressed code review comments and ci

fixed ci

-replace switch'case by indexing

- Add test case
- Add constructors
- Add getItem string

- Fix bugs
- Add more tests

- Tensor iterator
- asNumpy
- TextFileDataset

- Tensor(Numpy)

- Super > 2D
- Add more test cases for GeneratorDataset

- Change StartAddr to GetBuffer and GetMutableNuffer

- Raise an error if batch is used with strings

Clean-up work

Merge pull request !1157 from ms_yan/concat_dataset

Merge pull request !1126 from xiefangqi/voc_support_detection

change to use __add__ operation instead ds.concat