You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
tophand 797cd52c96 [feat][assistant][I40GXK]add new data operator Filter_Wikipedia_XML 4 years ago
..
CMakeLists.txt [feat][assistant][I40GXK]add new data operator Filter_Wikipedia_XML 4 years ago
basic_tokenizer_op.cc Revert "[feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset" 4 years ago
basic_tokenizer_op.h files_need_cleanup fixes 4 years ago
bert_tokenizer_op.cc mindspore path adjust 5 years ago
bert_tokenizer_op.h files_need_cleanup fixes 4 years ago
case_fold_op.cc Revert "[feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset" 4 years ago
case_fold_op.h remove print calls 5 years ago
data_utils.cc remove text offsets duplicate code 4 years ago
data_utils.h remove text offsets duplicate code 4 years ago
filter_wikipedia_xml_op.cc [feat][assistant][I40GXK]add new data operator Filter_Wikipedia_XML 4 years ago
filter_wikipedia_xml_op.h [feat][assistant][I40GXK]add new data operator Filter_Wikipedia_XML 4 years ago
jieba_tokenizer_op.cc pclint fixes to master 4 years ago
jieba_tokenizer_op.h pclint fixes to master 4 years ago
lookup_op.cc fixed code quality issues 4 years ago
lookup_op.h update lookup api to take in a type 5 years ago
ngram_op.cc change err info 4 years ago
ngram_op.h all the updated input order 4 years ago
normalize_utf8_op.cc Removed unused include 4 years ago
normalize_utf8_op.h add four new text API 5 years ago
regex_replace_op.cc Removed unused include 4 years ago
regex_replace_op.h files_need_cleanup fixes 4 years ago
regex_tokenizer_op.cc pclint fixes to master 4 years ago
regex_tokenizer_op.h files_need_cleanup fixes 4 years ago
sentence_piece_tokenizer_op.cc Change data() to c_str() when getting const char* 4 years ago
sentence_piece_tokenizer_op.h files_need_cleanup fixes 4 years ago
sliding_window_op.cc change err info 4 years ago
sliding_window_op.h remove print calls 5 years ago
to_number_op.cc add header file 4 years ago
to_number_op.h all the updated input order 4 years ago
to_vectors_op.cc [fix] [assistant] [I3ZSQM] add new data operator Vectors 4 years ago
to_vectors_op.h [fix] [assistant] [I3ZSQM] add new data operator Vectors 4 years ago
tokenizer_op.cc fixed code quality issues 4 years ago
tokenizer_op.h files_need_cleanup fixes 4 years ago
truncate_sequence_pair_op.cc change err info 4 years ago
truncate_sequence_pair_op.h update cpp api & doc 4 years ago
unicode_char_tokenizer_op.cc add header file 4 years ago
unicode_char_tokenizer_op.h files_need_cleanup fixes 4 years ago
unicode_script_tokenizer_op.cc fixed code quality issues 4 years ago
unicode_script_tokenizer_op.h files_need_cleanup fixes 4 years ago
whitespace_tokenizer_op.cc pclint fixes to master 4 years ago
whitespace_tokenizer_op.h files_need_cleanup fixes 4 years ago
wordpiece_tokenizer_op.cc !17729 Fix pclint warning in master 4 years ago
wordpiece_tokenizer_op.h files_need_cleanup fixes 4 years ago