You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
qianlong 4f16f036be Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 5 years ago
..
1.txt Add UnicodeCharTokenizer for nlp 5 years ago
basic_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 5 years ago
bert_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 5 years ago
normalize.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 5 years ago
regex_replace.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 5 years ago
regex_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 5 years ago
wordpiece_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 5 years ago