Huawei_Technology
/
mindspore
Not watched
1
0
Fork 0
Code
Releases 13 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
7793 Commits
11 Branches
1.3 GB
177 Download
Tree: b2bc8e651d
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'b2bc8e651d'
${ noResults }
mindspore/tests/ut/data/dataset/testTokenizerData/regex_tokenizer.txt

regex_tokenizer.txt 70 B
Raw Normal View History

Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp add CaseFold, NormalizeUTF8 add RegexReplace add RegexTokenizer add BasicTokenizer add WordpieceTokenizer add BertTokenizer
5 years ago
​
​
 123 
  1. Welcome to Shenzhen!

  2. 北京欢迎您!Welcome to Beijing!

  3. 12￥+36￥=?

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.