Huawei_Technology
/
mindspore
Not watched
1
0
Fork 0
Code
Releases 13 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
7793 Commits
11 Branches
1.3 GB
177 Download
Tree: b2bc8e651d
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'b2bc8e651d'
${ noResults }
mindspore/tests/ut/data/dataset/testTokenizerData/regex_replace.txt

regex_replace.txt 109 B
Raw Normal View History

Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp add CaseFold, NormalizeUTF8 add RegexReplace add RegexTokenizer add BasicTokenizer add WordpieceTokenizer add BertTokenizer
5 years ago
​
​
​
​
​
​
​
 12345678 
  1. Hello World

  2. Let's Go

  3. 1:hello

  4. 2:world

  5. 31:beijing

  6. Welcome to China!

  7.   我	不想  长大	

  8. Welcome to Shenzhen!

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.