fastNLP/reproduction/sequence_labelling/cws/readme.md

四个数据集的统计信息，最原始的数据可以从[http://sighan.cs.uchicago.edu/bakeoff2005/]()下载。

| pku   | # of sents | # of tokens |
| ----- | ---------- | ----------- |
| train | 17173      | 1650222     |
| dev   | 1881       | 176226      |
| test  | 1944       | 172733      |
| total | 20998      | 1999181     |


| cityu | # of sents | # of tokens |
| ----- | ---------- | ----------- |
| train | 47696      | 2164907     |
| dev   | 5323       | 238447      |
| test  | 1492       | 67690       |
| total | 54511      | 2471044     |


| msra  | # of sents | # of tokens |
| ----- | ---------- | ----------- |
| train | 78242      | 3644550     |
| dev   | 8676       | 405919      |
| test  | 3985       | 184355      |
| total | 90903      | 4234824     |


| as    | # of sents | # of tokens |
| ----- | ---------- | ----------- |
| train | 638273     | 7536586     |
| dev   | 70680      | 831464      |
| test  | 14429      | 197681      |
| total | 723382     | 8565731     |