yh
39f3accace
增加Embedding的文档
6 years ago
ChenXin
df0bc2aec3
根据代码修改了文档
6 years ago
yh_cc
1babf5384f
Vocabulary中no_create_entry的bug修复
6 years ago
yh
1cc115e977
Merge branch 'dev0.5.0' of https://github.com/fastnlp/fastNLP into dev0.5.0
6 years ago
yh_cc
2c9a6e0ba4
1. 修改ELMO加载allennlp的权重;
6 years ago
yh
a40f57ae4a
修复Vocabulary在建好词表之后新加入词导致的pad index错乱的问题
6 years ago
yh_cc
8f7ed07441
1. 在vocabulary的from_dataset中增加no_create_entry_dataset选项,用于传递dev和test
2. 调整各种Embedding的实现,使得确保来自dev和test的未发现词使用unk的表示
3. 在Embedding中增加dropout_word的选项,使得可以随机drop掉词语
4. 以及其它若干小的bug
6 years ago
yh_cc
6309eafd25
1. 在fieldarray中支持split,int等handy的function
2. 重大更新,支持ElmoEmbedding, BertEmbedding
6 years ago
yh_cc
d71f0eef13
序列标注的SemiCRFRelay中文分词.
6 years ago
yunfan
9a5cc3801c
- update sst data loader
- add Option
6 years ago
ChenXin
96437f9e26
更新了一些文档格式
6 years ago
ChenXin
bdec6187a2
一些符合 PEP8 的微调
6 years ago
yh
aabbdb4df5
增加vocab的clear方法
6 years ago
ChenXin
32fdb48754
修改了 core 部分 import 的顺序,__all__ 暴露的内容
6 years ago
ChenXin
4926b33df0
core部分的测试和一些小修改
6 years ago
yunfan
443184f82e
- fix batch & vocab
6 years ago
ChenXin
cf08e00c69
全部改为相对路径引用
6 years ago
ChenXin
257eb2b9eb
修改了核心部分的一大部分文档,TODO:
1. 完善 trainer 和 tester 部分的文档
2. 研究注释样例与测试
6 years ago
yh_cc
5e9c406761
修改embedding为init_embed初始化
6 years ago
yh_cc
6e265e5ae9
完善了trainer,callback等的文档; 修改了部分代码的命名以使得代码从文档中隐藏
6 years ago
yunfan
001586fa3e
- add document
6 years ago
yh_cc
c1ee0b27df
1.DataSet.apply()报错时提供错误的index
2.Vocabulary.from_dataset(), index_dataset()提供报错时的vocab顺序
3.embedloader在embed读取时遇到不规则的数据跳过这一行.
6 years ago
yh_cc
b69f8985c8
1. 在embedding_loader中增加新的读取函数load_with_vocab(), load_without_vocab, 比之前的函数改变主要在(1)不再需要传入embed_dim(2)自动判断当前是word2vec还是glove.
2. vocabulary增加from_dataset(), index_dataset()函数。避免需要多行写index dataset的问题。
3. 在utils中新增一个cache_result()修饰器,用于cache函数的返回值。
4. callback中新增update_every属性
6 years ago
FengZiYjun
6a498bbdf2
* 给vocabulary添加遍历方法
6 years ago
FengZiYjun
f5ab7a5d45
* 将enas相关代码放到automl目录下
* 修复fast_param_mapping的一个bug
* Trainer添加自动创建save目录
* Vocabulary的打印,显示内容
6 years ago
yh
29eab18b78
1. CRF增加支持bmeso类型的tag 2. vocabulary中增加注释
6 years ago
FengZiYjun
bc7fc71faa
* update version number in doc
* update fastNLP.core.rst
* refine all docstrings in core/
7 years ago
FengZiYjun
f26f11608b
* 更新教程,放在在./tutorial
* remove unused codes in metrics.py
* add tests for DataSet
* add tests for FieldArray
* add tests for metrics.py
* fix predictor, add tests for predictor
* fix bucket sampler, add tests for bucket sampler
7 years ago
yunfan
52b1b18a76
fix bugs in vocab
7 years ago
FengZiYjun
6427e85e8f
升级Vocab:
* 增量添加单词到词典中
* lazy update: 当用到词典的时候才重新build
* 当新添加的词导致词典大小超出限制时,打印一个warning
Update Vocabulary:
* More words can be added after the building.
* Lazy update: rebuild automatically when vocab is used.
* print warning when max size is reached
7 years ago
FengZiYjun
837bef47dc
* add unit tests for instance, vocabulary
* remove and fix other unit tests
* add more code comments
7 years ago
yh
ce3b002263
check code init
7 years ago
yh
0292350c7a
vocabulary增加方法
7 years ago
yunfan
3a42c84a47
use counter in vocab, add a load func in baseloader
7 years ago
FengZiYjun
e9d7074ba1
* delete readme_example.py because it is oooooooout of date.
* rename preprocess.py into utils.py, because nothing about preprocess in it
* anything in loader/ and saver/ is moved directly into io/
* corresponding unit tests are moved to /test/io
* delete fastnlp.py, because we have new and better APIs
* rename Biaffine_parser/run_test.py to Biaffine_parser/main.py; Otherwise, test will fail.
* A looooooooooot of ancient codes to be refined...........
7 years ago
yunfan
053249420f
update parser, fix bugs varrnn & vocab
7 years ago
yunfan
102259df39
update biaffine parser
7 years ago
Coet
b80e5e8b29
Merge branch 'master' into dev
7 years ago
yunfan
ebbfcb7829
add dataset read functions
7 years ago
xuyige
b43d333738
clean some codes and fix some bugs
7 years ago
yunfan
1f680f24e5
update, fix bug
7 years ago
yunfan
2698094d8f
update embedding loader & vocab
7 years ago
FengZiYjun
0b86d7cf2b
Merge Preprocessor and DataSet
7 years ago
FengZiYjun
cb11a1f2dc
- analyze codes for language model, unable to run yet
- add character vocab in preprocessor
- add dataset loader for language model dataset
- other minor adjustments
- preserve only a little example data for language model
7 years ago
yunfan
e8cc702737
add default switch
7 years ago
yunfan
9c7f3cf261
add vocabulary into preprocessor
7 years ago
yunfan
3f4544759d
add unittest of data, fix bug
7 years ago