yh_cc
af6a9da78d
Merge branch 'dev' of github.com:choosewhatulike/fastNLP-private into dev
7 years ago
yh_cc
6d36dbe7fb
完善测试
7 years ago
yunfan
702fa1d95c
- update attention
- fix tests
7 years ago
ChenXin
8039f4dd45
讨论并整合了若干模块
7 years ago
yunfan
e864aecb03
- add Const
- fix bugs
7 years ago
yh_cc
28ece53df0
本地暂存
7 years ago
yunfan
e025350ea8
Merge branch 'dev' of https://github.com/choosewhatulike/fastNLP-private into pr
7 years ago
yunfan
c344f7a2f9
- add pad sequence for lstm
- add csv, conll, json filereader
- update dataloader
- remove useless dataloader
- fix trainer loss print
- fix tests
7 years ago
yh_cc
c1ee0b27df
1.DataSet.apply()报错时提供错误的index
2.Vocabulary.from_dataset(), index_dataset()提供报错时的vocab顺序
3.embedloader在embed读取时遇到不规则的数据跳过这一行.
7 years ago
yh_cc
b69f8985c8
1. 在embedding_loader中增加新的读取函数load_with_vocab(), load_without_vocab, 比之前的函数改变主要在(1)不再需要传入embed_dim(2)自动判断当前是word2vec还是glove.
2. vocabulary增加from_dataset(), index_dataset()函数。避免需要多行写index dataset的问题。
3. 在utils中新增一个cache_result()修饰器,用于cache函数的返回值。
4. callback中新增update_every属性
7 years ago
yh_cc
29f81e79ad
准备发布0.4.0版本“
7 years ago
FengZiYjun
986541139a
整理所有dataset loader,建立单元测试
7 years ago
FengZiYjun
5d8f6960a7
* fix README figure
* refine code style
7 years ago
hazelnutsgz
5f4ab131ac
Add a loader for conll2003 dataset
7 years ago
FengZiYjun
27e9453d19
* fix processor.py
* add code comments
* merge *_saver.py & *_loader.py in io/
* (ancient codes) rename Loss into LossFromTorch
7 years ago
FengZiYjun
f62060339e
All tests pass. Ready to merge.
* 更新Loss的接口形参跟metric保持一致
* 添加对几种loss的测试
* embed_loader采用维度独立的方法采样
* 对应测试代码的修改
7 years ago
FengZiYjun
cc440b5ed6
All tests pass.
* 更新测试代码,跑通所有测试,覆盖率65%
* refine代码规范和某些注释
* fix tester self.use_cuda未赋值先使用的bug
* 添加tutorial样例数据——tutorial_sample_dataset.csv
* 【unsolved】embed_loader在计算np.cov时遇到segmentation fault
7 years ago
FengZiYjun
5824b7f4c7
跑通tutorial,修复一些bugs:
* dataset检查slice开始位置,确保结果不为空
* fieldarray检查content不为空
* optimizer接受的model params是一个generator,不能赋值
* code style refine
7 years ago
FengZiYjun
3120cdd09a
更新embed_loader:
* 添加fast_load_embedding方法,用vocab的词索引pre-trained中的embedding
* 如果vocab有词没出现在pre-train中,从已有embedding中正态采样
Update embed_loader:
* add fast_load_embedding method, to index pre-trained embedding with words in Vocab
* If words in Vocab are not exist in pre-trained, sample them from normal distribution computed by current embeddings
7 years ago
FengZiYjun
3d66975091
* refine code comments
* refine code style
* set up unit tests for Batch, DataSet, FieldArray
* remove a lot of out-of-date unit tests, to get testing passed
7 years ago
FengZiYjun
e9d7074ba1
* delete readme_example.py because it is oooooooout of date.
* rename preprocess.py into utils.py, because nothing about preprocess in it
* anything in loader/ and saver/ is moved directly into io/
* corresponding unit tests are moved to /test/io
* delete fastnlp.py, because we have new and better APIs
* rename Biaffine_parser/run_test.py to Biaffine_parser/main.py; Otherwise, test will fail.
* A looooooooooot of ancient codes to be refined...........
7 years ago