FengZiYjun
db0a789d61
* final clean up
* remove conflicts
* all tests passed
7 years ago
FengZiYjun
720a264eb3
* rename DataSet.get_fields() into get_all_fields()
* add DataSet.get_field(), to fetch a FieldArray based on its name
* remove old tutorials & add new tutorials
7 years ago
FengZiYjun
447746d9f5
* remove unused codes in losses.py & metrics.py
* refine code style
* fix tests
* add a new tutorial
7 years ago
FengZiYjun
27e9453d19
* fix processor.py
* add code comments
* merge *_saver.py & *_loader.py in io/
* (ancient codes) rename Loss into LossFromTorch
7 years ago
yh
306eee9690
Merge branch 'trainer' of github.com:FengZiYjun/fastNLP into trainer
7 years ago
yh
cd83866527
bug fix in LossInForward
7 years ago
FengZiYjun
6129a31c1d
* fix tests
* clean up unused codes
7 years ago
yh
aea931812b
1. trainer中losser修改为loss
7 years ago
yh
1158556236
1. 优化trainer checkcode过程的报错信息
7 years ago
yh
e779409cf8
conflict fix
7 years ago
yh
4dff3ec81f
修改losses中直接使用F.cross_entropy的情况,因为这些函数的signature是(input, target)
7 years ago
FengZiYjun
f26f11608b
* 更新教程,放在在./tutorial
* remove unused codes in metrics.py
* add tests for DataSet
* add tests for FieldArray
* add tests for metrics.py
* fix predictor, add tests for predictor
* fix bucket sampler, add tests for bucket sampler
7 years ago
yh
7c261faf19
Merge branch 'trainer' of github.com:FengZiYjun/fastNLP into trainer
7 years ago
yh
87e5d44b01
。
7 years ago
yunfan
52b1b18a76
fix bugs in vocab
7 years ago
yh
79ae38738f
conflict solved
7 years ago
yh
62c63f159a
test loss
7 years ago
FengZiYjun
27833d06ae
FieldArray only check type when is_input or is_target is set.
7 years ago
FengZiYjun
9acdb54fc8
优化loss在missing和duplicate时报错的信息:返回loss初始化约定接受的key
7 years ago
yh
abe5ec7261
Merge branch 'trainer' of github.com:FengZiYjun/fastNLP into trainer
# Conflicts:
# test/core/test_trainer.py
7 years ago
yh
a1a41c2d8b
1. unused报错运行错误修复
2. loss中修复一个错误
3. metric中fast_param调整
7 years ago
FengZiYjun
4b099bb0dd
* add tqdm in requirements.txt
* fix FieldArray type check bugs
7 years ago
FengZiYjun
661780b975
Improve FieldArray. Support nested list and a list of np.array
7 years ago
yh
1fb1df4a31
1. metric修改fast_param
2. trainer中update_every改为print_every, 因为update_every可能引起optimizer update的误解
3. fieldarray content支持使用np.ndarray初始化
7 years ago
yh
785c41ded5
Merge branch 'trainer' of github.com:FengZiYjun/fastNLP into trainer
# Conflicts:
# fastNLP/core/dataset.py
# fastNLP/core/trainer.py
# test/core/test_trainer.py
Trainer support print_train and tqdm train.
7 years ago
yh
beb55f5288
* change trainer iterating into tqdm
7 years ago
yunfan
ad3c5b6ef0
add magic iter in dataset
7 years ago
FengZiYjun
513876d5db
Updates:
* fix losses的_fast_param_map的bug
* Trainer添加sampelr初始化参数,并调整参数顺序
* refine codes
7 years ago
FengZiYjun
6f58ec34b4
Updates:
* DataSet修改__repr__,优化print(datset)的输出
* Instance修改__repr__,优化print的输出
* Optimizer优化传参提示
* Trainer去除kwargs参数
* losses.py加个参数
* 对应test code的修改
7 years ago
FengZiYjun
f62060339e
All tests pass. Ready to merge.
* 更新Loss的接口形参跟metric保持一致
* 添加对几种loss的测试
* embed_loader采用维度独立的方法采样
* 对应测试代码的修改
7 years ago
FengZiYjun
cc440b5ed6
All tests pass.
* 更新测试代码,跑通所有测试,覆盖率65%
* refine代码规范和某些注释
* fix tester self.use_cuda未赋值先使用的bug
* 添加tutorial样例数据——tutorial_sample_dataset.csv
* 【unsolved】embed_loader在计算np.cov时遇到segmentation fault
7 years ago
FengZiYjun
5824b7f4c7
跑通tutorial,修复一些bugs:
* dataset检查slice开始位置,确保结果不为空
* fieldarray检查content不为空
* optimizer接受的model params是一个generator,不能赋值
* code style refine
7 years ago
yh
b2e09b745e
Merge branch 'trainer' of github.com:FengZiYjun/fastNLP into trainer
7 years ago
yh
d19850b397
* add _fast_call_evaluate mechanism in MetricBase
7 years ago
FengZiYjun
11c82ab2e7
跑通test_trainer.py,联调结束,准备发布
7 years ago
FengZiYjun
201f5109d6
Updates:
* improve Loss initialization interface
* improve test codes for trainer
7 years ago
yh
8dd73a9205
Merge branch 'trainer' of github.com:FengZiYjun/fastNLP into trainer
7 years ago
yh
234ceb6fa3
fix bug in MetricBase
7 years ago
FengZiYjun
125c2718e4
Update
* fix bug in DataSet.split
* fix bugs in FieldArray, to allow content as a list
* fix bug in losses check
* ...
7 years ago
yh
c2d2137500
bug fix in MetricAccuracy
7 years ago
yh
8d7d2b428c
initial test for AccuracyMetric
7 years ago
FengZiYjun
fb5215ae73
fix bug in Trainer about metric_key
更新Optimizer: 多种初始化方法
1. SGD()
2. SGD(0.01)
3. SGD(lr=0.01)
4. SGD(lr=0.01, momentum=0.9)
5. SGD(model.parameters(), lr=0.1, momentum=0.9)
7 years ago
xuyige
ba7b17661c
Merge branch 'trainer' of https://github.com/FengZiYjun/fastNLP into check
7 years ago
xuyige
6d36190be4
update LossBase class
7 years ago
FengZiYjun
8a7077fed2
更新Optimizer:
optimizer.SGD(lr=xxx);如果没有传入parameters,则在trainer中帮他加入parameter
7 years ago
FengZiYjun
3120cdd09a
更新embed_loader:
* 添加fast_load_embedding方法,用vocab的词索引pre-trained中的embedding
* 如果vocab有词没出现在pre-train中,从已有embedding中正态采样
Update embed_loader:
* add fast_load_embedding method, to index pre-trained embedding with words in Vocab
* If words in Vocab are not exist in pre-trained, sample them from normal distribution computed by current embeddings
7 years ago
FengZiYjun
6427e85e8f
升级Vocab:
* 增量添加单词到词典中
* lazy update: 当用到词典的时候才重新build
* 当新添加的词导致词典大小超出限制时,打印一个warning
Update Vocabulary:
* More words can be added after the building.
* Lazy update: rebuild automatically when vocab is used.
* print warning when max size is reached
7 years ago
FengZiYjun
07e227aa4d
add interface of Loss
7 years ago
FengZiYjun
6839bb91cc
Add auto type detection/conversion in FieldArray
* In init, detect content type to be Python int, float, or str.
* In append(), check type consistence.
* In init & append(), int will be cast into float if they occur together.
* Map Python type into numpy dtype
* Raise error if type detection fails.
7 years ago
FengZiYjun
da901ed5b0
* DataSet __getitem__ returns copy of Instance
* refine interface of set_target & set_input
* rename DataSet.Instance into DataSet.DataSetIter
* remove unused methods in DataSet.DataSetIter
* remove __setattr__ in DataSet; It is dangerous.
* comment adjustment
7 years ago