FengZiYjun
7ecd8c9c14
finish POS tagging API
7 years ago
FengZiYjun
887c6fec70
add pos-tag training script
7 years ago
yh
e267f925cf
新增cws的train_context.py
7 years ago
yh
897c43fc3b
1. CRF增加constrain, 用于限制跃迁,比如BMES中B不能跃迁到S
2. metric增加SpanFMetric,可以用于计算sequence labelling的performance
3. 分词复现任务根据新版接口做了部分调整。
7 years ago
FengZiYjun
27e9453d19
* fix processor.py
* add code comments
* merge *_saver.py & *_loader.py in io/
* (ancient codes) rename Loss into LossFromTorch
7 years ago
yunfan
2aaa381827
refine git commits
7 years ago
yunfan
d643a7a894
update set_target, batch's as_numpy
7 years ago
FengZiYjun
090f7aef5b
* fixing unit tests
7 years ago
yh
1d5bb0a3b6
bug fix“
7 years ago
yh
8906155ca2
为api建立一个Analyzer
7 years ago
FengZiYjun
e9d7074ba1
* delete readme_example.py because it is oooooooout of date.
* rename preprocess.py into utils.py, because nothing about preprocess in it
* anything in loader/ and saver/ is moved directly into io/
* corresponding unit tests are moved to /test/io
* delete fastnlp.py, because we have new and better APIs
* rename Biaffine_parser/run_test.py to Biaffine_parser/main.py; Otherwise, test will fail.
* A looooooooooot of ancient codes to be refined...........
7 years ago
yunfan
b6a0d33cb1
add parser api
7 years ago
yh
77786509df
pos与cws开发上传
7 years ago
yh
7d97e9365d
增加新的processor“
7 years ago
yh
1496031182
新增pos output processor
7 years ago
yh
d5afffee73
新增端到端pos处理到parser的过度代码
7 years ago
yh_cc
10379e9c74
当前为segapp的方式,但是貌似准确率不行,尝试修改为crf 4tag模式试一试
7 years ago
yunfan
822aaf6286
fix and update tester, trainer, seq_model, add parser pipeline builder
7 years ago
FengZiYjun
4be15a5b43
保存pos tag 脚本
7 years ago
yh
9667c524a4
基本完善了cws的predict
7 years ago
yh
9fc20ac7b8
增加infer的pipeline
7 years ago
yh
dc7f8ef8d4
bug fix
7 years ago
yh_cc
7df33b23ea
Merge branch 'dataset' of github.com:yhcc/fastNLP into dataset
7 years ago
FengZiYjun
e2b14ed33d
Merge remote-tracking branch 'origin/dataset' into dataset
7 years ago
FengZiYjun
5dd0f74d6d
- 添加pos_tagger API, pipeline跑通
- 修复processor的bug
- 更新core/的若干组件, 去除batch的冗余参数
- CRF有个打字错误?已修复
- 更新pos tag 训练脚本
7 years ago
yh_cc
3e50ca8a72
创建了一个测试context
7 years ago
yh_cc
de3feeaf5a
调整CWS函数的位置
7 years ago
yh_cc
752efc57fd
Merge branch 'dataset' of github.com:yhcc/fastNLP into dataset
7 years ago
yh_cc
ea1c8c1100
当前版本分词准确率已达正常分词分数
7 years ago
FengZiYjun
ec9fd32d60
improve trainer: log mean and std of model params, and sum of gradients
7 years ago
FengZiYjun
cd68d78d50
Merge remote-tracking branch 'origin/dataset' into dataset
# Conflicts:
# fastNLP/api/pipeline.py
# fastNLP/api/pos_tagger.py
# fastNLP/api/processor.py
# fastNLP/modules/decoder/CRF.py
7 years ago
FengZiYjun
26e3abdf58
- 修改pos tag训练脚本,可以跑
- 在api中创建converter.py
- Pipeline添加初始化方法,方便一次性添加processors
- 删除pos_tagger.py
- 优化整体code style
7 years ago
yh_cc
10bb2810ab
Merge branch 'dataset' of github.com:yhcc/fastNLP into dataset
7 years ago
yh_cc
73ba3b5eec
bug fix for pipeline
7 years ago
yunfan
a6ab34fd38
fix crf
7 years ago
yh_cc
3cb98ddcf2
Sampler中增加了一个BucketSampler, CWS的训练基本可以实现
7 years ago
yh_cc
69a138eb18
修改了遇到的若干问题,增加了分词任务的一些方法
7 years ago
yh_cc
dc0124cf02
修改model到models
7 years ago
yh_cc
25a53ac5c9
修改processor适配昨天的sao操作
7 years ago
yh
d818e91380
增加dataset自动创建对应的array
7 years ago
yh
515e4f4987
移动processor到processor.py
7 years ago
yh
89ce85b6ed
Merge branch 'dataset' of https://github.com/yhcc/fastNLP into dataset
7 years ago
yh
38aa207ea2
新增cws converter, io
7 years ago
FengZiYjun
12e9a93b52
Merge remote-tracking branch 'origin/dataset' into dataset
7 years ago
FengZiYjun
79105381f5
- add interfaces for pos_tagging API
- update predictor.py to remove unused methods
- update model_loader.py & model_saver.py to support entire model saving & loading
- update pos tagging training script
7 years ago
yh
1b9daa1985
新增CWS的部分功能
7 years ago
yunfan
053249420f
update parser, fix bugs varrnn & vocab
7 years ago
yunfan
3192c9ac66
update trainer
7 years ago
yunfan
c14d9f4d66
update biaffine
7 years ago
yunfan
830d223344
add transformer
7 years ago