caorongyu.cry
35644fa0a7
[to #42322933 ] change star3 to space_T_cn
1. 合并star和star3框架
2. 修改star和star3的model type
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10492793
3 years ago
caorongyu.cry
9edfd7e50c
[to #42322933 ] update tableqa params
1. 增加传入table_id
2. 将result和table的结构统一
3. 默认开启is_use_sqlite
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10492027
3 years ago
zhangyanzhao.zyz
824ee8232c
[to #42322933 ]更新语义相关性任务英文名称为text ranking,修改对应变量名和类名
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10491951
3 years ago
yichang.zyc
683ee5bfed
[to #42322933 ]use Tasks.ocr_recognition
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10490937
3 years ago
yuze.zyz
f7f7eb21dc
[to #42322933 ]Fix the logic of fast tokenizer
1. Change the logic of using fast tokenizer from mode to user arguments and tokenizer_config.json
This is to fix the problem of RANER must use fast tokenizer in some special models.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10488982
3 years ago
liugao.lg
baff7c5b64
[to #42322933 ]add ofa-ocr-recogniiton pipeline
新增ofa关于日常场景文字识别的任务,主要包括:
1、新增pipeline及task名称定义;
2、新增pipeline、task、model及prepreocess核心类方法的代码逻辑;
3、其它同步修正的小细节逻辑;
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10471089
3 years ago
yuze.zyz
acba1786b0
[to #42322933 ] Fix bug in UT daily
1. Fix bugs in daily test
2. Fix a bug that the updating of lr is before the first time of updating of optimizer
TODO this will still cause warnings when GA is above 1
3. Remove the judgement of mode in text-classification's preprocessor to fit the base trainer(Bug)
Update some regression bins to fit the preprocessor
4. Update the regression tool to let outer code modify atol and rtol
5. Add the default metric for text-classification task
6. Remove the useless ckpt conversion method in bert to avoid the requirement of tf when loading modeling_bert
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10430764
3 years ago
shichen.fsc
1483c64638
[to #42322933 ] Fix ASR error when resample failed, and add all asr models UT, add apply-cmvn for pytorch models
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10465241
3 years ago
yongfei.zyf
6438c41144
[to #42322933 ]Support video url input processing
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10437340
3 years ago
hemu.zp
3b1f1a0252
[to #42322933 ] Add GPT3 tensor parallel inference
添加基于 Megatron-v3 的 GPT3 tensor 并行的推理代码
复用 DistributedPipeline 与 megatron-util
适用模型:1.3B/2.7B/13B 参数的 GPT-3 预训练生成模型
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10416721
3 years ago
hemu.zp
271e2a2a99
[to #42322933 ] Add gpt_neo model
1. 添加 gpt_neo 模型,因 checkpoint 归属于 Langboat 还未上传到模型库,已线下完成测试
2. 添加 text-generation task models 与 head,后续会将 gpt3,palm 等已上线文本生成模型统一为 backbone + head 结构的 task models
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10404249
3 years ago
shichen.fsc
542c4ce1b3
[to #42322933 ] Fix bug in KWS when setting customized keyword
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10412829
3 years ago
zhangzhicheng.zzc
0eb823b764
[to #42322933 ] support t5_with_translation
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10383770
* T5 support translate
3 years ago
caorongyu.cry
7145990054
[to #42322933 ] reivse model problem and remove history sql for demo
相比于master上的tableqa,做出了如下修复:
1. 修复了schema linking中的问题。
2. 同时设置了有history sql和没有history sql的两种输入
3. 增加了sqlite执行逻辑,可以返回sql执行结果
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10365114
3 years ago
zhangzhicheng.zzc
e240edea7e
[to #42322933 ]t5 bug fixex
3 years ago
yichang.zyc
a05f37f826
ofa clip add license
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10340433
3 years ago
zhangzhicheng.zzc
d721fabb34
[to #42322933 ]bert with sequence classification / token classification/ fill mask refactor
1.新增支持原始bert模型(非easynlp的 backbone prefix版本)
2.支持bert的在sequence classification/fill mask /token classification上的backbone head形式
3.统一了sequence classification几个任务的pipeline到一个类
4.fill mask 支持backbone head形式
5.token classification的几个子任务(ner,word seg, part of speech)的preprocessor 统一到了一起TokenClassificationPreprocessor
6. sequence classification的几个子任务(single classification, pair classification)的preprocessor 统一到了一起SequenceClassificationPreprocessor
7. 改动register中 cls的group_key 赋值位置,之前的group_key在多个decorators的情况下,会被覆盖,obj_cls的group_key信息不正确
8. 基于backbone head形式将 原本group_key和 module同名的情况尝试做调整,如下在modelscope/pipelines/nlp/sequence_classification_pipeline.py 中
原本
@PIPELINES.register_module(
Tasks.sentiment_classification, module_name=Pipelines.sentiment_classification)
改成
@PIPELINES.register_module(
Tasks.text_classification, module_name=Pipelines.sentiment_classification)
相应的configuration.json也有改动,这样的改动更符合任务和pipline(子任务)的关系。
8. 其他相应改动为支持上述功能
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10041463
3 years ago
yuze.zyz
c498d88d48
[to #42322933 ] add license declaration
1. add license declaration
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10216802
3 years ago
zhangzhicheng.zzc
5e4894870b
[to #42322933 ]add t5 model / text2text generation task
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10191736
* add T5 for generation
3 years ago
hemu.zp
69f8928dd2
[to #42322933 ] Replace mplug input 'question' with 'text'
mplug 相关任务 pipeline 输入字段统一为 'image' + 'text'
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10236282
3 years ago
shuying.shu
a22a4e9a3a
[to #42322933 ]add license header
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10222223
3 years ago
bin.xue
90b214b2e3
[to #42322933 ] add copyright information for audio modules
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10186837
3 years ago
zhangzhicheng.zzc
2c05a34924
[to #42322933 ] bug fix for fairseq
3 years ago
lingcai.wl
0548d92de8
[to #44657982 ] fix some demo problems
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10083156
3 years ago
caorongyu.cry
77cfcf0a9a
[to #42322933 ] commit nlp_convai_text2sql_pretrain_cn inference process to modelscope
commit nlp_convai_text2sql_pretrain_cn inference process to modelscope
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10025155
3 years ago
dingkun.ldk
54e1a6d88b
[to #42322933 ]830NLP 篇章排序/文本表示模型代码check
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9856179
3 years ago
zhangzhicheng.zzc
937d3ca67b
[to #42322933 ] bugs:circular dependency fixed/ word_segmentation output
3 years ago
ly119399
b41b10f897
[to #42322933 ] space finetune on generation task
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10061562
3 years ago
bin.xue
4be7737122
[to #42322933 ] audio pipelines accept url as input
3 years ago
cyc385202
d4759e4c24
[to #42322933 ] 加入space模型在banking数据集上的finetune代码
加入space模型在banking数据集上的微调代码
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10006792
3 years ago
shichen.fsc
a8fd9c4afe
[to #42322933 ] add new pipeline - PoNet for fill-mask
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10019083
3 years ago
dingkun.ldk
7ed4015bdc
[to #42322933 ]支持词性标注
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9980774
3 years ago
suluyan.sly
904374d329
[to #42322933 ] feat: plug inference
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9931748
3 years ago
peter.lx
780330897a
[to #42322933 ] add Deberta v2 modeling and fill_mask task, with master merged
add Deberta v2 modeling and fill_mask task, with master merged
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9966511
3 years ago
fubang.zfb
8f81807537
[to #42322933 ] 关系抽取
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9938140
3 years ago
yongfei.zyf
c8b6030b8e
[to #42322933 ] Add hicossl_video_embedding_pipeline to maas lib
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9969472
3 years ago
xuanjie.wxb
e5c9ded870
[to #42322933 ] add lstm-crf ner model code
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9901220
* add lstm-crf ner model code
3 years ago
shuying.shu
a9deb3895c
[to #42322933 ] movie scene segmentation模型接入
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9872869
3 years ago
shichen.fsc
2b380f0410
[to #42322933 ] add new Task - document segmentation
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9942858
* [Add] add document-segmentation
3 years ago
hemu.zp
a9089570e5
[to #42322933 ] Add mplug retrieval pipeline and finetune
支持 MPLUG 模型 image-text-retrieval 任务的 pipeline 和 finetune
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9919955
3 years ago
hemu.zp
20a935d406
[to #42322933 ] add gpt3 base finetune
添加 gpt3 中小模型单机单卡下的 finetune 代码
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9899004
3 years ago
tanfan.zjh
dc45fce542
[to #42322933 ]新增FAQ问答模型
Maas新增FAQ问答模型
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9797053
3 years ago
pangda
83b0adf0a2
[to #42322933 ] fix bug for multi-lang text
支持多语言tokenize(830模型)
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9900916
3 years ago
hemu.zp
b92e2ca0a0
[to #42322933 ] add vqa and caption finetuning for mplug
添加 mplug 模型 caption 及 vqa 任务的 finetuning 支持
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9858028
3 years ago
menrui.mr
427f0e83ea
[to #42322933 ]ofa文生图接入clip reranking后处理 & 修复预处理中的一个Bug
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9880918
3 years ago
james.wjg
5f326e46f3
cv/video_summarization
新增video summarization模型的inference和finetune
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9840532
3 years ago
hemu.zp
161555f97d
[to #42322933 ] add mplug image-caption
添加 MPLUG 模型 image-captioning 任务
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9745826
3 years ago
piaoyu.lxy
87290ed6f0
[to #42322933 ] fix punkt file
删除模型仓库中多余的文件,只保留punkt.zip文件,运行时解压
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9800149
3 years ago
Yingda Chen
f53b242332
[to #42322933 ] add onnx model and onnx constant
3 years ago
jiangnana.jnn
76482cc3ea
[to #43850241 ] fix processor and collate_fn
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9644184
* fix ditributed training and eval
3 years ago