xingjun.wxj
|
1b4d5ccb9c
|
[to #42322933]MsDataset upload and load supports directory.
上传和下载支持多文件操作
|
3 years ago |
zhangzhicheng.zzc
|
d721fabb34
|
[to #42322933]bert with sequence classification / token classification/ fill mask refactor
1.新增支持原始bert模型(非easynlp的 backbone prefix版本)
2.支持bert的在sequence classification/fill mask /token classification上的backbone head形式
3.统一了sequence classification几个任务的pipeline到一个类
4.fill mask 支持backbone head形式
5.token classification的几个子任务(ner,word seg, part of speech)的preprocessor 统一到了一起TokenClassificationPreprocessor
6. sequence classification的几个子任务(single classification, pair classification)的preprocessor 统一到了一起SequenceClassificationPreprocessor
7. 改动register中 cls的group_key 赋值位置,之前的group_key在多个decorators的情况下,会被覆盖,obj_cls的group_key信息不正确
8. 基于backbone head形式将 原本group_key和 module同名的情况尝试做调整,如下在modelscope/pipelines/nlp/sequence_classification_pipeline.py 中
原本
@PIPELINES.register_module(
Tasks.sentiment_classification, module_name=Pipelines.sentiment_classification)
改成
@PIPELINES.register_module(
Tasks.text_classification, module_name=Pipelines.sentiment_classification)
相应的configuration.json也有改动,这样的改动更符合任务和pipline(子任务)的关系。
8. 其他相应改动为支持上述功能
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10041463
|
3 years ago |
wenmeng.zwm
|
6808e9a301
|
[to #44902099] add license for framework files
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10189613
|
3 years ago |
xingjun.wxj
|
7cb72cc46e
|
[to #42322933]MsDataset upload bugfix for 0830 version.
CR link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10109035#tab=changes&file=8348e8153b2f4a6dbd52e471b4980542355408ed
Please refer to aone links:
1. https://aone.alibaba-inc.com/v2/project/1162242/bug#viewIdentifier=b622c099e2199bc034401fbe&openWorkitemIdentifier=44889184
2. https://aone.alibaba-inc.com/v2/project/1162242/bug#viewIdentifier=b622c099e2199bc034401fbe&openWorkitemIdentifier=44858810
3. https://aone.alibaba-inc.com/v2/project/1162242/bug#viewIdentifier=b622c099e2199bc034401fbe&openWorkitemIdentifier=44857728
4. https://aone.alibaba-inc.com/v2/project/1162242/bug#viewIdentifier=b622c099e2199bc034401fbe&openWorkitemIdentifier=44658972
|
3 years ago |
shuying.shu
|
a9deb3895c
|
[to #42322933] movie scene segmentation模型接入
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9872869
|
3 years ago |
feiwu.yfw
|
2b64cf2bb6
|
[to #42322933]支持从dataset json文件中获取参数
* dataset json file add args
|
3 years ago |
feiwu.yfw
|
39485426e7
|
[to #42322933]:fix msdataset
* 修复了zip文件不同打包模式下返回路径错误问题。
* 修复了替换了数据集文件重新下载时校验失败问题。
* 修复dataset oss文件在 REUSE 模式下重复下载的问题。
* 修复了csv数据集的meta json文件中某个split的meta和file字段都为''时加载所有split失败的问题。
* 修复了不同版本datasets路径不一致的问题。
|
3 years ago |
xingjun.wxj
|
44033290d4
|
[to #42322933]MsDataset 支持上传数据集压缩包和meta
1. MsDataset支持upload数据文件(压缩包)
2. MsDataset支持clone和upload meta data
3. 使用MsDataset.load()下载数据集,支持web端显示数据集下载计数
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9831232
|
3 years ago |
feiwu.yfw
|
35548bd492
|
[to #43875101]
msdataset add coco dataset
unify taskdataset and ms dataset
fix hf datasets
|
3 years ago |
feiwu.yfw
|
743e876981
|
[to #43660556] msdataset数据集加载
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9552632
* load csv dataset from modelscoop
|
3 years ago |
feiwu.yfw
|
2c3875c0e1
|
[to #43299989] Fix msdataset
* fix msdataset
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9436292
* fix msdataset
|
3 years ago |
feiwu.yfw
|
5da470fd5d
|
[to #42791465, #42779255, #42777959, #42757844, #42756050, #42746916, #42743595, #42791863] fix: fix msdataset
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9174075
* fix msdataset
|
3 years ago |
yingda.chen
|
6702b29e21
|
[to #42794773]rename pydataset to msdataset
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9165402
|
3 years ago |