Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10227608

# Conflicts:
#	modelscope/models/multi_modal/ofa_for_all_tasks.py
#	modelscope/msdatasets/ms_dataset.py
#	modelscope/trainers/utils/inference.py

# Conflicts:
#	modelscope/preprocessors/multi_modal.py
#	modelscope/trainers/trainer.py
#	tests/pipelines/test_ofa_tasks.py

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10185634

    * fix dist training

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10189613

1. Fix a bug in trainer's progress bar
2. Fix a bug that trainer does not support dataset in config file
3. Add feature: support go on training via checkpoint file
4. Add feature: support fixed filename when saving best checkpoint
5. Fix a bug that no id2label in config file after finetune of nlp models
6. Fix some other bugs
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10138906

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10142442

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10103248

    * update EasyCV MSDataset

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9856179

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10061562

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9935664

    * adapt to msdataset for EasyCV

加入space模型在banking数据集上的微调代码
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10006792

1. Fix random seed for trainer and init it at the first line of init
2. Add a regress test for fixed training
3. Change the dataset 'dureader_robust_qg' to 'DuReader_robust-QG'
4. Change some datasets from loading hf.datasets to loading msdataset.load
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10029509

1. 修复 palm，gpt3，mplug 模型存在的 finetune 后保存 checkpoint 与原有 checkpoint key 字段存在区别无法使用 from_pretrained 导入的问题
2. 调整 test_finetune_mplug.py 为只保存训练结束时的 checkpoint，减少 ci 耗时
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10016517

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10011826

、

1. Support saving the best checkpoint for inference
2. Fix a bug that _max_iters field does not exist in trainer
3. Fix a bug that function in lambda_lr field cannot be saved to file
4. Fix a bug that save_pretrained would not be called by iterating
5. Fix a bug that interval is not passed from BestCkptHook's init
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9972765

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9970626

    * fix trainer unittest

# Conflicts:
#	modelscope/preprocessors/multi_modal.py

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9872869

* dataset json file add args

1. add child-tuning optimizer and ut
2. fix a training bug which can cause interruption after cross-evaluation
3. move model.params from cfg to default args in build_optimizer to prevent the saving of params in save_pretrained
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9891963

支持 MPLUG 模型 image-text-retrieval 任务的 pipeline 和 finetune
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9919955

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9892528

* 修复了zip文件不同打包模式下返回路径错误问题。
* 修复了替换了数据集文件重新下载时校验失败问题。
* 修复dataset oss文件在 REUSE 模式下重复下载的问题。
* 修复了csv数据集的meta json文件中某个split的meta和file字段都为''时加载所有split失败的问题。
 * 修复了不同版本datasets路径不一致的问题。

添加 gpt3 中小模型单机单卡下的 finetune 代码
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9899004

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9781849

    * support EasyCV

添加 mplug 模型 caption 及 vqa 任务的 finetuning 支持
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9858028

新增video summarization模型的inference和finetune
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9840532

msdataset add coco dataset
unify taskdataset and ms dataset
fix hf datasets

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9791200

    * fix trainer about iters_per_epoch

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9763330

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9644184

    * fix ditributed training and eval

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9691512

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9662406

    * bug fix for nlp backbone-head trainers

* csanmt finetune wxp

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9660779

    * fix unittest

1. remove comments 
2. fix a bug that ws assert failure for english input
3. add an english input test for ws
3. remove a test case which the dataset can not be visited by outer website
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9657140

移除测试中 damo/nlp_palm2.0_text-generation_english-base beta 分支的使用

1. Refine the nlp finetune examples
2. Remove some useless code
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9631158