Merge pull request !30953 from wanyiming/dynamic_wd

Merge pull request !31040 from Bert0108/reduce_operators_arg

Merge pull request !31041 from yangzhenzhang/add-check-for-conv2d

Merge pull request !30459 from liuluobin/ut_master

Merge pull request !30661 from Xiaoda/124-moe-changes

Merge pull request !30469 from wangjun/0223_pp

Merge pull request !30491 from jiahongQian/master

Merge pull request !30118 from zhuyuxiao/I4S85V

Merge pull request !29997 from Xiaoda/124-moe-changes

Merge pull request !30483 from lichen/pipeline_opt_detection

1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.

Merge pull request !30466 from yanghaoran/master

Merge pull request !30431 from jiahongQian/master

Merge pull request !30367 from yao_yf/auto_parallel_adasum_support_data_parallel

Merge pull request !30369 from lilei/add_virtualdataset_ut

Merge pull request !30294 from yao_yf/auto_parallel_adasum_checks_and_ut

Merge pull request !29820 from wangshengnan123/moe_topk_routing

Merge pull request !30167 from Bert0108/ut_conformer

Merge pull request !29612 from liuluobin/crop_resize_master