Merge pull request !30661 from Xiaoda/124-moe-changes

Merge pull request !30469 from wangjun/0223_pp

Merge pull request !30491 from jiahongQian/master

Merge pull request !30118 from zhuyuxiao/I4S85V

Merge pull request !29997 from Xiaoda/124-moe-changes

Merge pull request !30483 from lichen/pipeline_opt_detection

1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.

Merge pull request !30466 from yanghaoran/master

Merge pull request !30431 from jiahongQian/master

Merge pull request !30367 from yao_yf/auto_parallel_adasum_support_data_parallel

Merge pull request !30369 from lilei/add_virtualdataset_ut

Merge pull request !30294 from yao_yf/auto_parallel_adasum_checks_and_ut

Merge pull request !29820 from wangshengnan123/moe_topk_routing

Merge pull request !30167 from Bert0108/ut_conformer

Merge pull request !29612 from liuluobin/crop_resize_master

Merge pull request !29453 from yangzhenzhang/support-pad-mode-for-conv2d

Merge pull request !29505 from huangxinjing/fx_allgather_cast

Merge pull request !29421 from liuluobin/parallel_ops_master

Merge pull request !29087 from bichaoyang/master

Merge pull request !29265 from jiangzhenguang/resize_bilinear