Merge pull request !32529 from huangxinjing/fix_model_limit

2. Add virutal assign add to the operator

Merge pull request !32341 from yangzhenzhang/add-parallel-operators

Merge pull request !31941 from lilei/modify_stridedslice_for_master

Merge pull request !31899 from FRHW-WANG/D-Rec-deliver

Merge pull request !31988 from liuluobin/ops_impl

Merge pull request !32005 from bichaoyang/master

Merge pull request !31164 from huangxinjing/fx_global_norm_error

Merge pull request !31252 from liuluobin/element_wise_ops

2. remove pattern match error as the origin pattern match will find no operator if there is only one parameter

Merge pull request !30953 from wanyiming/dynamic_wd

Merge pull request !31040 from Bert0108/reduce_operators_arg

Merge pull request !31041 from yangzhenzhang/add-check-for-conv2d

Merge pull request !30459 from liuluobin/ut_master

Merge pull request !30661 from Xiaoda/124-moe-changes

Merge pull request !30469 from wangjun/0223_pp

Merge pull request !30491 from jiahongQian/master

Merge pull request !30118 from zhuyuxiao/I4S85V

Merge pull request !29997 from Xiaoda/124-moe-changes

Merge pull request !30483 from lichen/pipeline_opt_detection

1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.

Merge pull request !30466 from yanghaoran/master

Merge pull request !30431 from jiahongQian/master