Merge pull request !30661 from Xiaoda/124-moe-changes

Merge pull request !30614 from huangxinjing/fix_cast_error

Merge pull request !30496 from huangbingjian/support_scipy

Merge pull request !18827 from Wangsong95/lfw_dataset

Merge pull request !30260 from danishfarid/re-enable_AT

Merge pull request !30469 from wangjun/0223_pp

Merge pull request !30491 from jiahongQian/master

tests re-enable

code check fix 1

remove self

lint fixing

lint fix 2

remove test as per req

Merge pull request !30118 from zhuyuxiao/I4S85V

Merge pull request !29997 from Xiaoda/124-moe-changes

Merge pull request !30483 from lichen/pipeline_opt_detection

1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.

Merge pull request !30259 from cathwong/ckw_at_save_api

Merge pull request !30466 from yanghaoran/master

Merge pull request !30431 from jiahongQian/master

Merge pull request !30367 from yao_yf/auto_parallel_adasum_support_data_parallel

Merge pull request !18738 from Wangsong95/lsun_dataset

Merge pull request !30369 from lilei/add_virtualdataset_ut

Merge pull request !29819 from huangxinjing/add_global_norm_search

Merge pull request !29705 from markuskunej/offload_multi_col_ds

Merge pull request !30189 from zhangzhaoju/master_outmost

Merge pull request !30301 from lianliguang/change-error-log

Merge pull request !30294 from yao_yf/auto_parallel_adasum_checks_and_ut

Merge pull request !25559 from 桂宁馨/Asin

The description of "outmost network" in error message instead by
method/function name

Merge pull request !30190 from huangxinjing/fix_ci_error

Merge pull request !26420 from 桂宁馨/Asinh

Merge pull request !30064 from changzherui/mod_ckpt_type_err

Merge pull request !30181 from lianliguang/change-dynshape-to-tensorshape

Merge pull request !29820 from wangshengnan123/moe_topk_routing