Merge pull request !30553 from h.farahat/reset

Merge pull request !27943 from harshvardhangupta/save_load_at_config

Merge pull request !22608 from Wangsong95/kitti_dataset

Merge pull request !30781 from cathwong/ckw_ut_fixup_map_python_multiproc

Merge pull request !30714 from markuskunej/offload_typecast

Merge pull request !30679 from cathwong/ckw_ut_map_python_multiproc

Merge pull request !30661 from Xiaoda/124-moe-changes

Merge pull request !30614 from huangxinjing/fix_cast_error

Merge pull request !30496 from huangbingjian/support_scipy

Merge pull request !18827 from Wangsong95/lfw_dataset

Merge pull request !30260 from danishfarid/re-enable_AT

Merge pull request !30469 from wangjun/0223_pp

Merge pull request !30491 from jiahongQian/master

tests re-enable

code check fix 1

remove self

lint fixing

lint fix 2

remove test as per req

Merge pull request !30118 from zhuyuxiao/I4S85V

Merge pull request !29997 from Xiaoda/124-moe-changes

Merge pull request !30483 from lichen/pipeline_opt_detection

1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.

Merge pull request !30259 from cathwong/ckw_at_save_api

Merge pull request !30466 from yanghaoran/master

Merge pull request !30431 from jiahongQian/master

Merge pull request !30367 from yao_yf/auto_parallel_adasum_support_data_parallel

Merge pull request !18738 from Wangsong95/lsun_dataset

Merge pull request !30369 from lilei/add_virtualdataset_ut

Merge pull request !29819 from huangxinjing/add_global_norm_search

Merge pull request !29705 from markuskunej/offload_multi_col_ds

Merge pull request !30189 from zhangzhaoju/master_outmost