Merge pull request !21805 from yangzhenzhang/auto-parallel-support-adafactor-opt

Merge pull request !21809 from huangxinjing/transformer_improved

Merge pull request !21761 from yao_yf/comm_recompute_interface

split attention inputs from two into three

Add input check and modify test case

Fix check error and add check to transformer inputs

add input check to the transformers

Merge pull request !21627 from zhoufeng/alltoall_exception

Signed-off-by: zhoufeng <zhoufeng54@huawei.com>

Merge pull request !21528 from huangxinjing/add_print

Merge pull request !20960 from lichen/add_replace_graph_for_conv2d

Merge pull request !20520 from lichen/add_op_AllToAllv

Merge pull request !20411 from gziyan/enable_opt_shard_predict

1) users configure sharding strategies for operators;
2) framework will propagate the strategies from configured-ops to
non-configured ops using BFS;
3) the propagation goal is to minimize redistribution communication
cost;

Merge pull request !16457 from lichen/pipeline_split_adapt_master

From: @gong_zi_yan
Reviewed-by: @yangzhenzhang,@stsuteng
Signed-off-by: @stsuteng

From: @hanyang001
Reviewed-by: @stsuteng,@yangzhenzhang
Signed-off-by: @stsuteng