Merge pull request !27856 from yangzhenzhang/use-neighborexchangev2-for-conv2d

Merge pull request !27525 from yangzhenzhang/support-opt-parallel-for-adafactor

Merge pull request !27439 from Xiaoda/117-add-cumsum-op

Merge pull request !27428 from lilei/modify_virtualdataset_for_master

Merge pull request !27401 from jiahongQian/master

Merge pull request !27237 from lilei/modify_virtualdataset_for_master

Merge pull request !27251 from lilei/add_parallel_ut

Merge pull request !27024 from jiahongQian/master

However, to be forward compatitable, it is suitable to keep 'auto_parallel_search_mode' available. This PR recovers the 'auto_parallel_search_mode' interface and adds a warning when using this old interface.

This PR also deals with other codestyle things.

Merge pull request !26807 from Xiaoda/113-auto-parallel-search-mode-changes-to-search-mode

Merge pull request !26804 from yao_yf/virtual_dataset_avoid_auto_parallel

Merge pull request !26619 from 王志伟/parallel_ut_refactoring1

2) 'sharding_propagation' moves to 'search_mode';

Merge pull request !26006 from yao_yf/add_transformer_slice_activation_config

Merge pull request !26414 from yao_yf/fault_recover_by_mirror_group_fix_opt_shard

slice recompute activation

1) when propagate sharding strategy from one op to another, try to find the strategy with zero communication cost;
2) if there is no such strategy, find the strategy with minimum communication cost, and raise a warning;

Merge pull request !26350 from yangzhenzhang/add-check-for-resize-op

Merge pull request !26036 from yangzhenzhang/add-output-strategy-for-op-init

1) using 'swc index of strategy_cost_' as reshape's selected strategy;
2) when encountering reshape in BFS, select the 'swc index' with zero communication cost;
3) when encountering a reshape that is already visited, check whether there exists communication between reshape and current operator. It is OK if communication happens between two configured operators;
4) currently, two consecutive reshapes are not supported;
5) adjusting BFS structure in graph_costmodel.cc;
6) adjusting some code in step_auto_parallel.cc to avoid cyclomatic complexity.

Add support for no pipeline accugradient

Add delay tag for fusion op

Optimizer the visite order

add mirror for mini step control

Move the group to attributes

Add gradient_shard control for the mini step

Fix code stype

Fix ut description

Add interface

Merge pull request !25765 from TuDouNi/neighborExchangeV2

Merge pull request !24790 from zhuyuxiao/master

Merge pull request !24568 from zhuyuxiao/master