yao_yf
dc7dc7d3fa
dataset strategy set
4 years ago
Xiaoda Zhang
bb5d4212f7
enable All2All in infering redistribution ops
4 years ago
Xiaoda Zhang
04381273b3
Add the sharding propagation function:
1) users configure sharding strategies for operators;
2) framework will propagate the strategies from configured-ops to
non-configured ops using BFS;
3) the propagation goal is to minimize redistribution communication
cost;
5 years ago
yao_yf
d2dc22ff71
parallel_weight_init_adapt_pipeline_increment_predict
4 years ago
i-robot
b79400296a
!17959 fix optimizer weight shard config
Merge pull request !17959 from gziyan/add_optimizer_weight_shard_config
4 years ago
kswang
8aa0450b8d
set dtype for allreduce fusion
4 years ago
Ziyan
95ac0f6d58
fix optimizer weight shard config
4 years ago
Ziyan
2a752f24bf
enable not fully use opt shard
5 years ago
liujunzhu
6541b96c40
Add communication parallel mode.
5 years ago
yangzhenzhang
a70d616841
mini step grad accumulation
5 years ago
yangzhenzhang
7303c3d3b8
add group ckpt
5 years ago
yangzhenzhang
9da3f9bec9
mini step grad accumulation
5 years ago
yangzhenzhang
278e82a849
update pipeline parallel
5 years ago
lichenever
2e1c43483e
add auto parallel pipeline
5 years ago
Yi Huaijie
d7faa77b5e
support int64 shape
5 years ago
Xiaoda Zhang
fba2bfeb54
overwrite strategies for star graph structure
5 years ago
huangxinjing
4ef439e27b
Add stage information for ops and strategy
5 years ago
yao_yf
d4cfe55c04
rename mirror_mean to gradients_mean
5 years ago
yao_yf
8f7aa5bd5a
auto parallel context modify
5 years ago
Yi Huaijie
394be43492
raise RuntimeError when set different mode after Initializer created
5 years ago
Yi Huaijie
89a4ebf8a1
parallel mode must be set before create an initializer
5 years ago
liubuyu
a499d4e47c
decoupling core and debug
5 years ago
liubuyu
76dc80e7b7
Unified code style
5 years ago
liubuyu
43c79eb853
mindspore path adjust
5 years ago