chenhaozhe
15d37e5db9
imporve convergence of loss in bert
4 years ago
wangnan39@huawei.com
7188a14215
modify api detect_overflow name in TrainOneStepWithLossScaleCell
4 years ago
wangnan39@huawei.com
4d69e21933
optimizer class TrainOneStepWithLossScaleCell
4 years ago
He Wei
7d9a783993
[auto-monad] Support side-effects by auto-monad
The basic idea is: exploits data dependency to control the execution order
of side-effect operations, and keep the semantics of ANF unchanged.
The ControlDepend primitive is removed and there are two primitives added:
1. UpdateState:
```
a = Assign(para, value)
```
became:
```
a = Assign(para, value, u)
u = UpdateState(u, a)
```
2. Load:
```
x = Add(para, value)
```
became:
```
p = Load(para, u)
x = Add(p, value)
u = UpdateState(u, p)
```
4 years ago
mindspore-ci-bot
3bf5681ebc
!12093 adapt api for optimizer import
From: @sl_wang
Reviewed-by: @wang_zi_dong
Signed-off-by:
4 years ago
mwang
8f8eee4b5e
bert thor supports lr configuration in config.py
4 years ago
mindspore-ci-bot
51e7b6481a
!12034 modify instructions of preprocessing msra dataset
From: @shibeiji
Reviewed-by: @c_34,@guoqi1024
Signed-off-by: @c_34
4 years ago
mindspore-ci-bot
1c12b84757
!11992 [Bert][Gpu]Sync modify of bert script from r1.1 to master
From: @hanhuifeng2020
Reviewed-by: @gaoxiong1,@anyrenwei
Signed-off-by: @anyrenwei
4 years ago
shibeiji
95ccebb811
fix bugs of MSRA dataset preprocessing
4 years ago
mindspore-ci-bot
ed4207f698
!12022 Add a new optimizer THOR option to BERT pretrain script.
From: @i_yangmh
Reviewed-by: @wang_zi_dong,@kisnwang
Signed-off-by: @kisnwang
4 years ago
mwang
0b312c2385
thor generalization code submit
4 years ago
MingHan-Y
67a4c62b4b
Add new optimizer THOR option to BERT pretrain script.
4 years ago
hanhuifeng2020
53d4510ea6
[Bert][Gpu]Sync modify of bert script from r1.1 to master
4 years ago
mindspore-ci-bot
e897eb4c41
!11915 Change TensorAdd to Add, merge from r1.1 to master
From: @liangzhibo
Reviewed-by: @ginfung,@zh_qh
Signed-off-by: @zh_qh
4 years ago
mindspore-ci-bot
df265b6d6b
!11912 update submoudle akg, close graph kernel ascend ci testcases
From: @looop5
Reviewed-by:
Signed-off-by:
4 years ago
l00591931
9ec100d069
Change TensorAdd to Add, from r1.1 to master
4 years ago
looop5
0161209e40
update submoudle akg, close graph kernel ascend ci testcases
4 years ago
shibeiji
cc18b206c9
bert ner for adaption of MSRA dataset
4 years ago
mindspore-ci-bot
9fa0499fa0
Change GatherV2 to Gather r1.1 to master
4 years ago
shibeiji
8b1a8a6bc1
bert ner for msra dataset
4 years ago
shibeiji
3ec3f038ad
extract bert embedding tables in construct
4 years ago
chenhaozhe
4899a0b4c1
fix code spell
4 years ago
chenhaozhe
8eae3262f3
fix some description about bert and yolov3
4 years ago
mindspore-ci-bot
52953f16fc
!11027 Use logic id instead of physic id in get_distribute_pretrain_cmd.py
From: @c_34
Reviewed-by: @ljl0711,@liangchenghui
Signed-off-by: @liangchenghui
4 years ago
mindspore-ci-bot
30560be800
!11066 GPU add restrict for bert script
From: @VectorSL
Reviewed-by: @gaoxiong1,@dylangeng,@anyrenwei
Signed-off-by: @gaoxiong1
4 years ago
chenhaozhe
e2b740e812
add physic_logic_ids in get_distribute_pretrain_cmd.py
change ways to get physic_logic_ids
4 years ago
VectorSL
c13cd24e38
add restrict for gpu only
4 years ago
mindspore-ci-bot
84303c539f
!11003 upload chinese readme
From: @yoonlee666
Reviewed-by:
Signed-off-by:
4 years ago
yoonlee666
2d1e8beec7
add chinese readme
4 years ago
VectorSL
33277fe64b
fix bert init
4 years ago
shibeiji
f0b08e8bff
all reduce after each step in gradients accumulation mode for bert
5 years ago
mindspore-ci-bot
811a785555
!10886 GPU update bert scripts
From: @VectorSL
Reviewed-by: @gaoxiong1,@c_34,@gaoxiong1
Signed-off-by: @c_34
4 years ago
VectorSL
0c97835662
update control flow int adamweightdecay for bert
4 years ago
mindspore-ci-bot
02c107b0e9
!10874 修改GPU设置device_id的问题
From: @yuzhenhua666
Reviewed-by: @c_34,@linqingke
Signed-off-by: @c_34
4 years ago
yuzhenhua
6b858480c8
fix GPU device_id bug
4 years ago
shibeiji
812b4b0eab
extract embedding table from unified interface
5 years ago
hanhuifeng2020
65bfefd92b
bert on gpu for pre training script supports loss scale
5 years ago
Xiao Tianci
31fed1a2f6
change code to import APIs from mindspore.dataset rather than mindspore.dataset.engine
5 years ago
yuzhenhua
4128fb1155
modify export to support mindir
5 years ago
hanhuifeng2020
3988376b67
Performance optimization of Bert on GPU by the graph_kernel
5 years ago
wilfChen
62652cc29d
bert readme update
5 years ago
mindspore-ci-bot
59ca2ac708
!9693 add multi machine instruction for bert
From: @yoonlee666
Reviewed-by:
Signed-off-by:
5 years ago
yoonlee666
fe9443bfc6
add multi-machine
5 years ago
tronzhang
17d6f1c2f9
add option for graph kernel and mixed precision
5 years ago
mindspore-ci-bot
7131707359
!9366 fix bert export bug
From: @yuzhenhua666
Reviewed-by: @c_34
Signed-off-by: @c_34
5 years ago
yuzhenhua
0d1f209d69
fix bert export bug
5 years ago
mindspore-ci-bot
0c7ba7a7fa
!9354 mode_mindir_suffix
From: @bai-yangfan
Reviewed-by: @c_34,@guoqi1024
Signed-off-by: @c_34
5 years ago
bai-yangfan
c46c4dffe4
mindir_suffix
5 years ago
caozhou
b1189cd118
compatible name
5 years ago
yoonlee666
a744ef9113
add squad
5 years ago