Browse Source

!9192 reset the default activation function for bert

From: @shibeiji
Reviewed-by: @c_34,@liangchenghui
Signed-off-by: @c_34
tags/v1.1.0
mindspore-ci-bot Gitee 5 years ago
parent
commit
bebc37b389
1 changed files with 1 additions and 1 deletions
  1. +1
    -1
      model_zoo/official/nlp/bert/src/config.py

+ 1
- 1
model_zoo/official/nlp/bert/src/config.py View File

@@ -103,7 +103,7 @@ if cfg.bert_network == 'large':
num_hidden_layers=24, num_hidden_layers=24,
num_attention_heads=16, num_attention_heads=16,
intermediate_size=4096, intermediate_size=4096,
hidden_act="fast_gelu",
hidden_act="gelu",
hidden_dropout_prob=0.1, hidden_dropout_prob=0.1,
attention_probs_dropout_prob=0.1, attention_probs_dropout_prob=0.1,
max_position_embeddings=512, max_position_embeddings=512,


Loading…
Cancel
Save