Browse Source

update lstm readme

tags/v1.0.0
CaoJian 5 years ago
parent
commit
c465f2bf5c
3 changed files with 153 additions and 40 deletions
  1. +81
    -40
      model_zoo/official/nlp/lstm/README.md
  2. +37
    -0
      model_zoo/official/nlp/lstm/script/run_eval_cpu.sh
  3. +35
    -0
      model_zoo/official/nlp/lstm/script/run_train_cpu.sh

+ 81
- 40
model_zoo/official/nlp/lstm/README.md View File

@@ -58,6 +58,16 @@ LSTM contains embeding, encoder and decoder modules. Encoder module consists of
bash run_eval_gpu.sh 0 ./aclimdb ./glove_dir lstm-20_390.ckpt
```

- runing on CPU

```bash
# run training example
bash run_train_cpu.sh ./aclimdb ./glove_dir

# run evaluation example
bash run_eval_cpu.sh ./aclimdb ./glove_dir lstm-20_390.ckpt
```


# [Script Description](#contents)

@@ -69,14 +79,16 @@ LSTM contains embeding, encoder and decoder modules. Encoder module consists of
   ├── README.md # descriptions about LSTM
   ├── script
   │   ├── run_eval_gpu.sh # shell script for evaluation on GPU
   │   └── run_train_gpu.sh # shell script for training on GPU
   │   ├── run_eval_cpu.sh # shell script for evaluation on CPU
   │   ├── run_train_gpu.sh # shell script for training on GPU
   │   └── run_train_cpu.sh # shell script for training on CPU
   ├── src
   │   ├── config.py # parameter configuration
   │   ├── dataset.py # dataset preprocess
   │   ├── imdb.py # imdb dataset read script
   │   └── lstm.py # Sentiment model
   ├── eval.py # evaluation script
   └── train.py # training script
   ├── eval.py # evaluation script on both GPU and CPU
   └── train.py # training script on both GPU and CPU
```


@@ -154,60 +166,89 @@ config.py:

- Set options in `config.py`, including learning rate and network hyperparameters.

- Run `sh run_train_gpu.sh` for training.
- runing on GPU

Run `sh run_train_gpu.sh` for training.

``` bash
bash run_train_gpu.sh 0 ./aclimdb ./glove_dir
```

The above shell script will run distribute training in the background. You will get the loss value as following:
```shell
# grep "loss is " log.txt
epoch: 1 step: 390, loss is 0.6003723
epcoh: 2 step: 390, loss is 0.35312173
...
```

- runing on CPU

``` bash
bash run_train_gpu.sh 0 ./aclimdb ./glove_dir
```
Run `sh run_train_cpu.sh` for training.

The above shell script will run distribute training in the background. You will get the loss value as following:
```shell
# grep "loss is " log.txt
epoch: 1 step: 390, loss is 0.6003723
epcoh: 2 step: 390, loss is 0.35312173
...
```
``` bash
bash run_train_cpu.sh ./aclimdb ./glove_dir
```

The above shell script will train in the background. You will get the loss value as following:

```shell
# grep "loss is " log.txt
epoch: 1 step: 390, loss is 0.6003723
epcoh: 2 step: 390, loss is 0.35312173
...
```


## [Evaluation Process](#contents)

- Run `bash run_eval_gpu.sh` for evaluation.
- evaluation on GPU

Run `bash run_eval_gpu.sh` for evaluation.

``` bash
bash run_eval_gpu.sh 0 ./aclimdb ./glove_dir lstm-20_390.ckpt
```
``` bash
bash run_eval_gpu.sh 0 ./aclimdb ./glove_dir lstm-20_390.ckpt
```

- evaluation on CPU

Run `bash run_eval_cpu.sh` for evaluation.

``` bash
bash run_eval_cpu.sh ./aclimdb ./glove_dir lstm-20_390.ckpt
```

# [Model Description](#contents)
## [Performance](#contents)

### Training Performance

| Parameters | LSTM |
| -------------------------- | -------------------------------------------------------------- |
| Resource | Tesla V100-SMX2-16GB |
| uploaded Date | 08/06/2020 (month/day/year) |
| MindSpore Version | 0.6.0-beta |
| Dataset | aclimdb_v1 |
| Training Parameters | epoch=20, batch_size=64 |
| Optimizer | Momentum |
| Loss Function | Softmax Cross Entropy |
| Speed | 1022 (1pcs) |
| Loss | 0.12 |
| Params (M) | 6.45 |
| Checkpoint for inference | 292.9M (.ckpt file) |
| Scripts | https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/nlp/lstm |
| Parameters | LSTM (GPU) | LSTM (CPU) |
| -------------------------- | -------------------------------------------------------------- | -------------------------- |
| Resource | Tesla V100-SMX2-16GB | Ubuntu X86-i7-8565U-16GB |
| uploaded Date | 08/06/2020 (month/day/year) | 08/06/2020 (month/day/year)|
| MindSpore Version | 0.6.0-beta | 0.6.0-beta |
| Dataset | aclimdb_v1 | aclimdb_v1 |
| Training Parameters | epoch=20, batch_size=64 | epoch=20, batch_size=64 |
| Optimizer | Momentum | Momentum |
| Loss Function | Softmax Cross Entropy | Softmax Cross Entropy |
| Speed | 1022 (1pcs) | 20 |
| Loss | 0.12 | 0.12 |
| Params (M) | 6.45 | 6.45 |
| Checkpoint for inference | 292.9M (.ckpt file) | 292.9M (.ckpt file) |
| Scripts | [lstm script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/nlp/lstm) | [lstm script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/nlp/lstm) |


### Evaluation Performance

| Parameters | LSTM |
| ------------------- | --------------------------- |
| Resource | Tesla V100-SMX2-16GB |
| uploaded Date | 08/06/2020 (month/day/year) |
| MindSpore Version | 0.6.0-beta |
| Dataset | aclimdb_v1 |
| batch_size | 64 |
| Accuracy | 84% |
| Parameters | LSTM (GPU) | LSTM (CPU) |
| ------------------- | --------------------------- | ---------------------------- |
| Resource | Tesla V100-SMX2-16GB | Ubuntu X86-i7-8565U-16GB |
| uploaded Date | 08/06/2020 (month/day/year) | 08/06/2020 (month/day/year) |
| MindSpore Version | 0.6.0-beta | 0.6.0-beta |
| Dataset | aclimdb_v1 | aclimdb_v1 |
| batch_size | 64 | 64 |
| Accuracy | 84% | 83% |


# [Description of Random Situation](#contents)


+ 37
- 0
model_zoo/official/nlp/lstm/script/run_eval_cpu.sh View File

@@ -0,0 +1,37 @@
#!/bin/bash
# Copyright 2020 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================

echo "=============================================================================================================="
echo "Please run the scipt as: "
echo "bash run_eval_cpu.sh ACLIMDB_DIR GLOVE_DIR CKPT_FILE"
echo "for example: bash run_eval_cpu.sh ./aclimdb ./glove_dir lstm-20_390.ckpt"
echo "=============================================================================================================="

ACLIMDB_DIR=$1
GLOVE_DIR=$2
CKPT_FILE=$3

mkdir -p ms_log
CUR_DIR=`pwd`
export GLOG_log_dir=${CUR_DIR}/ms_log
export GLOG_logtostderr=0
python eval.py \
--device_target="CPU" \
--aclimdb_path=$ACLIMDB_DIR \
--glove_path=$GLOVE_DIR \
--preprocess=false \
--preprocess_path=./preprocess \
--ckpt_path=$CKPT_FILE > log.txt 2>&1 &

+ 35
- 0
model_zoo/official/nlp/lstm/script/run_train_cpu.sh View File

@@ -0,0 +1,35 @@
#!/bin/bash
# Copyright 2020 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================

echo "=============================================================================================================="
echo "Please run the scipt as: "
echo "bash run_train_cpu.sh ACLIMDB_DIR GLOVE_DIR"
echo "for example: bash run_train_gpu.sh ./aclimdb ./glove_dir"
echo "=============================================================================================================="

ACLIMDB_DIR=$1
GLOVE_DIR=$2

mkdir -p ms_log
CUR_DIR=`pwd`
export GLOG_log_dir=${CUR_DIR}/ms_log
export GLOG_logtostderr=0
python train.py \
--device_target="CPU" \
--aclimdb_path=$ACLIMDB_DIR \
--glove_path=$GLOVE_DIR \
--preprocess=true \
--preprocess_path=./preprocess > log.txt 2>&1 &

Loading…
Cancel
Save