Huawei_Technology
/
mindspore
Not watched
1
0
Fork 0
Code
Releases 13 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
21056 Commits
11 Branches
1.3 GB
314 Download
 
 
 
 
 
 
Tree: 37d88db4a4
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '37d88db4a4'
${ noResults }
mindspore/model_zoo/official/cv/vgg16
zhaoting 519de5c32b clean static checking 5 years ago
..
scripts vgg16+imagenet performance optimizing 5 years ago
src clean static checking 5 years ago
README.md 1. removed the link of apply form 5 years ago
README_CN.md 1. removed the link of apply form 5 years ago
eval.py remove device_id setting when use GPU 5 years ago
export.py fix GPU device_id bug 5 years ago
mindspore_hub_conf.py vgg16 hub support 5 years ago
train.py vgg16+imagenet performance optimizing 5 years ago

README.md

Contents

VGG Description

VGG, a very deep convolutional networks for large-scale image recognition, was proposed in 2014 and won the 1th place in object localization and 2th place in image classification task in ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).

Paper: Simonyan K, zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. arXiv preprint arXiv:1409.1556, 2014.

Model Architecture

VGG 16 network is mainly consisted by several basic modules (including convolution and pooling layer) and three continuous Dense layer.
here basic modules mainly include basic operation like: 3×3 conv and 2×2 max pooling.

Dataset

Note that you can run the scripts based on the dataset mentioned in original paper or widely used in relevant domain/network architecture. In the following sections, we will introduce how to run the scripts using the related dataset below.

Dataset used: CIFAR-10

  • CIFAR-10 Dataset size：175M，60,000 32*32 colorful images in 10 classes
    • Train：146M，50,000 images
    • Test：29.3M，10,000 images
  • Data format: binary files
    • Note: Data will be processed in src/dataset.py

Dataset used: ImageNet2012

  • Dataset size: ~146G, 1.28 million colorful images in 1000 classes
    • Train: 140G, 1,281,167 images
    • Test: 6.4G, 50, 000 images
  • Data format: RGB images
    • Note: Data will be processed in src/dataset.py

Dataset organize way

CIFAR-10

Unzip the CIFAR-10 dataset to any path you want and the folder structure should be as follows:

.
├── cifar-10-batches-bin  # train dataset
└── cifar-10-verify-bin   # infer dataset

ImageNet2012

Unzip the ImageNet2012 dataset to any path you want and the folder should include train and eval dataset as follows:

.
└─dataset
  ├─ilsvrc                # train dataset
  └─validation_preprocess # evaluate dataset

Features

Mixed Precision

The mixed precision training method accelerates the deep learning neural network training process by using both the single-precision and half-precision data formats, and maintains the network precision achieved by the single-precision training at the same time. Mixed precision training can accelerate the computation process, reduce memory usage, and enable a larger model or batch size to be trained on specific hardware.

For FP16 operators, if the input data type is FP32, the backend of MindSpore will automatically handle it with reduced precision. Users could check the reduced-precision operators by enabling INFO log and then searching ‘reduce precision’.

Environment Requirements

Quick Start

After installing MindSpore via the official website, you can start training and evaluation as follows:

  • Running on Ascend
# run training example
python train.py  --data_path=[DATA_PATH] --device_id=[DEVICE_ID] > output.train.log 2>&1 &

# run distributed training example
sh run_distribute_train.sh [RANL_TABLE_JSON] [DATA_PATH]

# run evaluation example
python eval.py --data_path=[DATA_PATH]  --pre_trained=[PRE_TRAINED] > output.eval.log 2>&1 &

For distributed training, a hccl configuration file with JSON format needs to be created in advance.
Please follow the instructions in the link below:
https://gitee.com/mindspore/mindspore/tree/master/model_zoo/utils/hccl_tools

  • Running on GPU
# run training example
python train.py --device_target="GPU" --device_id=[DEVICE_ID] --dataset=[DATASET_TYPE] --data_path=[DATA_PATH] > output.train.log 2>&1 &

# run distributed training example
sh run_distribute_train_gpu.sh [DATA_PATH]

# run evaluation example
python eval.py --device_target="GPU" --device_id=[DEVICE_ID] --dataset=[DATASET_TYPE] --data_path=[DATA_PATH]  --pre_trained=[PRE_TRAINED] > output.eval.log 2>&1 &

Script Description

Script and Sample Code

├── model_zoo
    ├── README.md                                 // descriptions about all the models
    ├── vgg16
        ├── README.md                             // descriptions about googlenet
        ├── scripts
        │   ├── run_distribute_train.sh           // shell script for distributed training on Ascend
        │   ├── run_distribute_train_gpu.sh       // shell script for distributed training on GPU
        ├── src
        │   ├── utils
        │   │   ├── logging.py                    // logging format setting
        │   │   ├── sampler.py                    // create sampler for dataset
        │   │   ├── util.py                       // util function
        │   │   ├── var_init.py                   // network parameter init method
        │   ├── config.py                         // parameter configuration
        │   ├── crossentropy.py                   // loss calculation
        │   ├── dataset.py                        // creating dataset
        │   ├── linear_warmup.py                  // linear leanring rate
        │   ├── warmup_cosine_annealing_lr.py     // consine anealing learning rate
        │   ├── warmup_step_lr.py                 // step or multi step learning rate
        │   ├──vgg.py                             // vgg architecture
        ├── train.py                              // training script
        ├── eval.py                               // evaluation script

Script Parameters

Training

usage: train.py [--device_target TARGET][--data_path DATA_PATH]
                [--dataset  DATASET_TYPE][--is_distributed VALUE]
                [--device_id DEVICE_ID][--pre_trained PRE_TRAINED]
                [--ckpt_path CHECKPOINT_PATH][--ckpt_interval INTERVAL_STEP]

parameters/options:
  --device_target       the training backend type, Ascend or GPU, default is Ascend.
  --dataset             the dataset type, cifar10 or imagenet2012.
  --is_distributed      the  way of traing, whether do distribute traing, value can be 0 or 1.
  --data_path           the storage path of dataset
  --device_id           the device which used to train model.
  --pre_trained         the pretrained checkpoint file path.
  --ckpt_path           the path to save checkpoint.
  --ckpt_interval       the epoch interval for saving checkpoint.

Evaluation

usage: eval.py [--device_target TARGET][--data_path DATA_PATH]
               [--dataset  DATASET_TYPE][--pre_trained PRE_TRAINED]
               [--device_id DEVICE_ID]

parameters/options:
  --device_target       the evaluation backend type, Ascend or GPU, default is Ascend.
  --dataset             the dataset type, cifar10 or imagenet2012.
  --data_path           the storage path of dataset.
  --device_id           the device which used to evaluate model.
  --pre_trained         the checkpoint file path used to evaluate model.

Parameter configuration

Parameters for both training and evaluation can be set in config.py.

  • config for vgg16, CIFAR-10 dataset
"num_classes": 10,                   # dataset class num
"lr": 0.01,                          # learning rate
"lr_init": 0.01,                     # initial learning rate
"lr_max": 0.1,                       # max learning rate
"lr_epochs": '30,60,90,120',         # lr changing based epochs
"lr_scheduler": "step",              # learning rate mode
"warmup_epochs": 5,                  # number of warmup epoch
"batch_size": 64,                    # batch size of input tensor
"max_epoch": 70,                     # only valid for taining, which is always 1 for inference
"momentum": 0.9,                     # momentum
"weight_decay": 5e-4,                # weight decay
"loss_scale": 1.0,                   # loss scale
"label_smooth": 0,                   # label smooth
"label_smooth_factor": 0,            # label smooth factor
"buffer_size": 10,                   # shuffle buffer size
"image_size": '224,224',             # image size
"pad_mode": 'same',                  # pad mode for conv2d
"padding": 0,                        # padding value for conv2d
"has_bias": False,                   # whether has bias in conv2d
"batch_norm": True,                  # whether has batch_norm in conv2d
"keep_checkpoint_max": 10,           # only keep the last keep_checkpoint_max checkpoint
"initialize_mode": "XavierUniform",  # conv2d init mode
"has_dropout": True                  # whether using Dropout layer
  • config for vgg16, ImageNet2012 dataset
"num_classes": 1000,                 # dataset class num
"lr": 0.01,                          # learning rate
"lr_init": 0.01,                     # initial learning rate
"lr_max": 0.1,                       # max learning rate
"lr_epochs": '30,60,90,120',         # lr changing based epochs
"lr_scheduler": "cosine_annealing",  # learning rate mode
"warmup_epochs": 0,                  # number of warmup epoch
"batch_size": 32,                    # batch size of input tensor
"max_epoch": 150,                    # only valid for taining, which is always 1 for inference
"momentum": 0.9,                     # momentum
"weight_decay": 1e-4,                # weight decay
"loss_scale": 1024,                  # loss scale
"label_smooth": 1,                   # label smooth
"label_smooth_factor": 0.1,          # label smooth factor
"buffer_size": 10,                   # shuffle buffer size
"image_size": '224,224',             # image size
"pad_mode": 'pad',                   # pad mode for conv2d
"padding": 1,                        # padding value for conv2d
"has_bias": True,                    # whether has bias in conv2d
"batch_norm": False,                 # whether has batch_norm in conv2d
"keep_checkpoint_max": 10,           # only keep the last keep_checkpoint_max checkpoint
"initialize_mode": "KaimingNormal",  # conv2d init mode
"has_dropout": True                  # whether using Dropout layer

Training Process

Training

Run vgg16 on Ascend

  • Training using single device(1p), using CIFAR-10 dataset in default
python train.py --data_path=your_data_path --device_id=6 > out.train.log 2>&1 &

The python command above will run in the background, you can view the results through the file out.train.log.

After training, you'll get some checkpoint files in specified ckpt_path, default in ./output directory.

You will get the loss value as following:

# grep "loss is " output.train.log
epoch: 1 step: 781, loss is 2.093086
epcoh: 2 step: 781, loss is 1.827582
...
  • Distributed Training
sh run_distribute_train.sh rank_table.json your_data_path

The above shell script will run distribute training in the background, you can view the results through the file train_parallel[X]/log.

You will get the loss value as following:

# grep "result: " train_parallel*/log
train_parallel0/log:epoch: 1 step: 97, loss is 1.9060308
train_parallel0/log:epcoh: 2 step: 97, loss is 1.6003821
...
train_parallel1/log:epoch: 1 step: 97, loss is 1.7095519
train_parallel1/log:epcoh: 2 step: 97, loss is 1.7133579
...
...

About rank_table.json, you can refer to the distributed training tutorial.
Attention This will bind the processor cores according to the device_num and total processor numbers. If you don't expect to run pretraining with binding processor cores, remove the operations about taskset in scripts/run_distribute_train.sh

Run vgg16 on GPU

  • Training using single device(1p)
python train.py  --device_target="GPU" --dataset="imagenet2012" --is_distributed=0 --data_path=$DATA_PATH  > output.train.log 2>&1 &
  • Distributed Training
# distributed training(8p)
bash scripts/run_distribute_train_gpu.sh /path/ImageNet2012/train"

Evaluation Process

Evaluation

  • Do eval as follows, need to specify dataset type as "cifar10" or "imagenet2012"
# when using cifar10 dataset
python eval.py --data_path=your_data_path --dataset="cifar10" --device_target="Ascend" --pre_trained=./*-70-781.ckpt > output.eval.log 2>&1 &

# when using imagenet2012 dataset
python eval.py --data_path=your_data_path --dataset="imagenet2012" --device_target="GPU" --pre_trained=./*-150-5004.ckpt > output.eval.log 2>&1 &
  • The above python command will run in the background, you can view the results through the file output.eval.log. You will get the accuracy as following:
# when using cifar10 dataset
# grep "result: " output.eval.log
result: {'acc': 0.92}

# when using the imagenet2012 dataset
after allreduce eval: top1_correct=36636, tot=50000, acc=73.27%
after allreduce eval: top5_correct=45582, tot=50000, acc=91.16%

Model Description

Performance

Training Performance

Parameters VGG16(Ascend) VGG16(GPU)
Model Version VGG16 VGG16
Resource Ascend 910 ；CPU 2.60GHz，192cores；Memory，755G NV SMX2 V100-32G
uploaded Date 10/28/2020 10/28/2020
MindSpore Version 1.0.0 1.0.0
Dataset CIFAR-10 ImageNet2012
Training Parameters epoch=70, steps=781, batch_size = 64, lr=0.1 epoch=150, steps=40036, batch_size = 32, lr=0.1
Optimizer Momentum Momentum
Loss Function SoftmaxCrossEntropy SoftmaxCrossEntropy
outputs probability probability
Loss 0.01 1.5~2.0
Speed 1pc: 79 ms/step; 8pcs: 104 ms/step 1pc: 81 ms/step; 8pcs 94.4ms/step
Total time 1pc: 72 mins; 8pcs: 11.8 mins 8pcs: 19.7 hours
Checkpoint for Fine tuning 1.1G(.ckpt file) 1.1G(.ckpt file)
Scripts vgg16

Evaluation Performance

Parameters VGG16(Ascend) VGG16(GPU)
Model Version VGG16 VGG16
Resource Ascend 910 GPU
Uploaded Date 10/28/2020 10/28/2020
MindSpore Version 1.0.0 1.0.0
Dataset CIFAR-10, 10,000 images ImageNet2012, 5000 images
batch_size 64 32
outputs probability probability
Accuracy 1pc: 93.4% 1pc: 73.0%;

Description of Random Situation

In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py.

ModelZoo Homepage

Please check the official homepage.

Delete task

Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.

cancel
confirm

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.

C++ Python Text Unity3D Asset C other