!8635 Add SqueezeNet to model zoo

From: @GreyZzzzzzXh Reviewed-by: Signed-off-by:
5 years ago · 22528d117a
--- a/model_zoo/official/cv/squeezenet/README.md
+++ b/model_zoo/official/cv/squeezenet/README.md
@@ -0,0 +1,690 @@
 # Contents

 - [SqueezeNet Description](#squeezenet-description)
 - [Model Architecture](#model-architecture)
 - [Dataset](#dataset)
 - [Features](#features)
    - [Mixed Precision](#mixed-precision)
 - [Environment Requirements](#environment-requirements)
 - [Quick Start](#quick-start)    
 - [Script Description](#script-description)
    - [Script and Sample Code](#script-and-sample-code)
    - [Script Parameters](#script-parameters)
    - [Training Process](#training-process)
    - [Evaluation Process](#evaluation-process)
 - [Model Description](#model-description)
    - [Performance](#performance)  
        - [Evaluation Performance](#evaluation-performance)
        - [Inference Performance](#inference-performance)
    - [How to use](#how-to-use)
        - [Inference](#inference) 
        - [Continue Training on the Pretrained Model](#continue-training-on-the-pretrained-model)
       - [Transfer Learning](#transfer-learning)
 - [Description of Random Situation](#description-of-random-situation)
 - [ModelZoo Homepage](#modelzoo-homepage)


 # [SqueezeNet Description](#contents)

 SqueezeNet is a lightweight and efficient CNN model proposed by Han et al., published in ICLR-2017. SqueezeNet has 50x fewer parameters than AlexNet, but the model performance (accuracy) is close to AlexNet.

 These are examples of training SqueezeNet/SqueezeNet_Residual with CIFAR-10/ImageNet dataset in MindSpore. SqueezeNet_Residual adds residual operation on the basis of SqueezeNet, which can improve the accuracy of the model without increasing the amount of parameters.

 [Paper](https://arxiv.org/abs/1602.07360):  Forrest N. Iandola and Song Han and Matthew W. Moskewicz and Khalid Ashraf and William J. Dally and Kurt Keutzer. "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size"


 # [Model Architecture](#contents)

 SqueezeNet is composed of fire modules. A fire module mainly includes two layers of convolution operations: one is the squeeze layer using a **1x1 convolution** kernel; the other is an expand layer using a mixture of **1x1** and **3x3 convolution** kernels.

 # [Dataset](#contents)

 Dataset used: [CIFAR-10](<http://www.cs.toronto.edu/~kriz/cifar.html>) 

 - Dataset size：175M，60,000 32*32 colorful images in 10 classes
  - Train：146M，50,000 images  
  - Test：29M，10,000 images 
 - Data format：binary files
  - Note：Data will be processed in src/dataset.py

 Dataset used: [ImageNet2012](http://www.image-net.org/)

 - Dataset size: 125G, 1250k colorful images in 1000 classes
 	- Train: 120G, 1200k images
 	- Test: 5G, 50k images
 - Data format: RGB images.
 	- Note: Data will be processed in src/dataset.py 

 # [Features](#contents)

 ## Mixed Precision

 The [mixed precision](https://www.mindspore.cn/tutorial/training/en/master/advanced_use/enable_mixed_precision.html) training method accelerates the deep learning neural network training process by using both the single-precision and half-precision data formats, and maintains the network precision achieved by the single-precision training at the same time. Mixed precision training can accelerate the computation process, reduce memory usage, and enable a larger model or batch size to be trained on specific hardware. 
 For FP16 operators, if the input data type is FP32, the backend of MindSpore will automatically handle it with reduced precision. Users could check the reduced-precision operators by enabling INFO log and then searching ‘reduce precision’.



 # [Environment Requirements](#contents)

 - Hardware（Ascend/GPU）
  - Prepare hardware environment with Ascend or GPU processor. If you want to try Ascend  , please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to ascend@huawei.com. Once approved, you can get the resources. 
 - Framework
  - [MindSpore](https://www.mindspore.cn/install/en)
 - For more information, please check the resources below：
  - [MindSpore Tutorials](https://www.mindspore.cn/tutorial/training/en/master/index.html)
  - [MindSpore Python API](https://www.mindspore.cn/doc/api_python/en/master/index.html)



 # [Quick Start](#contents)

 After installing MindSpore via the official website, you can start training and evaluation as follows: 

 - runing on Ascend

  ```
  # distributed training
  Usage: sh scripts/run_distribute_train.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [RANK_TABLE_FILE] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)

  # standalone training
  Usage: sh scripts/run_standalone_train.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)

  # run evaluation example
  Usage: sh scripts/run_eval.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [CHECKPOINT_PATH]
  ```

 - running on GPU

  ```
  # distributed training example
  sh scripts/run_distribute_train_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)

  # standalone training example
  sh scripts/run_standalone_train_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)

  # run evaluation example
  sh scripts/run_eval_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [CHECKPOINT_PATH]
  ```



 # [Script Description](#contents)

 ## [Script and Sample Code](#contents)

 ```
 .
 └── squeezenet
  ├── README.md
  ├── scripts
    ├── run_distribute_train.sh            # launch ascend distributed training(8 pcs)
    ├── run_standalone_train.sh            # launch ascend standalone training(1 pcs)
    ├── run_distribute_train_gpu.sh        # launch gpu distributed training(8 pcs)
    ├── run_standalone_train_gpu.sh        # launch gpu standalone training(1 pcs)
    ├── run_eval.sh                        # launch ascend evaluation
    └── run_eval_gpu.sh                    # launch gpu evaluation
  ├── src
    ├── config.py                          # parameter configuration
    ├── dataset.py                         # data preprocessing
    ├── CrossEntropySmooth.py              # loss definition for ImageNet dataset
    ├── lr_generator.py                    # generate learning rate for each step
    └── squeezenet.py                      # squeezenet architecture, including squeezenet and squeezenet_residual
  ├── train.py                             # train net
  ├── eval.py                              # eval net
  └── export.py                            # export checkpoint files into geir/onnx
 ```

 ## [Script Parameters](#contents)

 Parameters for both training and evaluation can be set in config.py

 - config for SqueezeNet, CIFAR-10 dataset

  ```py
  "class_num": 10,                  # dataset class num
  "batch_size": 32,                 # batch size of input tensor
  "loss_scale": 1024,               # loss scale
  "momentum": 0.9,                  # momentum
  "weight_decay": 1e-4,             # weight decay 
  "epoch_size": 120,                # only valid for taining, which is always 1 for inference 
  "pretrain_epoch_size": 0,         # epoch size that model has been trained before loading pretrained checkpoint, actual training epoch size is equal to epoch_size minus pretrain_epoch_size
  "save_checkpoint": True,          # whether save checkpoint or not
  "save_checkpoint_epochs": 1,      # the epoch interval between two checkpoints. By default, the last checkpoint will be saved after the last step
  "keep_checkpoint_max": 10,        # only keep the last keep_checkpoint_max checkpoint
  "save_checkpoint_path": "./",     # path to save checkpoint
  "warmup_epochs": 5,               # number of warmup epoch
  "lr_decay_mode": "poly"           # decay mode for generating learning rate
  "lr_init": 0,                     # initial learning rate
  "lr_end": 0,                      # final learning rate
  "lr_max": 0.01,                   # maximum learning rate
  ```

 - config for SqueezeNet, ImageNet dataset

  ```py
  "class_num": 1000,                # dataset class num
  "batch_size": 32,                 # batch size of input tensor
  "loss_scale": 1024,               # loss scale
  "momentum": 0.9,                  # momentum
  "weight_decay": 7e-5,             # weight decay 
  "epoch_size": 200,                # only valid for taining, which is always 1 for inference 
  "pretrain_epoch_size": 0,         # epoch size that model has been trained before loading pretrained checkpoint, actual training epoch size is equal to epoch_size minus pretrain_epoch_size
  "save_checkpoint": True,          # whether save checkpoint or not
  "save_checkpoint_epochs": 1,      # the epoch interval between two checkpoints. By default, the last checkpoint will be saved after the last step
  "keep_checkpoint_max": 10,        # only keep the last keep_checkpoint_max checkpoint
  "save_checkpoint_path": "./",     # path to save checkpoint
  "warmup_epochs": 0,               # number of warmup epoch
  "lr_decay_mode": "poly"           # decay mode for generating learning rate
  "use_label_smooth": True,         # label smooth
  "label_smooth_factor": 0.1,       # label smooth factor
  "lr_init": 0,                     # initial learning rate
  "lr_end": 0,                      # final learning rate
  "lr_max": 0.01,                   # maximum learning rate
  ```

 - config for SqueezeNet_Residual, CIFAR-10 dataset

  ```py
  "class_num": 10,                  # dataset class num
  "batch_size": 32,                 # batch size of input tensor
  "loss_scale": 1024,               # loss scale
  "momentum": 0.9,                  # momentum
  "weight_decay": 1e-4,             # weight decay 
  "epoch_size": 150,                # only valid for taining, which is always 1 for inference 
  "pretrain_epoch_size": 0,         # epoch size that model has been trained before loading pretrained checkpoint, actual training epoch size is equal to epoch_size minus pretrain_epoch_size
  "save_checkpoint": True,          # whether save checkpoint or not
  "save_checkpoint_epochs": 1,      # the epoch interval between two checkpoints. By default, the last checkpoint will be saved after the last step
  "keep_checkpoint_max": 10,        # only keep the last keep_checkpoint_max checkpoint
  "save_checkpoint_path": "./",     # path to save checkpoint
  "warmup_epochs": 5,               # number of warmup epoch
  "lr_decay_mode": "linear"         # decay mode for generating learning rate
  "lr_init": 0,                     # initial learning rate
  "lr_end": 0,                      # final learning rate
  "lr_max": 0.01,                   # maximum learning rate
  ```

 - config for SqueezeNet_Residual, ImageNet dataset

  ```py
  "class_num": 1000,                # dataset class num
  "batch_size": 32,                 # batch size of input tensor
  "loss_scale": 1024,               # loss scale
  "momentum": 0.9,                  # momentum
  "weight_decay": 7e-5,             # weight decay 
  "epoch_size": 300,                # only valid for taining, which is always 1 for inference 
  "pretrain_epoch_size": 0,         # epoch size that model has been trained before loading pretrained checkpoint, actual training epoch size is equal to epoch_size minus pretrain_epoch_size
  "save_checkpoint": True,          # whether save checkpoint or not
  "save_checkpoint_epochs": 1,      # the epoch interval between two checkpoints. By default, the last checkpoint will be saved after the last step
  "keep_checkpoint_max": 10,        # only keep the last keep_checkpoint_max checkpoint
  "save_checkpoint_path": "./",     # path to save checkpoint
  "warmup_epochs": 0,               # number of warmup epoch
  "lr_decay_mode": "cosine"         # decay mode for generating learning rate
  "use_label_smooth": True,         # label smooth
  "label_smooth_factor": 0.1,       # label smooth factor
  "lr_init": 0,                     # initial learning rate
  "lr_end": 0,                      # final learning rate
  "lr_max": 0.01,                   # maximum learning rate
  ```

 For more configuration details, please refer the script `config.py`.

 ## [Training Process](#contents)

 ### Usage
 #### Running on Ascend

  ```
  # distributed training
  Usage: sh scripts/run_distribute_train.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [RANK_TABLE_FILE] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)

  # standalone training
  Usage: sh scripts/run_standalone_train.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)
  ```

 For distributed training, a hccl configuration file with JSON format needs to be created in advance.

 Please follow the instructions in the link [hccl_tools](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/utils/hccl_tools).

 Training result will be stored in the example path, whose folder name begins with "train" or "train_parallel". Under this, you can find checkpoint file together with result like the followings in log.

 #### Running on GPU

 ```
 # distributed training example
 sh scripts/run_distribute_train_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)

 # standalone training example
 sh scripts/run_standalone_train_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)
 ```

 ### Result

 - Training SqueezeNet with CIFAR-10 dataset

 ```
 # standalone training result
 epoch: 1 step 1562, loss is 1.7103254795074463
 epoch: 2 step 1562, loss is 2.06101131439209
 epoch: 3 step 1562, loss is 1.5594401359558105
 epoch: 4 step 1562, loss is 1.4127278327941895
 epoch: 5 step 1562, loss is 1.2140142917633057
 ...
 ```

 - Training SqueezeNet with ImageNet dataset

 ```
 # distribute training result(8 pcs)
 epoch: 1 step 5004, loss is 5.716324329376221
 epoch: 2 step 5004, loss is 5.350603103637695
 epoch: 3 step 5004, loss is 4.580031394958496
 epoch: 4 step 5004, loss is 4.784664154052734
 epoch: 5 step 5004, loss is 4.136358261108398
 ...
 ```

 - Training SqueezeNet_Residual with CIFAR-10 dataset

 ```
 # standalone training result
 epoch: 1 step 1562, loss is 2.298271656036377
 epoch: 2 step 1562, loss is 2.2728664875030518
 epoch: 3 step 1562, loss is 1.9493038654327393
 epoch: 4 step 1562, loss is 1.7553865909576416
 epoch: 5 step 1562, loss is 1.3370063304901123
 ...
 ```
 - Training SqueezeNet_Residual with ImageNet dataset

 ```
 # distribute training result(8 pcs)
 epoch: 1 step 5004, loss is 6.802495002746582
 epoch: 2 step 5004, loss is 6.386072158813477
 epoch: 3 step 5004, loss is 5.513605117797852
 epoch: 4 step 5004, loss is 5.312961101531982
 epoch: 5 step 5004, loss is 4.888848304748535
 ...
 ```

 ## [Evaluation Process](#contents)

 ### Usage

 #### Running on Ascend
 ```
 # evaluation
 Usage: sh scripts/run_eval.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [CHECKPOINT_PATH]
 ```

 ```
 # evaluation example
 sh scripts/run_eval.sh squeezenet cifar10 0 ~/cifar-10-verify-bin train/squeezenet_cifar10-120_1562.ckpt
 ```

 checkpoint can be produced in training process.

 #### Running on GPU
 ```
 sh scripts/run_eval_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [CHECKPOINT_PATH]
 ```

 ### Result

 Evaluation result will be stored in the example path, whose folder name is "eval". Under this, you can find result like the followings in log.

 - Evaluating SqueezeNet with CIFAR-10 dataset

 ```
 result: {'top_1_accuracy': 0.8896233974358975, 'top_5_accuracy': 0.9965945512820513}
 ```

 - Evaluating SqueezeNet with ImageNet dataset

 ```
 result: {'top_1_accuracy': 0.5851472471190781, 'top_5_accuracy': 0.8105393725992317}
 ```

 - Evaluating SqueezeNet_Residual with CIFAR-10 dataset

 ```
 result: {'top_1_accuracy': 0.9077524038461539, 'top_5_accuracy': 0.9969951923076923}
 ```

 - Evaluating SqueezeNet_Residual with ImageNet dataset

 ```
 result: {'top_1_accuracy': 0.6094950384122919, 'top_5_accuracy': 0.826324423815621}
 ```
 


 # [Model Description](#contents)
 ## [Performance](#contents)

 ### Evaluation Performance 

 #### SqueezeNet on CIFAR-10
 | Parameters                 | Ascend                                                      |
 | -------------------------- | ----------------------------------------------------------- |
 | Model Version              | SqueezeNet                                                  |
 | Resource                   | Ascend 910 ；CPU 2.60GHz，192cores；Memory，755G             |
 | uploaded Date              | 11/06/2020 (month/day/year)                                 |
 | MindSpore Version          | 1.0.0                                                       |
 | Dataset                    | CIFAR-10                                                    |
 | Training Parameters        | epoch=120, steps=195, batch_size=32, lr=0.01                |
 | Optimizer                  | Momentum                                                    |
 | Loss Function              | Softmax Cross Entropy                                       |
 | outputs                    | probability                                                 |
 | Loss                       | 0.0496                                                      |
 | Speed                      | 1pc: 16.7 ms/step;  8pcs: 17.0 ms/step                      |
 | Total time                 | 1pc: 55.5 mins;  8pcs: 15.0 mins                            |
 | Parameters (M)             | 4.8                                                         |
 | Checkpoint for Fine tuning | 6.4M (.ckpt file)                                           |
 | Scripts                    | [squeezenet script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/squeezenet) |

 #### SqueezeNet on ImageNet
 | Parameters                 | Ascend                                                      |
 | -------------------------- | ----------------------------------------------------------- |
 | Model Version              | SqueezeNet                                                  |
 | Resource                   | Ascend 910 ；CPU 2.60GHz，192cores；Memory，755G             |
 | uploaded Date              | 11/06/2020 (month/day/year)                                 |
 | MindSpore Version          | 1.0.0                                                       |
 | Dataset                    | ImageNet                                                    |
 | Training Parameters        | epoch=200, steps=5004, batch_size=32, lr=0.01               |
 | Optimizer                  | Momentum                                                    |
 | Loss Function              | Softmax Cross Entropy                                       |
 | outputs                    | probability                                                 |
 | Loss                       | 2.9150                                                      |
 | Speed                      | 8pcs: 19.9 ms/step                                          |
 | Total time                 | 8pcs: 5.2 hours                                             |
 | Parameters (M)             | 4.8                                                         |
 | Checkpoint for Fine tuning | 13.3M (.ckpt file)                                          |
 | Scripts                    | [squeezenet script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/squeezenet) |

 #### SqueezeNet_Residual on CIFAR-10
 | Parameters                 | Ascend                                                      |
 | -------------------------- | ----------------------------------------------------------- |
 | Model Version              | SqueezeNet_Residual                                         |
 | Resource                   | Ascend 910 ；CPU 2.60GHz，192cores；Memory，755G             |
 | uploaded Date              | 11/06/2020 (month/day/year)                                 |
 | MindSpore Version          | 1.0.0                                                       |
 | Dataset                    | CIFAR-10                                                    |
 | Training Parameters        | epoch=150, steps=195, batch_size=32, lr=0.01                |
 | Optimizer                  | Momentum                                                    |
 | Loss Function              | Softmax Cross Entropy                                       |
 | outputs                    | probability                                                 |
 | Loss                       | 0.0641                                                      |
 | Speed                      | 1pc: 16.9 ms/step;  8pcs: 17.3 ms/step                      |
 | Total time                 | 1pc: 68.6 mins;  8pcs: 20.9 mins                            |
 | Parameters (M)             | 4.8                                                         |
 | Checkpoint for Fine tuning | 6.5M (.ckpt file)                                           |
 | Scripts                    | [squeezenet script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/squeezenet) |

 #### SqueezeNet_Residual on ImageNet
 | Parameters                 | Ascend                                                      |
 | -------------------------- | ----------------------------------------------------------- |
 | Model Version              | SqueezeNet_Residual                                         |
 | Resource                   | Ascend 910 ；CPU 2.60GHz，192cores；Memory，755G             |
 | uploaded Date              | 11/06/2020 (month/day/year)                                 |
 | MindSpore Version          | 1.0.0                                                       |
 | Dataset                    | ImageNet                                                    |
 | Training Parameters        | epoch=300, steps=5004, batch_size=32, lr=0.01               |
 | Optimizer                  | Momentum                                                    |
 | Loss Function              | Softmax Cross Entropy                                       |
 | outputs                    | probability                                                 |
 | Loss                       | 2.9040                                                      |
 | Speed                      | 8pcs: 20.2 ms/step                                          |
 | Total time                 | 8pcs: 8.0 hours                                             |
 | Parameters (M)             | 4.8                                                         |
 | Checkpoint for Fine tuning | 15.3M (.ckpt file)                                          |
 | Scripts                    | [squeezenet script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/squeezenet) |



 ### Inference Performance

 #### SqueezeNet on CIFAR-10
 | Parameters          | Ascend                      |
 | ------------------- | --------------------------- |
 | Model Version       | SqueezeNet                  |
 | Resource            | Ascend 910                  |
 | Uploaded Date       | 11/06/2020 (month/day/year) |
 | MindSpore Version   | 1.0.0                       |
 | Dataset             | CIFAR-10                    |
 | batch_size          | 32                          |
 | outputs             | probability                 |
 | Accuracy            | 1pc: 89.0%;  8pcs: 84.4%    |

 #### SqueezeNet on ImageNet
 | Parameters          | Ascend                      |
 | ------------------- | --------------------------- |
 | Model Version       | SqueezeNet                  |
 | Resource            | Ascend 910                  |
 | Uploaded Date       | 11/06/2020 (month/day/year) |
 | MindSpore Version   | 1.0.0                       |
 | Dataset             | ImageNet                    |
 | batch_size          | 32                          |
 | outputs             | probability                 |
 | Accuracy            | 8pcs: 58.5%(TOP1), 81.1%(TOP5)       |

 #### SqueezeNet_Residual on CIFAR-10
 | Parameters          | Ascend                      |
 | ------------------- | --------------------------- |
 | Model Version       | SqueezeNet_Residual         |
 | Resource            | Ascend 910                  |
 | Uploaded Date       | 11/06/2020 (month/day/year) |
 | MindSpore Version   | 1.0.0                       |
 | Dataset             | CIFAR-10                    |
 | batch_size          | 32                          |
 | outputs             | probability                 |
 | Accuracy            | 1pc: 90.8%;  8pcs: 87.4%    |

 #### SqueezeNet_Residual on ImageNet
 | Parameters          | Ascend                      |
 | ------------------- | --------------------------- |
 | Model Version       | SqueezeNet_Residual         |
 | Resource            | Ascend 910                  |
 | Uploaded Date       | 11/06/2020 (month/day/year) |
 | MindSpore Version   | 1.0.0                       |
 | Dataset             | ImageNet                    |
 | batch_size          | 32                          |
 | outputs             | probability                 |
 | Accuracy            | 8pcs: 60.9%(TOP1), 82.6%(TOP5)       |

 ## [How to use](#contents)
 ### Inference

 If you need to use the trained model to perform inference on multiple hardware platforms, such as GPU, Ascend 910 or Ascend 310, you can refer to this [Link](https://www.mindspore.cn/tutorial/training/en/master/advanced_use/migrate_3rd_scripts.html). Following the steps below, this is a simple example:

 - Running on Ascend

  ```
  # Set context
  device_id = int(os.getenv('DEVICE_ID'))
  context.set_context(mode=context.GRAPH_MODE,
                      device_target='Ascend',
                      device_id=device_id)
  
  # Load unseen dataset for inference
  dataset = create_dataset(dataset_path=args_opt.dataset_path,
                           do_train=False,
                           batch_size=config.batch_size,
                           target='Ascend')

  # Define model
  net = squeezenet(num_classes=config.class_num)
  loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
  model = Model(net,
                loss_fn=loss,
                metrics={'top_1_accuracy', 'top_5_accuracy'})

  # Load pre-trained model
  param_dict = load_checkpoint(args_opt.checkpoint_path)
  load_param_into_net(net, param_dict)
  net.set_train(False)
  
  # Make predictions on the unseen dataset
  acc = model.eval(dataset)
  print("accuracy: ", acc)
  ```

 - Running on GPU:

  ```
  # Set context
  device_id = int(os.getenv('DEVICE_ID'))
  context.set_context(mode=context.GRAPH_MODE,
                      device_target='GPU',
                      device_id=device_id)
  
  # Load unseen dataset for inference
  dataset = create_dataset(dataset_path=args_opt.dataset_path,
                           do_train=False,
                           batch_size=config.batch_size,
                           target='GPU')

  # Define model
  net = squeezenet(num_classes=config.class_num)
  loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
  model = Model(net,
                loss_fn=loss,
                metrics={'top_1_accuracy', 'top_5_accuracy'})

  # Load pre-trained model
  param_dict = load_checkpoint(args_opt.checkpoint_path)
  load_param_into_net(net, param_dict)
  net.set_train(False)
  
  # Make predictions on the unseen dataset
  acc = model.eval(dataset)
  print("accuracy: ", acc)
  ```

 ### Continue Training on the Pretrained Model 

 - running on Ascend

  ```
  # Load dataset
  dataset = create_dataset(dataset_path=args_opt.dataset_path,
                           do_train=True,
                           repeat_num=1,
                           batch_size=config.batch_size,
                           target='Ascend')
  step_size = dataset.get_dataset_size()
  
  # define net
  net = squeezenet(num_classes=config.class_num)

  # load checkpoint
  if args_opt.pre_trained:
      param_dict = load_checkpoint(args_opt.pre_trained)
      load_param_into_net(net, param_dict)

  # init lr
  lr = get_lr(lr_init=config.lr_init,
              lr_end=config.lr_end,
              lr_max=config.lr_max,
              total_epochs=config.epoch_size,
              warmup_epochs=config.warmup_epochs,
              pretrain_epochs=config.pretrain_epoch_size,
              steps_per_epoch=step_size,
              lr_decay_mode=config.lr_decay_mode)
  lr = Tensor(lr)
  loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
  loss_scale = FixedLossScaleManager(config.loss_scale, 
                                     drop_overflow_update=False)
  opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()),
                 lr,
                 config.momentum,
                 config.weight_decay,
                 config.loss_scale,
                 use_nesterov=True)
  model = Model(net,
                loss_fn=loss,
                optimizer=opt,
                loss_scale_manager=loss_scale,
                metrics={'acc'},
                amp_level="O2",
                keep_batchnorm_fp32=False)

  # Set callbacks 
  config_ck = CheckpointConfig(
      save_checkpoint_steps=config.save_checkpoint_epochs * step_size,
      keep_checkpoint_max=config.keep_checkpoint_max)
  time_cb = TimeMonitor(data_size=step_size)
  ckpt_cb = ModelCheckpoint(prefix=args_opt.net + '_' + args_opt.dataset,
                            directory=ckpt_save_dir,
                            config=config_ck)
  loss_cb = LossMonitor()
  
  # Start training
  model.train(config.epoch_size - config.pretrain_epoch_size, dataset,
              callbacks=[time_cb, ckpt_cb, loss_cb])
  print("train success")
  ```

 - running on GPU

  ```
  # Load dataset
  dataset = create_dataset(dataset_path=args_opt.dataset_path,
                           do_train=True,
                           repeat_num=1,
                           batch_size=config.batch_size,
                           target='Ascend')
  step_size = dataset.get_dataset_size()
  
  # define net
  net = squeezenet(num_classes=config.class_num)

  # load checkpoint
  if args_opt.pre_trained:
      param_dict = load_checkpoint(args_opt.pre_trained)
      load_param_into_net(net, param_dict)

  # init lr
  lr = get_lr(lr_init=config.lr_init,
              lr_end=config.lr_end,
              lr_max=config.lr_max,
              total_epochs=config.epoch_size,
              warmup_epochs=config.warmup_epochs,
              pretrain_epochs=config.pretrain_epoch_size,
              steps_per_epoch=step_size,
              lr_decay_mode=config.lr_decay_mode)
  lr = Tensor(lr)
  loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
  opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()),
                 lr,
                 config.momentum,
                 config.weight_decay,
                 use_nesterov=True)
  model = Model(net, loss_fn=loss, optimizer=opt, metrics={'acc'})

  # Set callbacks 
  config_ck = CheckpointConfig(
      save_checkpoint_steps=config.save_checkpoint_epochs * step_size,
      keep_checkpoint_max=config.keep_checkpoint_max)
  time_cb = TimeMonitor(data_size=step_size)
  ckpt_cb = ModelCheckpoint(prefix=args_opt.net + '_' + args_opt.dataset,
                            directory=ckpt_save_dir,
                            config=config_ck)
  loss_cb = LossMonitor()
  
  # Start training
  model.train(config.epoch_size - config.pretrain_epoch_size, dataset,
              callbacks=[time_cb, ckpt_cb, loss_cb])
  print("train success")
  ```

 ### Transfer Learning
 To be added.


 # [Description of Random Situation](#contents)

 In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py. 


 # [ModelZoo Homepage](#contents)  
 Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).  
--- a/model_zoo/official/cv/squeezenet/eval.py
+++ b/model_zoo/official/cv/squeezenet/eval.py
@@ -0,0 +1,95 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """eval squeezenet."""
 import os
 import argparse
 from mindspore import context
 from mindspore.common import set_seed
 from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits
 from mindspore.train.model import Model
 from mindspore.train.serialization import load_checkpoint, load_param_into_net
 from src.CrossEntropySmooth import CrossEntropySmooth

 parser = argparse.ArgumentParser(description='Image classification')
 parser.add_argument('--net', type=str, default='squeezenet', choices=['squeezenet', 'squeezenet_residual'],
                    help='Model.')
 parser.add_argument('--dataset', type=str, default='cifar10', choices=['cifar10', 'imagenet'], help='Dataset.')
 parser.add_argument('--checkpoint_path', type=str, default=None, help='Checkpoint file path')
 parser.add_argument('--dataset_path', type=str, default=None, help='Dataset path')
 parser.add_argument('--device_target', type=str, default='Ascend', help='Device target')
 args_opt = parser.parse_args()

 set_seed(1)

 if args_opt.net == "squeezenet":
    from src.squeezenet import SqueezeNet as squeezenet
    if args_opt.dataset == "cifar10":
        from src.config import config1 as config
        from src.dataset import create_dataset_cifar as create_dataset
    else:
        from src.config import config2 as config
        from src.dataset import create_dataset_imagenet as create_dataset
 else:
    from src.squeezenet import SqueezeNet_Residual as squeezenet
    if args_opt.dataset == "cifar10":
        from src.config import config3 as config
        from src.dataset import create_dataset_cifar as create_dataset
    else:
        from src.config import config4 as config
        from src.dataset import create_dataset_imagenet as create_dataset

 if __name__ == '__main__':
    target = args_opt.device_target

    # init context
    device_id = int(os.getenv('DEVICE_ID'))
    context.set_context(mode=context.GRAPH_MODE,
                        device_target=target,
                        device_id=device_id)

    # create dataset
    dataset = create_dataset(dataset_path=args_opt.dataset_path,
                             do_train=False,
                             batch_size=config.batch_size,
                             target=target)
    step_size = dataset.get_dataset_size()

    # define net
    net = squeezenet(num_classes=config.class_num)

    # load checkpoint
    param_dict = load_checkpoint(args_opt.checkpoint_path)
    load_param_into_net(net, param_dict)
    net.set_train(False)

    # define loss
    if args_opt.dataset == "imagenet":
        if not config.use_label_smooth:
            config.label_smooth_factor = 0.0
        loss = CrossEntropySmooth(sparse=True,
                                  reduction='mean',
                                  smooth_factor=config.label_smooth_factor,
                                  num_classes=config.class_num)
    else:
        loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')

    # define model
    model = Model(net,
                  loss_fn=loss,
                  metrics={'top_1_accuracy', 'top_5_accuracy'})

    # eval model
    res = model.eval(dataset)
    print("result:", res, "ckpt=", args_opt.checkpoint_path)
--- a/model_zoo/official/cv/squeezenet/export.py
+++ b/model_zoo/official/cv/squeezenet/export.py
@@ -0,0 +1,54 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """
 ##############export checkpoint file into air and onnx models#################
 python export.py --net squeezenet --dataset cifar10 --checkpoint_path squeezenet_cifar10-120_1562.ckpt
 """

 import argparse
 import numpy as np
 from mindspore import Tensor
 from mindspore.train.serialization import load_checkpoint, load_param_into_net, export

 if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Image classification')
    parser.add_argument('--net', type=str, default='squeezenet', choices=['squeezenet', 'squeezenet_residual'],
                        help='Model.')
    parser.add_argument('--dataset', type=str, default='cifar10', choices=['cifar10', 'imagenet'], help='Dataset.')
    parser.add_argument('--checkpoint_path', type=str, default=None, help='Checkpoint file path')
    args_opt = parser.parse_args()

    if args_opt.net == "squeezenet":
        from src.squeezenet import SqueezeNet as squeezenet
    else:
        from src.squeezenet import SqueezeNet_Residual as squeezenet
    if args_opt.dataset == "cifar10":
        num_classes = 10
    else:
        num_classes = 1000

    onnx_filename = args_opt.net + '_' + args_opt.dataset + '.onnx'
    air_filename = args_opt.net + '_' + args_opt.dataset + '.air'

    net = squeezenet(num_classes=num_classes)

    assert args_opt.checkpoint_path is not None, "checkpoint_path is None."

    param_dict = load_checkpoint(args_opt.checkpoint_path)
    load_param_into_net(net, param_dict)

    input_arr = Tensor(np.zeros([1, 3, 227, 227], np.float32))
    export(net, input_arr, file_name=onnx_filename, file_format="ONNX")
    export(net, input_arr, file_name=air_filename, file_format="AIR")
--- a/model_zoo/official/cv/squeezenet/scripts/run_distribute_train.sh
+++ b/model_zoo/official/cv/squeezenet/scripts/run_distribute_train.sh
@@ -0,0 +1,99 @@
 #!/bin/bash
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================

 if [ $# != 4 ] && [ $# != 5 ]
 then 
 	echo "Usage: sh scripts/run_distribute_train.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [RANK_TABLE_FILE] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)"
 exit 1
 fi

 if [ $1 != "squeezenet" ] && [ $1 != "squeezenet_residual" ]
 then 
    echo "error: the selected net is neither squeezenet nor squeezenet_residual"
 exit 1
 fi

 if [ $2 != "cifar10" ] && [ $2 != "imagenet" ]
 then 
    echo "error: the selected dataset is neither cifar10 nor imagenet"
 exit 1
 fi

 get_real_path(){
  if [ "${1:0:1}" == "/" ]; then
    echo "$1"
  else
    echo "$(realpath -m $PWD/$1)"
  fi
 }

 PATH1=$(get_real_path $3)
 PATH2=$(get_real_path $4)

 if [ $# == 5 ]
 then 
    PATH3=$(get_real_path $5)
 fi

 if [ ! -f $PATH1 ]
 then 
    echo "error: RANK_TABLE_FILE=$PATH1 is not a file"
 exit 1
 fi 

 if [ ! -d $PATH2 ]
 then 
    echo "error: DATASET_PATH=$PATH2 is not a directory"
 exit 1
 fi 

 if [ $# == 5 ] && [ ! -f $PATH3 ]
 then
    echo "error: PRETRAINED_CKPT_PATH=$PATH3 is not a file"
 exit 1
 fi

 ulimit -u unlimited
 export DEVICE_NUM=8
 export RANK_SIZE=8
 export RANK_TABLE_FILE=$PATH1

 export SERVER_ID=0
 rank_start=$((DEVICE_NUM * SERVER_ID))

 for((i=0; i<${DEVICE_NUM}; i++))
 do
    export DEVICE_ID=${i}
    export RANK_ID=$((rank_start + i))
    rm -rf ./train_parallel$i
    mkdir ./train_parallel$i
    cp ./train.py ./train_parallel$i
    cp -r ./src ./train_parallel$i
    cd ./train_parallel$i || exit
    echo "start training for rank $RANK_ID, device $DEVICE_ID"
    env > env.log
    if [ $# == 4 ]
    then	    
        python train.py --net=$1 --dataset=$2 --run_distribute=True --device_num=$DEVICE_NUM --dataset_path=$PATH2 &> log &
    fi
    
    if [ $# == 5 ]
    then
        python train.py --net=$1 --dataset=$2 --run_distribute=True --device_num=$DEVICE_NUM --dataset_path=$PATH2 --pre_trained=$PATH3 &> log &
    fi

    cd ..
 done
--- a/model_zoo/official/cv/squeezenet/scripts/run_distribute_train_gpu.sh
+++ b/model_zoo/official/cv/squeezenet/scripts/run_distribute_train_gpu.sh
@@ -0,0 +1,85 @@
 #!/bin/bash
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================

 if [ $# != 3 ] && [ $# != 4 ]
 then 
 	  echo "Usage: sh scripts/run_distribute_train_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)"
 exit 1
 fi

 if [ $1 != "squeezenet" ] && [ $1 != "squeezenet_residual" ]
 then 
    echo "error: the selected net is neither squeezenet nor squeezenet_residual"
 exit 1
 fi

 if [ $2 != "cifar10" ] && [ $2 != "imagenet" ]
 then 
    echo "error: the selected dataset is neither cifar10 nor imagenet"
 exit 1
 fi

 get_real_path(){
  if [ "${1:0:1}" == "/" ]; then
    echo "$1"
  else
    echo "$(realpath -m $PWD/$1)"
  fi
 }

 PATH1=$(get_real_path $3)

 if [ $# == 4 ]
 then 
    PATH2=$(get_real_path $4)
 fi


 if [ ! -d $PATH1 ]
 then 
    echo "error: DATASET_PATH=$PATH1 is not a directory"
 exit 1
 fi 

 if [ $# == 5 ] && [ ! -f $PATH2 ]
 then
    echo "error: PRETRAINED_CKPT_PATH=$PATH2 is not a file"
 exit 1
 fi

 ulimit -u unlimited
 export DEVICE_NUM=8
 export RANK_SIZE=8

 rm -rf ./train_parallel
 mkdir ./train_parallel
 cp ./train.py ./train_parallel
 cp -r ./src ./train_parallel
 cd ./train_parallel || exit

 if [ $# == 3 ]
 then	    
        mpirun --allow-run-as-root -n $RANK_SIZE --output-filename log_output --merge-stderr-to-stdout \
 	      python train.py --net=$1 --dataset=$2 --run_distribute=True \
 	      --device_num=$DEVICE_NUM --device_target="GPU" --dataset_path=$PATH1 &> log &
 fi
    
 if [ $# == 4 ]
 then
        mpirun --allow-run-as-root -n $RANK_SIZE --output-filename log_output --merge-stderr-to-stdout \
          python train.py --net=$1 --dataset=$2 --run_distribute=True \
 	  --device_num=$DEVICE_NUM --device_target="GPU" --dataset_path=$PATH1 --pre_trained=$PATH2 &> log &
 fi
--- a/model_zoo/official/cv/squeezenet/scripts/run_eval.sh
+++ b/model_zoo/official/cv/squeezenet/scripts/run_eval.sh
@@ -0,0 +1,76 @@
 #!/bin/bash
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================

 if [ $# != 5 ]
 then 
    echo "Usage: sh scripts/run_eval.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [CHECKPOINT_PATH]"
 exit 1
 fi

 if [ $1 != "squeezenet" ] && [ $1 != "squeezenet_residual" ]
 then 
    echo "error: the selected net is neither squeezenet nor squeezenet_residual"
 exit 1
 fi

 if [ $2 != "cifar10" ] && [ $2 != "imagenet" ]
 then 
    echo "error: the selected dataset is neither cifar10 nor imagenet"
 exit 1
 fi

 get_real_path(){
  if [ "${1:0:1}" == "/" ]; then
    echo "$1"
  else
    echo "$(realpath -m $PWD/$1)"
  fi
 }

 PATH1=$(get_real_path $4)
 PATH2=$(get_real_path $5)


 if [ ! -d $PATH1 ]
 then 
    echo "error: DATASET_PATH=$PATH1 is not a directory"
 exit 1
 fi 

 if [ ! -f $PATH2 ]
 then 
    echo "error: CHECKPOINT_PATH=$PATH2 is not a file"
 exit 1
 fi 

 ulimit -u unlimited
 export DEVICE_NUM=1
 export DEVICE_ID=$3
 export RANK_SIZE=$DEVICE_NUM
 export RANK_ID=0

 if [ -d "eval" ];
 then
    rm -rf ./eval
 fi
 mkdir ./eval
 cp ./eval.py ./eval
 cp -r ./src ./eval
 cd ./eval || exit
 env > env.log
 echo "start evaluation for device $DEVICE_ID"
 python eval.py --net=$1 --dataset=$2 --dataset_path=$PATH1 --checkpoint_path=$PATH2 &> log &
 cd ..
--- a/model_zoo/official/cv/squeezenet/scripts/run_eval_gpu.sh
+++ b/model_zoo/official/cv/squeezenet/scripts/run_eval_gpu.sh
@@ -0,0 +1,76 @@
 #!/bin/bash
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================

 if [ $# != 5 ]
 then 
    echo "Usage: sh scripts/run_eval_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [CHECKPOINT_PATH]"
 exit 1
 fi

 if [ $1 != "squeezenet" ] && [ $1 != "squeezenet_residual" ]
 then 
    echo "error: the selected net is neither squeezenet nor squeezenet_residual"
 exit 1
 fi

 if [ $2 != "cifar10" ] && [ $2 != "imagenet" ]
 then 
    echo "error: the selected dataset is neither cifar10 nor imagenet"
 exit 1
 fi

 get_real_path(){
  if [ "${1:0:1}" == "/" ]; then
    echo "$1"
  else
    echo "$(realpath -m $PWD/$1)"
  fi
 }

 PATH1=$(get_real_path $4)
 PATH2=$(get_real_path $5)


 if [ ! -d $PATH1 ]
 then 
    echo "error: DATASET_PATH=$PATH1 is not a directory"
 exit 1
 fi 

 if [ ! -f $PATH2 ]
 then 
    echo "error: CHECKPOINT_PATH=$PATH2 is not a file"
 exit 1
 fi 

 ulimit -u unlimited
 export DEVICE_NUM=1
 export DEVICE_ID=$3
 export RANK_SIZE=$DEVICE_NUM
 export RANK_ID=0

 if [ -d "eval" ];
 then
    rm -rf ./eval
 fi
 mkdir ./eval
 cp ./eval.py ./eval
 cp -r ./src ./eval
 cd ./eval || exit
 env > env.log
 echo "start evaluation for device $DEVICE_ID"
 python eval.py --net=$1 --dataset=$2 --dataset_path=$PATH1 --checkpoint_path=$PATH2 --device_target="GPU" &> log &
 cd ..
--- a/model_zoo/official/cv/squeezenet/scripts/run_standalone_train.sh
+++ b/model_zoo/official/cv/squeezenet/scripts/run_standalone_train.sh
@@ -0,0 +1,87 @@
 #!/bin/bash
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================

 if [ $# != 4 ] && [ $# != 5 ]
 then 
    echo "Usage: sh scripts/run_standalone_train.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)"
 exit 1
 fi

 if [ $1 != "squeezenet" ] && [ $1 != "squeezenet_residual" ]
 then 
    echo "error: the selected net is neither squeezenet nor squeezenet_residual"
 exit 1
 fi

 if [ $2 != "cifar10" ] && [ $2 != "imagenet" ]
 then 
    echo "error: the selected dataset is neither cifar10 nor imagenet"
 exit 1
 fi

 get_real_path(){
  if [ "${1:0:1}" == "/" ]; then
    echo "$1"
  else
    echo "$(realpath -m $PWD/$1)"
  fi
 }

 PATH1=$(get_real_path $4)

 if [ $# == 5 ]
 then
    PATH2=$(get_real_path $5)
 fi

 if [ ! -d $PATH1 ]
 then 
    echo "error: DATASET_PATH=$PATH1 is not a directory"
 exit 1
 fi

 if [ $# == 5 ] && [ ! -f $PATH2 ]
 then
    echo "error: PRETRAINED_CKPT_PATH=$PATH2 is not a file"
 exit 1
 fi

 ulimit -u unlimited
 export DEVICE_NUM=1
 export DEVICE_ID=$3
 export RANK_ID=0
 export RANK_SIZE=1

 if [ -d "train" ];
 then
    rm -rf ./train
 fi
 mkdir ./train
 cp ./train.py ./train
 cp -r ./src ./train
 cd ./train || exit
 echo "start training for device $DEVICE_ID"
 env > env.log
 if [ $# == 4 ]
 then
    python train.py --net=$1 --dataset=$2 --dataset_path=$PATH1 &> log &
 fi

 if [ $# == 5 ]
 then
    python train.py --net=$1 --dataset=$2 --dataset_path=$PATH1 --pre_trained=$PATH2 &> log &
 fi
 cd ..
--- a/model_zoo/official/cv/squeezenet/scripts/run_standalone_train_gpu.sh
+++ b/model_zoo/official/cv/squeezenet/scripts/run_standalone_train_gpu.sh
@@ -0,0 +1,87 @@
 #!/bin/bash
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================

 if [ $# != 4 ] && [ $# != 5 ]
 then 
    echo "Usage: sh scripts/run_standalone_train_gpu.sh [squeezenet|squeezenet_residual] [cifar10|imagenet] [DEVICE_ID] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)"
 exit 1
 fi

 if [ $1 != "squeezenet" ] && [ $1 != "squeezenet_residual" ]
 then 
    echo "error: the selected net is neither squeezenet nor squeezenet_residual"
 exit 1
 fi

 if [ $2 != "cifar10" ] && [ $2 != "imagenet" ]
 then 
    echo "error: the selected dataset is neither cifar10 nor imagenet"
 exit 1
 fi

 get_real_path(){
  if [ "${1:0:1}" == "/" ]; then
    echo "$1"
  else
    echo "$(realpath -m $PWD/$1)"
  fi
 }

 PATH1=$(get_real_path $4)

 if [ $# == 5 ]
 then
    PATH2=$(get_real_path $5)
 fi

 if [ ! -d $PATH1 ]
 then 
    echo "error: DATASET_PATH=$PATH1 is not a directory"
 exit 1
 fi

 if [ $# == 5 ] && [ ! -f $PATH2 ]
 then
    echo "error: PRETRAINED_CKPT_PATH=$PATH2 is not a file"
 exit 1
 fi

 ulimit -u unlimited
 export DEVICE_NUM=1
 export DEVICE_ID=$3
 export RANK_ID=0
 export RANK_SIZE=1

 if [ -d "train" ];
 then
    rm -rf ./train
 fi
 mkdir ./train
 cp ./train.py ./train
 cp -r ./src ./train
 cd ./train || exit
 echo "start training for device $DEVICE_ID"
 env > env.log
 if [ $# == 4 ]
 then
    python train.py --net=$1 --dataset=$2 --device_target="GPU" --dataset_path=$PATH1 &> log &
 fi

 if [ $# == 5 ]
 then
    python train.py --net=$1 --dataset=$2 --device_target="GPU" --dataset_path=$PATH1 --pre_trained=$PATH2 &> log &
 fi
 cd ..
--- a/model_zoo/official/cv/squeezenet/src/CrossEntropySmooth.py
+++ b/model_zoo/official/cv/squeezenet/src/CrossEntropySmooth.py
@@ -0,0 +1,38 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """define loss function for network"""
 import mindspore.nn as nn
 from mindspore import Tensor
 from mindspore.common import dtype as mstype
 from mindspore.nn.loss.loss import _Loss
 from mindspore.ops import functional as F
 from mindspore.ops import operations as P


 class CrossEntropySmooth(_Loss):
    """CrossEntropy"""
    def __init__(self, sparse=True, reduction='mean', smooth_factor=0., num_classes=1000):
        super(CrossEntropySmooth, self).__init__()
        self.onehot = P.OneHot()
        self.sparse = sparse
        self.on_value = Tensor(1.0 - smooth_factor, mstype.float32)
        self.off_value = Tensor(1.0 * smooth_factor / (num_classes - 1), mstype.float32)
        self.ce = nn.SoftmaxCrossEntropyWithLogits(reduction=reduction)

    def construct(self, logit, label):
        if self.sparse:
            label = self.onehot(label, F.shape(logit)[1], self.on_value, self.off_value)
        loss = self.ce(logit, label)
        return loss
--- a/model_zoo/official/cv/squeezenet/src/config.py
+++ b/model_zoo/official/cv/squeezenet/src/config.py
@@ -0,0 +1,102 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """
 network config setting, will be used in train.py and eval.py
 """
 from easydict import EasyDict as ed

 # config for squeezenet, cifar10
 config1 = ed({
    "class_num": 10,
    "batch_size": 32,
    "loss_scale": 1024,
    "momentum": 0.9,
    "weight_decay": 1e-4,
    "epoch_size": 120,
    "pretrain_epoch_size": 0,
    "save_checkpoint": True,
    "save_checkpoint_epochs": 1,
    "keep_checkpoint_max": 10,
    "save_checkpoint_path": "./",
    "warmup_epochs": 5,
    "lr_decay_mode": "poly",
    "lr_init": 0,
    "lr_end": 0,
    "lr_max": 0.01
 })

 # config for squeezenet, imagenet
 config2 = ed({
    "class_num": 1000,
    "batch_size": 32,
    "loss_scale": 1024,
    "momentum": 0.9,
    "weight_decay": 7e-5,
    "epoch_size": 200,
    "pretrain_epoch_size": 0,
    "save_checkpoint": True,
    "save_checkpoint_epochs": 1,
    "keep_checkpoint_max": 10,
    "save_checkpoint_path": "./",
    "warmup_epochs": 0,
    "lr_decay_mode": "poly",
    "use_label_smooth": True,
    "label_smooth_factor": 0.1,
    "lr_init": 0,
    "lr_end": 0,
    "lr_max": 0.01
 })

 # config for squeezenet_residual, cifar10
 config3 = ed({
    "class_num": 10,
    "batch_size": 32,
    "loss_scale": 1024,
    "momentum": 0.9,
    "weight_decay": 1e-4,
    "epoch_size": 150,
    "pretrain_epoch_size": 0,
    "save_checkpoint": True,
    "save_checkpoint_epochs": 1,
    "keep_checkpoint_max": 10,
    "save_checkpoint_path": "./",
    "warmup_epochs": 5,
    "lr_decay_mode": "linear",
    "lr_init": 0,
    "lr_end": 0,
    "lr_max": 0.01
 })

 # config for squeezenet_residual, imagenet
 config4 = ed({
    "class_num": 1000,
    "batch_size": 32,
    "loss_scale": 1024,
    "momentum": 0.9,
    "weight_decay": 7e-5,
    "epoch_size": 300,
    "pretrain_epoch_size": 0,
    "save_checkpoint": True,
    "save_checkpoint_epochs": 1,
    "keep_checkpoint_max": 10,
    "save_checkpoint_path": "./",
    "warmup_epochs": 0,
    "lr_decay_mode": "cosine",
    "use_label_smooth": True,
    "label_smooth_factor": 0.1,
    "lr_init": 0,
    "lr_end": 0,
    "lr_max": 0.01
 })
--- a/model_zoo/official/cv/squeezenet/src/dataset.py
+++ b/model_zoo/official/cv/squeezenet/src/dataset.py
@@ -0,0 +1,191 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """
 create train or eval dataset.
 """
 import os
 import mindspore.common.dtype as mstype
 import mindspore.dataset.engine as de
 import mindspore.dataset.vision.c_transforms as C
 import mindspore.dataset.transforms.c_transforms as C2
 from mindspore.communication.management import init, get_rank, get_group_size


 def create_dataset_cifar(dataset_path,
                         do_train,
                         repeat_num=1,
                         batch_size=32,
                         target="Ascend"):
    """
    create a train or evaluate cifar10 dataset
    Args:
        dataset_path(string): the path of dataset.
        do_train(bool): whether dataset is used for train or eval.
        repeat_num(int): the repeat times of dataset. Default: 1
        batch_size(int): the batch size of dataset. Default: 32
        target(str): the device target. Default: Ascend

    Returns:
        dataset
    """
    if target == "Ascend":
        device_num, rank_id = _get_rank_info()
    else:
        init()
        rank_id = get_rank()
        device_num = get_group_size()

    if device_num == 1:
        ds = de.Cifar10Dataset(dataset_path,
                               num_parallel_workers=8,
                               shuffle=True)
    else:
        ds = de.Cifar10Dataset(dataset_path,
                               num_parallel_workers=8,
                               shuffle=True,
                               num_shards=device_num,
                               shard_id=rank_id)

    # define map operations
    if do_train:
        trans = [
            C.RandomCrop((32, 32), (4, 4, 4, 4)),
            C.RandomHorizontalFlip(prob=0.5),
            C.RandomColorAdjust(brightness=0.4, contrast=0.4, saturation=0.4),
            C.Resize((227, 227)),
            C.Rescale(1.0 / 255.0, 0.0),
            C.Normalize([0.4914, 0.4822, 0.4465], [0.2023, 0.1994, 0.2010]),
            C.CutOut(112),
            C.HWC2CHW()
        ]
    else:
        trans = [
            C.Resize((227, 227)),
            C.Rescale(1.0 / 255.0, 0.0),
            C.Normalize([0.4914, 0.4822, 0.4465], [0.2023, 0.1994, 0.2010]),
            C.HWC2CHW()
        ]

    type_cast_op = C2.TypeCast(mstype.int32)

    ds = ds.map(operations=type_cast_op,
                input_columns="label",
                num_parallel_workers=8)
    ds = ds.map(operations=trans,
                input_columns="image",
                num_parallel_workers=8)

    # apply batch operations
    ds = ds.batch(batch_size, drop_remainder=True)

    # apply dataset repeat operation
    ds = ds.repeat(repeat_num)

    return ds


 def create_dataset_imagenet(dataset_path,
                            do_train,
                            repeat_num=1,
                            batch_size=32,
                            target="Ascend"):
    """
    create a train or eval imagenet dataset

    Args:
        dataset_path(string): the path of dataset.
        do_train(bool): whether dataset is used for train or eval.
        repeat_num(int): the repeat times of dataset. Default: 1
        batch_size(int): the batch size of dataset. Default: 32
        target(str): the device target. Default: Ascend

    Returns:
        dataset
    """
    if target == "Ascend":
        device_num, rank_id = _get_rank_info()
    else:
        init()
        rank_id = get_rank()
        device_num = get_group_size()

    if device_num == 1:
        ds = de.ImageFolderDataset(dataset_path,
                                   num_parallel_workers=8,
                                   shuffle=True)
    else:
        ds = de.ImageFolderDataset(dataset_path,
                                   num_parallel_workers=8,
                                   shuffle=True,
                                   num_shards=device_num,
                                   shard_id=rank_id)

    image_size = 227
    mean = [0.485 * 255, 0.456 * 255, 0.406 * 255]
    std = [0.229 * 255, 0.224 * 255, 0.225 * 255]

    # define map operations
    if do_train:
        trans = [
            C.RandomCropDecodeResize(image_size,
                                     scale=(0.08, 1.0),
                                     ratio=(0.75, 1.333)),
            C.RandomHorizontalFlip(prob=0.5),
            C.RandomColorAdjust(brightness=0.4, contrast=0.4, saturation=0.4),
            C.Normalize(mean=mean, std=std),
            C.CutOut(112),
            C.HWC2CHW()
        ]
    else:
        trans = [
            C.Decode(),
            C.Resize((256, 256)),
            C.CenterCrop(image_size),
            C.Normalize(mean=mean, std=std),
            C.HWC2CHW()
        ]

    type_cast_op = C2.TypeCast(mstype.int32)

    ds = ds.map(operations=type_cast_op,
                input_columns="label",
                num_parallel_workers=8)
    ds = ds.map(operations=trans,
                input_columns="image",
                num_parallel_workers=8)

    # apply batch operations
    ds = ds.batch(batch_size, drop_remainder=True)

    # apply dataset repeat operation
    ds = ds.repeat(repeat_num)

    return ds


 def _get_rank_info():
    """
    get rank size and rank id
    """
    rank_size = int(os.environ.get("RANK_SIZE", 1))

    if rank_size > 1:
        rank_size = get_group_size()
        rank_id = get_rank()
    else:
        rank_size = 1
        rank_id = 0

    return rank_size, rank_id
--- a/model_zoo/official/cv/squeezenet/src/lr_generator.py
+++ b/model_zoo/official/cv/squeezenet/src/lr_generator.py
@@ -0,0 +1,106 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """learning rate generator"""
 import math
 import numpy as np


 def get_lr(lr_init, lr_end, lr_max, total_epochs, warmup_epochs,
           pretrain_epochs, steps_per_epoch, lr_decay_mode):
    """
    generate learning rate array

    Args:
        lr_init(float): init learning rate
        lr_end(float): end learning rate
        lr_max(float): max learning rate
        total_epochs(int): total epoch of training
        warmup_epochs(int): number of warmup epochs
        pretrain_epochs(int): number of pretrain epochs
        steps_per_epoch(int): steps of one epoch
        lr_decay_mode(string): learning rate decay mode,
                               including steps, poly, linear or cosine

    Returns:
        np.array, learning rate array
    """

    lr_each_step = []
    total_steps = steps_per_epoch * total_epochs
    warmup_steps = steps_per_epoch * warmup_epochs
    pretrain_steps = steps_per_epoch * pretrain_epochs
    decay_steps = total_steps - warmup_steps

    if lr_decay_mode == 'steps':
        decay_epoch_index = [
            0.3 * total_steps, 0.6 * total_steps, 0.8 * total_steps
        ]
        for i in range(total_steps):
            if i < decay_epoch_index[0]:
                lr = lr_max
            elif i < decay_epoch_index[1]:
                lr = lr_max * 0.1
            elif i < decay_epoch_index[2]:
                lr = lr_max * 0.01
            else:
                lr = lr_max * 0.001
            lr_each_step.append(lr)

    elif lr_decay_mode == 'poly':
        for i in range(total_steps):
            if i < warmup_steps:
                lr = linear_warmup_lr(i, warmup_steps, lr_max, lr_init)
            else:
                base = (1.0 - (i - warmup_steps) / decay_steps)
                lr = lr_max * base * base
            lr_each_step.append(lr)

    elif lr_decay_mode == 'linear':
        for i in range(total_steps):
            if i < warmup_steps:
                lr = linear_warmup_lr(i, warmup_steps, lr_max, lr_init)
            else:
                lr = lr_max - (lr_max - lr_end) * (i -
                                                   warmup_steps) / decay_steps
            lr_each_step.append(lr)

    elif lr_decay_mode == 'cosine':
        for i in range(total_steps):
            if i < warmup_steps:
                lr = linear_warmup_lr(i, warmup_steps, lr_max, lr_init)
            else:
                linear_decay = (total_steps - i) / decay_steps
                cosine_decay = 0.5 * (
                    1 + math.cos(math.pi * 2 * 0.47 *
                                 (i - warmup_steps) / decay_steps))
                decayed = linear_decay * cosine_decay + 0.00001
                lr = lr_max * decayed
            lr_each_step.append(lr)

    else:
        raise NotImplementedError(
            'Learning rate decay mode [{:s}] cannot be recognized'.format(
                lr_decay_mode))

    lr_each_step = np.array(lr_each_step).astype(np.float32)
    learning_rate = lr_each_step[pretrain_steps:]

    return learning_rate


 def linear_warmup_lr(current_step, warmup_steps, base_lr, init_lr):
    lr_inc = (base_lr - init_lr) / warmup_steps
    lr = init_lr + lr_inc * current_step
    return lr
--- a/model_zoo/official/cv/squeezenet/src/squeezenet.py
+++ b/model_zoo/official/cv/squeezenet/src/squeezenet.py
@@ -0,0 +1,216 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """Squeezenet."""
 import mindspore.nn as nn
 from mindspore.common import initializer as weight_init
 from mindspore.ops import operations as P


 class Fire(nn.Cell):
    def __init__(self, inplanes, squeeze_planes, expand1x1_planes,
                 expand3x3_planes):
        super(Fire, self).__init__()
        self.inplanes = inplanes
        self.squeeze = nn.Conv2d(inplanes,
                                 squeeze_planes,
                                 kernel_size=1,
                                 has_bias=True)
        self.squeeze_activation = nn.ReLU()
        self.expand1x1 = nn.Conv2d(squeeze_planes,
                                   expand1x1_planes,
                                   kernel_size=1,
                                   has_bias=True)
        self.expand1x1_activation = nn.ReLU()
        self.expand3x3 = nn.Conv2d(squeeze_planes,
                                   expand3x3_planes,
                                   kernel_size=3,
                                   pad_mode='same',
                                   has_bias=True)
        self.expand3x3_activation = nn.ReLU()
        self.concat = P.Concat(axis=1)

    def construct(self, x):
        x = self.squeeze_activation(self.squeeze(x))
        return self.concat((self.expand1x1_activation(self.expand1x1(x)),
                            self.expand3x3_activation(self.expand3x3(x))))


 class SqueezeNet(nn.Cell):
    r"""SqueezeNet model architecture from the `"SqueezeNet: AlexNet-level
    accuracy with 50x fewer parameters and <0.5MB model size"
    <https://arxiv.org/abs/1602.07360>`_ paper.

    Get SqueezeNet neural network.

    Args:
        num_classes (int): Class number.

    Returns:
        Cell, cell instance of SqueezeNet neural network.

    Examples:
        >>> net = SqueezeNet(10)
    """
    def __init__(self, num_classes=10):
        super(SqueezeNet, self).__init__()

        self.features = nn.SequentialCell([
            nn.Conv2d(3,
                      96,
                      kernel_size=7,
                      stride=2,
                      pad_mode='valid',
                      has_bias=True),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=3, stride=2),
            Fire(96, 16, 64, 64),
            Fire(128, 16, 64, 64),
            Fire(128, 32, 128, 128),
            nn.MaxPool2d(kernel_size=3, stride=2),
            Fire(256, 32, 128, 128),
            Fire(256, 48, 192, 192),
            Fire(384, 48, 192, 192),
            Fire(384, 64, 256, 256),
            nn.MaxPool2d(kernel_size=3, stride=2),
            Fire(512, 64, 256, 256),
        ])

        # Final convolution is initialized differently from the rest
        self.final_conv = nn.Conv2d(512,
                                    num_classes,
                                    kernel_size=1,
                                    has_bias=True)
        self.dropout = nn.Dropout(keep_prob=0.5)
        self.relu = nn.ReLU()
        self.mean = P.ReduceMean(keep_dims=True)
        self.flatten = nn.Flatten()
        self.custom_init_weight()

    def custom_init_weight(self):
        """
        Init the weight of Conv2d in the net.
        """
        for _, cell in self.cells_and_names():
            if isinstance(cell, nn.Conv2d):
                if cell is self.final_conv:
                    cell.weight.set_data(
                        weight_init.initializer('normal', cell.weight.shape,
                                                cell.weight.dtype))
                else:
                    cell.weight.set_data(
                        weight_init.initializer('he_uniform',
                                                cell.weight.shape,
                                                cell.weight.dtype))
                if cell.bias is not None:
                    cell.bias.set_data(
                        weight_init.initializer('zeros', cell.bias.shape,
                                                cell.bias.dtype))

    def construct(self, x):
        x = self.features(x)
        x = self.dropout(x)
        x = self.final_conv(x)
        x = self.relu(x)
        x = self.mean(x, (2, 3))
        x = self.flatten(x)

        return x


 class SqueezeNet_Residual(nn.Cell):
    r"""SqueezeNet with simple bypass model architecture from the `"SqueezeNet:
    AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size"
    <https://arxiv.org/abs/1602.07360>`_ paper.

    Get SqueezeNet with simple bypass neural network.

    Args:
        num_classes (int): Class number.

    Returns:
        Cell, cell instance of SqueezeNet with simple bypass neural network.

    Examples:
        >>> net = SqueezeNet_Residual(10)
    """
    def __init__(self, num_classes=10):
        super(SqueezeNet_Residual, self).__init__()

        self.conv1 = nn.Conv2d(3,
                               96,
                               kernel_size=7,
                               stride=2,
                               pad_mode='valid',
                               has_bias=True)
        self.fire2 = Fire(96, 16, 64, 64)
        self.fire3 = Fire(128, 16, 64, 64)
        self.fire4 = Fire(128, 32, 128, 128)
        self.fire5 = Fire(256, 32, 128, 128)
        self.fire6 = Fire(256, 48, 192, 192)
        self.fire7 = Fire(384, 48, 192, 192)
        self.fire8 = Fire(384, 64, 256, 256)
        self.fire9 = Fire(512, 64, 256, 256)
        # Final convolution is initialized differently from the rest
        self.conv10 = nn.Conv2d(512, num_classes, kernel_size=1, has_bias=True)

        self.relu = nn.ReLU()
        self.max_pool2d = nn.MaxPool2d(kernel_size=3, stride=2)
        self.add = P.TensorAdd()
        self.dropout = nn.Dropout(keep_prob=0.5)
        self.mean = P.ReduceMean(keep_dims=True)
        self.flatten = nn.Flatten()
        self.custom_init_weight()

    def custom_init_weight(self):
        """
        Init the weight of Conv2d in the net.
        """
        for _, cell in self.cells_and_names():
            if isinstance(cell, nn.Conv2d):
                if cell is self.conv10:
                    cell.weight.set_data(
                        weight_init.initializer('normal', cell.weight.shape,
                                                cell.weight.dtype))
                else:
                    cell.weight.set_data(
                        weight_init.initializer('xavier_uniform',
                                                cell.weight.shape,
                                                cell.weight.dtype))
                if cell.bias is not None:
                    cell.bias.set_data(
                        weight_init.initializer('zeros', cell.bias.shape,
                                                cell.bias.dtype))

    def construct(self, x):
        x = self.conv1(x)
        x = self.relu(x)
        x = self.max_pool2d(x)
        x = self.fire2(x)
        x = self.add(x, self.fire3(x))
        x = self.fire4(x)
        x = self.max_pool2d(x)
        x = self.add(x, self.fire5(x))
        x = self.fire6(x)
        x = self.add(x, self.fire7(x))
        x = self.fire8(x)
        x = self.max_pool2d(x)
        x = self.add(x, self.fire9(x))
        x = self.dropout(x)
        x = self.conv10(x)
        x = self.relu(x)
        x = self.mean(x, (2, 3))
        x = self.flatten(x)

        return x
--- a/model_zoo/official/cv/squeezenet/train.py
+++ b/model_zoo/official/cv/squeezenet/train.py
@@ -0,0 +1,169 @@
 # Copyright 2020 Huawei Technologies Co., Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 # ============================================================================
 """train squeezenet."""
 import os
 import argparse
 from mindspore import context
 from mindspore import Tensor
 from mindspore.nn.optim.momentum import Momentum
 from mindspore.train.model import Model
 from mindspore.context import ParallelMode
 from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor
 from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits
 from mindspore.train.loss_scale_manager import FixedLossScaleManager
 from mindspore.train.serialization import load_checkpoint, load_param_into_net
 from mindspore.communication.management import init, get_rank, get_group_size
 from mindspore.common import set_seed
 from src.lr_generator import get_lr
 from src.CrossEntropySmooth import CrossEntropySmooth

 parser = argparse.ArgumentParser(description='Image classification')
 parser.add_argument('--net', type=str, default='squeezenet', choices=['squeezenet', 'squeezenet_residual'],
                    help='Model.')
 parser.add_argument('--dataset', type=str, default='cifar10', choices=['cifar10', 'imagenet'], help='Dataset.')
 parser.add_argument('--run_distribute', type=bool, default=False, help='Run distribute')
 parser.add_argument('--device_num', type=int, default=1, help='Device num.')
 parser.add_argument('--dataset_path', type=str, default=None, help='Dataset path')
 parser.add_argument('--device_target', type=str, default='Ascend', help='Device target')
 parser.add_argument('--pre_trained', type=str, default=None, help='Pretrained checkpoint path')
 args_opt = parser.parse_args()

 set_seed(1)

 if args_opt.net == "squeezenet":
    from src.squeezenet import SqueezeNet as squeezenet
    if args_opt.dataset == "cifar10":
        from src.config import config1 as config
        from src.dataset import create_dataset_cifar as create_dataset
    else:
        from src.config import config2 as config
        from src.dataset import create_dataset_imagenet as create_dataset
 else:
    from src.squeezenet import SqueezeNet_Residual as squeezenet
    if args_opt.dataset == "cifar10":
        from src.config import config3 as config
        from src.dataset import create_dataset_cifar as create_dataset
    else:
        from src.config import config4 as config
        from src.dataset import create_dataset_imagenet as create_dataset

 if __name__ == '__main__':
    target = args_opt.device_target
    ckpt_save_dir = config.save_checkpoint_path

    # init context
    context.set_context(mode=context.GRAPH_MODE,
                        device_target=target)
    if args_opt.run_distribute:
        if target == "Ascend":
            device_id = int(os.getenv('DEVICE_ID'))
            context.set_context(device_id=device_id,
                                enable_auto_mixed_precision=True)
            context.set_auto_parallel_context(
                device_num=args_opt.device_num,
                parallel_mode=ParallelMode.DATA_PARALLEL,
                gradients_mean=True)
            init()
        # GPU target
        else:
            init()
            context.set_auto_parallel_context(
                device_num=get_group_size(),
                parallel_mode=ParallelMode.DATA_PARALLEL,
                gradients_mean=True)
        ckpt_save_dir = config.save_checkpoint_path + "ckpt_" + str(
            get_rank()) + "/"

    # create dataset
    dataset = create_dataset(dataset_path=args_opt.dataset_path,
                             do_train=True,
                             repeat_num=1,
                             batch_size=config.batch_size,
                             target=target)
    step_size = dataset.get_dataset_size()

    # define net
    net = squeezenet(num_classes=config.class_num)

    # load checkpoint
    if args_opt.pre_trained:
        param_dict = load_checkpoint(args_opt.pre_trained)
        load_param_into_net(net, param_dict)

    # init lr
    lr = get_lr(lr_init=config.lr_init,
                lr_end=config.lr_end,
                lr_max=config.lr_max,
                total_epochs=config.epoch_size,
                warmup_epochs=config.warmup_epochs,
                pretrain_epochs=config.pretrain_epoch_size,
                steps_per_epoch=step_size,
                lr_decay_mode=config.lr_decay_mode)
    lr = Tensor(lr)

    # define loss
    if args_opt.dataset == "imagenet":
        if not config.use_label_smooth:
            config.label_smooth_factor = 0.0
        loss = CrossEntropySmooth(sparse=True,
                                  reduction='mean',
                                  smooth_factor=config.label_smooth_factor,
                                  num_classes=config.class_num)
    else:
        loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')

    # define opt, model
    if target == "Ascend":
        loss_scale = FixedLossScaleManager(config.loss_scale,
                                           drop_overflow_update=False)
        opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()),
                       lr,
                       config.momentum,
                       config.weight_decay,
                       config.loss_scale,
                       use_nesterov=True)
        model = Model(net,
                      loss_fn=loss,
                      optimizer=opt,
                      loss_scale_manager=loss_scale,
                      metrics={'acc'},
                      amp_level="O2",
                      keep_batchnorm_fp32=False)
    else:
        # GPU target
        opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()),
                       lr,
                       config.momentum,
                       config.weight_decay,
                       use_nesterov=True)
        model = Model(net, loss_fn=loss, optimizer=opt, metrics={'acc'})

    # define callbacks
    time_cb = TimeMonitor(data_size=step_size)
    loss_cb = LossMonitor()
    cb = [time_cb, loss_cb]
    if config.save_checkpoint:
        config_ck = CheckpointConfig(
            save_checkpoint_steps=config.save_checkpoint_epochs * step_size,
            keep_checkpoint_max=config.keep_checkpoint_max)
        ckpt_cb = ModelCheckpoint(prefix=args_opt.net + '_' + args_opt.dataset,
                                  directory=ckpt_save_dir,
                                  config=config_ck)
        cb += [ckpt_cb]

    # train model
    model.train(config.epoch_size - config.pretrain_epoch_size,
                dataset,
                callbacks=cb)