Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
|
|
4 years ago | |
|---|---|---|
| .. | ||
| scripts | 4 years ago | |
| src | 4 years ago | |
| README.md | 4 years ago | |
| eval.py | 4 years ago | |
| train.py | 4 years ago | |
SimplePoseNet is a convolution-based neural network for the task of human pose estimation and tracking. It provides baseline methods that are surprisingly simple and effective, thus helpful for inspiring and evaluating new ideas for the field. State-of-the-art results are achieved on challenging benchmarks. More detail about this model can be found in:
B. Xiao, H. Wu, and Y. Wei, “Simple baselines for human pose estimation and tracking,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 472–487.
This repository contains a Mindspore implementation of SimplePoseNet based upon Microsoft's original Pytorch implementation (https://github.com/microsoft/human-pose-estimation.pytorch). The training and validating scripts are also included, and the evaluation results are shown in the Performance section.
The overall network architecture of SimplePoseNet is shown below:
Note that you can run the scripts based on the dataset mentioned in original paper or widely used in relevant domain/network architecture. In the following sections, we will introduce how to run the scripts using the related dataset below.
Dataset used: COCO2017
src/dataset.pyThe mixed precision training method accelerates the deep learning neural network training process by using both the single-precision and half-precision data formats, and maintains the network precision achieved by the single-precision training at the same time. Mixed precision training can accelerate the computation process, reduce memory usage, and enable a larger model or batch size to be trained on specific hardware. For FP16 operators, if the input data type is FP32, the backend of MindSpore will automatically handle it with reduced precision. Users could check the reduced-precision operators by enabling INFO log and then searching ‘reduce precision’.
To run the python scripts in the repository, you need to prepare the environment as follow:
SimplePoseNet use COCO2017 dataset to train and validate in this repository. Download the dataset from official website. You can place the dataset anywhere and tell the scripts where it is by modifying the DATASET.ROOT setting in configuration file src/config.py. For more information about the configuration file, please refer to Script Parameters.
You also need the person detection result of COCO val2017 to reproduce the multi-person pose estimation results, as mentioned in Dataset. Please checkout the author's repository, download and extract them under <ROOT>/experiments/, and make them look like this:
└─ <ROOT>
└─ experiments
└─ COCO_val2017_detections_AP_H_56_person.json
Before you start your training process, you need to obtain mindspore imagenet pretrained models. The model weight file can be obtained by running the Resnet training script in official model zoo. We also provide a pretrained model that can be used to train SimplePoseNet directly in GoogleDrive. The model file should be placed under <ROOT>/models/ like this:
└─ <ROOT>
└─ models
└─resnet50.ckpt
To train the model, run the shell script scripts/train_standalone.sh with the format below:
sh scripts/train_standalone.sh [device_id] [ckpt_path_to_save]
To validate the model, change the settings in src/config.py to the path of the model you want to validate. For example:
config.TEST.MODEL_FILE='results/xxxx.ckpt'
Then, run the shell script scripts/eval.sh with the format below:
sh scripts/eval.sh [device_id]
The structure of the files in this repository is shown below.
└─ mindspore-simpleposenet
├─ scripts
│ ├─ eval.sh // launch ascend standalone evaluation
│ ├─ train_distributed.sh // launch ascend distributed training
│ └─ train_standalone.sh // launch ascend standalone training
├─ src
│ ├─utils
│ │ ├─ transform.py // utils about image transformation
│ │ └─ nms.py // utils about nms
│ ├─evaluate
│ │ └─ coco_eval.py // evaluate result by coco
│ ├─ config.py // network and running config
│ ├─ dataset.py // dataset processor and provider
│ ├─ model.py // SimplePoseNet implementation
│ ├─ network_define.py // define loss
│ └─ predict.py // predict keypoints from heatmaps
├─ eval.py // evaluation script
├─ param_convert.py // model parameters conversion script
├─ train.py // training script
└─ README.md // descriptions about this repository
Configurations for both training and evaluation are set in src/config.py. All the settings are shown following.
# pose_resnet related params
POSE_RESNET.HEATMAP_SIZE = [48, 64] # heatmap size
POSE_RESNET.SIGMA = 2 # Gaussian hyperparameter in heatmap generation
POSE_RESNET.FINAL_CONV_KERNEL = 1 # final convolution kernel size
POSE_RESNET.DECONV_WITH_BIAS = False # deconvolution bias
POSE_RESNET.NUM_DECONV_LAYERS = 3 # the number of deconvolution layers
POSE_RESNET.NUM_DECONV_FILTERS = [256, 256, 256] # the filter size of deconvolution layers
POSE_RESNET.NUM_DECONV_KERNELS = [4, 4, 4] # kernel size of deconvolution layers
POSE_RESNET.NUM_LAYERS = 50 # number of layers(for resnet)
# common params for NETWORK
config.MODEL.NAME = 'pose_resnet' # model name
config.MODEL.INIT_WEIGHTS = True # init model weights by resnet
config.MODEL.PRETRAINED = './models/resnet50.ckpt' # pretrained model
config.MODEL.NUM_JOINTS = 17 # the number of keypoints
config.MODEL.IMAGE_SIZE = [192, 256] # image size
# dataset
config.DATASET.ROOT = '/data/coco2017/' # coco2017 dataset root
config.DATASET.TEST_SET = 'val2017' # folder name of test set
config.DATASET.TRAIN_SET = 'train2017' # folder name of train set
# data augmentation
config.DATASET.FLIP = True # random flip
config.DATASET.ROT_FACTOR = 40 # random rotation
config.DATASET.SCALE_FACTOR = 0.3 # random scale
# for train
config.TRAIN.BATCH_SIZE = 64 # batch size
config.TRAIN.BEGIN_EPOCH = 0 # begin epoch
config.TRAIN.END_EPOCH = 140 # end epoch
config.TRAIN.LR = 0.001 # initial learning rate
config.TRAIN.LR_FACTOR = 0.1 # learning rate reduce factor
config.TRAIN.LR_STEP = [90,120] # step to reduce lr
# test
config.TEST.BATCH_SIZE = 32 # batch size
config.TEST.FLIP_TEST = True # flip test
config.TEST.POST_PROCESS = True # post process
config.TEST.SHIFT_HEATMAP = True # shift heatmap
config.TEST.USE_GT_BBOX = False # use groundtruth bbox
config.TEST.MODEL_FILE = '' # model file to test
# detect bbox file
config.TEST.COCO_BBOX_FILE = 'experiments/COCO_val2017_detections_AP_H_56_person.json'
# nms
config.TEST.OKS_THRE = 0.9 # oks threshold
config.TEST.IN_VIS_THRE = 0.2 # visible threshold
config.TEST.BBOX_THRE = 1.0 # bbox threshold
config.TEST.IMAGE_THRE = 0.0 # image threshold
config.TEST.NMS_THRE = 1.0 # nms threshold
Run scripts/train_standalone.sh to train the model standalone. The usage of the script is:
sh scripts/train_standalone.sh [device_id] [ckpt_path_to_save]
For example, you can run the shell command below to launch the training procedure.
sh scripts/train_standalone.sh 0 results/standalone/
The script will run training in the background, you can view the results through the file train_log[X].txt as follows:
loading parse...
batch size :128
loading dataset from /data/coco2017/train2017
loaded 149813 records from coco dataset.
loading pretrained model ./models/resnet50.ckpt
start training, epoch size = 140
epoch: 1 step: 1170, loss is 0.000699
Epoch time: 492271.194, per step time: 420.745
epoch: 2 step: 1170, loss is 0.000586
Epoch time: 456265.617, per step time: 389.971
...
The model checkpoint will be saved into [ckpt_path_to_save].
Run scripts/train_distributed.sh to train the model distributed. The usage of the script is:
sh scripts/train_distributed.sh [rank_table] [ckpt_path_to_save] [device_number]
For example, you can run the shell command below to launch the distributed training procedure.
sh scripts/train_distributed.sh /home/rank_table.json results/distributed/ 4
The above shell script will run distribute training in the background. You can view the results through the file train_parallel[X]/log.txt as follows:
loading parse...
batch size :64
loading dataset from /data/coco2017/train2017
loaded 149813 records from coco dataset.
loading pretrained model ./models/resnet50.ckpt
start training, epoch size = 140
epoch: 1 step: 585, loss is 0.0007944
Epoch time: 236219.684, per step time: 403.794
epoch: 2 step: 585, loss is 0.000617
Epoch time: 164792.001, per step time: 281.696
...
The model checkpoint will be saved into [ckpt_path_to_save].
Change the settings in src/config.py to the path of the model you want to validate. For example:
config.TEST.MODEL_FILE='results/xxxx.ckpt'
Then, run scripts/eval.sh to evaluate the model with one Ascend processor. The usage of the script is:
sh scripts/eval.sh [device_id]
For example, you can run the shell command below to launch the validation procedure.
sh scripts/eval.sh 0
The above shell command will run validation procedure in the background. You can view the results through the file eval_log[X].txt. The result will be achieved as follows:
use flip test: True
loading model ckpt from results/distributed/sim-140_1170.ckpt
loading dataset from /data/coco2017/val2017
loading bbox file from experiments/COCO_val2017_detections_AP_H_56_person.json
Total boxes: 104125
1024 samples validated in 18.133189916610718 seconds
2048 samples validated in 4.724390745162964 seconds
...
| Parameters | Standalone | Distributed |
|---|---|---|
| Model Version | SimplePoseNet | SimplePoseNet |
| Resource | Ascend 910; OS Euler2.8 | 4 Ascend 910 cards; OS Euler2.8 |
| Uploaded Date | 12/18/2020 (month/day/year) | 12/18/2020 (month/day/year) |
| MindSpore Version | 1.1.0 | 1.1.0 |
| Dataset | COCO2017 | COCO2017 |
| Training Parameters | epoch=140, batch_size=128 | epoch=140, batch_size=64 |
| Optimizer | Adam | Adam |
| Loss Function | Mean Squared Error | Mean Squared Error |
| Outputs | heatmap | heatmap |
| Train Performance | mAP: 70.4 | mAP: 70.4 |
| Speed | 1pc: 389.915 ms/step | 4pc: 281.356 ms/step |
In src/dataset.py, we set the seed inside “create_dataset" function. We also use random seed in src/model.py to initial network weights.
Please check the official homepage.
MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
C++ Python Text Unity3D Asset C other