Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
|
|
4 years ago | |
|---|---|---|
| .. | ||
| model | 4 years ago | |
| scripts | 4 years ago | |
| src | 4 years ago | |
| .gitignore | 4 years ago | |
| Makefile | 4 years ago | |
| README.md | 4 years ago | |
| README_CN.md | 4 years ago | |
| prepare_and_run.sh | 4 years ago | |
This folder holds code for Training-on-Device of a LeNet model. Part of the code runs on a server using MindSpore infrastructure, another part uses MindSpore Lite conversion utility, and the last part is the actual training of the model on some android-based device.
LeNet is a very simple network which is composed of only 5 layers, 2 of which are convolutional layers and the remaining 3 are fully connected layers. Such a small network can be fully trained (from scratch) on a device in a short time. Therefore, it is a good example.
In this example we use the MNIST dataset of handwritten digits as published in THE MNIST DATABASE
Dataset size:52.4M,60,000 28*28 in 10 classes
Data format:binary files
The dataset directory structure is as follows:
mnist/
├── test
│ ├── t10k-images-idx3-ubyte
│ └── t10k-labels-idx1-ubyte
└── train
├── train-images-idx3-ubyte
└── train-labels-idx1-ubyte
After installing all the above mentioned, the script in the home directory could be run with the following arguments:
sh ./prepare_and_run.sh -D DATASET_PATH [-d MINDSPORE_DOCKER] [-r RELEASE.tar.gz] [-t arm64|x86]
where:
The provided prepare_and_run.sh script is performing the following:
.ms formatSee how to run the script and parameters definitions in the Quick Start Section
Within the model folder a prepare_model.sh script uses MindSpore infrastructure to export the model into a .mindir file. The user can specify a docker image on which MindSpore is installed. Otherwise, the python script will be run locally.
The script then converts the .mindir to a .ms format using the MindSpore ToD converter.
The script accepts a tar ball where the converter resides. Otherwise, the script will attempt to find the converter in the MindSpore ToD build output directory.
The lenet_tod.ms model file is then copied into the package folder as well as scripts, the MindSpore ToD library and the MNIST dataset.
Finally, the code (in src) is compiled for arm64 and the binary is copied into the package folder.
To run the code on the device the script first uses adb tool to push the package folder into the device. It then runs training (which takes some time) and finally runs evaluation of the trained model using the test data.
train_lenet/
├── Makefile # Makefile of src code
├── model
│ ├── lenet_export.py # Python script that exports the LeNet model to .mindir
│ ├── prepare_model.sh # script that export model (using docker) then converts it
│ └── train_utils.py # utility function used during the export
├── prepare_and_run.sh # main script that creates model, compiles it and send to device for running
├── README.md # English manual
├── README_CN.md # Chinese manual
├── scripts
│ ├── eval.sh # on-device script that load the train model and evaluates its accuracy
│ └── train.sh # on-device script that load the initial model and train it
├── src
│ ├── net_runner.cc # program that runs training/evaluation of models
│ ├── net_runner.h # net_runner header
│ └── utils.h # general utilities
When the prepare_and_run.sh script is run, the following folder is prepared. It is pushed to the device and then training runs
├── package
│ ├── bin
│ │ └── net_runner # the executable that performs the training/evaluation
│ ├── dataset
│ │ ├── test
│ │ │ ├── t10k-images-idx3-ubyte # test images
│ │ │ └── t10k-labels-idx1-ubyte # test labels
│ │ └── train
│ │ ├── train-images-idx3-ubyte # train images
│ │ └── train-labels-idx1-ubyte # train labels
│ ├── eval.sh # on-device script that load the train model and evaluates its accuracy
│ ├── lib
│ │ ├── libjpeg.so.62
│ │ ├── libminddata-lite.a
│ │ ├── libminddata-lite.so
│ │ ├── libmindspore-lite.a
│ │ ├── libmindspore-lite-jni.so
│ │ ├── libmindspore-lite.so
│ │ ├── libmindspore-lite-train.a
│ │ ├── libmindspore-lite-train-jni.so
│ │ ├── libmindspore-lite-train.so
│ │ ├── libturbojpeg.so.0
│ │ └── mindspore-lite-java.jar
│ ├── model
│ │ └── lenet_tod.ms # model to train
│ └── train.sh # on-device script that load the initial model and train it
阿对对队
C++ Python Text C Unity3D Asset other