| @@ -0,0 +1,119 @@ | |||||
| # 目录 | |||||
| <!-- TOC --> | |||||
| - [概述](#概述) | |||||
| - [数据集](#数据集) | |||||
| - [环境要求](#环境要求) | |||||
| - [快速入门](#快速入门) | |||||
| - [脚本详述](#脚本详述) | |||||
| <!-- /TOC --> | |||||
| # 概述 | |||||
| 本文主要讲解如何在端侧进行LeNet模型训练。首先在服务器或个人笔记本上进行模型转换;然后在安卓设备上训练模型。LeNet由2层卷积和3层全连接层组成,模型结构简单,因此可以在设备上快速训练。 | |||||
| # Dataset | |||||
| 本例使用[MNIST手写字数据集](http://yann.lecun.com/exdb/mnist/) | |||||
| - 数据集大小:52.4M, 60,000 28*28 10类 | |||||
| - 测试集:10,000 images | |||||
| - 训练集:60,000 images | |||||
| - 数据格式:二进制文件 | |||||
| - 注意:数据处理会在dataset.cc中进行。 | |||||
| - 数据集目录结构如下: | |||||
| ```text | |||||
| mnist/ | |||||
| ├── test | |||||
| │ ├── t10k-images-idx3-ubyte | |||||
| │ └── t10k-labels-idx1-ubyte | |||||
| └── train | |||||
| ├── train-images-idx3-ubyte | |||||
| └── train-labels-idx1-ubyte | |||||
| ``` | |||||
| # 环境要求 | |||||
| - 服务器或个人笔记本 | |||||
| - [MindSpore Framework](https://www.mindspore.cn/install/en): 建议使用Docker安装 | |||||
| - [MindSpore ToD Framework](https://www.mindspore.cn/tutorial/tod/en/use/prparation.html) | |||||
| - [Android NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) | |||||
| - [Android SDK](https://developer.android.com/studio?hl=zh-cn#cmdline-tools) | |||||
| - Android移动设备 | |||||
| # 快速入门 | |||||
| 安装完毕,在`./mindspore/mindspore/lite/examples/train_lenet`目录下执行脚本,命令如下: | |||||
| ```bash | |||||
| sh ./prepare_and_run.sh -D DATASET_PATH [-d MINDSPORE_DOCKER] [-r RELEASE.tar.gz] [-t arm64|x86] | |||||
| ``` | |||||
| 其中,`DATASET_PATH`是数据集路径;`MINDSPORE_DOCKER`是运行MindSpore的docker镜像,如果没有使用docker环境,则使用本地运行;`REALEASE.tar.gz`为端侧运行时训练工具压缩包绝对路径;`-t`选项为设备处理器架构,默认为`arm64`,如果输入`x86`则本地运行。注意:若在不同平台执行训练,需在先执行脚本前运行`make clean`指令。 | |||||
| # 脚本详述 | |||||
| `prepare_and_run.sh`脚本的功能如下: | |||||
| - 将Python模型文件转换为`.ms`文件。 | |||||
| - 编译训练源码并将相关文件传输到设备端 | |||||
| - 设备端执行训练 | |||||
| 运行命令参见[快速入门](#快速入门) | |||||
| ## 模型准备 | |||||
| 脚本`prepare_model.sh`会基于MIndSpore架构将Python模型转换为`lenet_tod.mindir`模型;然后,使用MindSpore ToD 模型转换工具将`lenet_tod.mindir`文件转换为`lenet_tod.ms`文件。如果没有docker环境,则本地执行转换。 | |||||
| ## 模型训练 | |||||
| 将`lenet_tod.ms`模型文件、训练脚本、MindSpore ToD库文件和`MNIST`数据集拷贝到`package`文件夹。`/src`文件夹中代码将会被编译成arm64架构版本,生成的二进制文件会被拷贝至`package`文件夹。最后使用`adb`工具将`package`文件夹传输至设备端,并执行训练。 | |||||
| # 工程目录 | |||||
| ``` python | |||||
| train_lenet/ | |||||
| ├── Makefile # Makefile of src code | |||||
| ├── model | |||||
| │ ├── lenet_export.py # Python script that exports the LeNet model to .mindir | |||||
| │ ├── prepare_model.sh # script that export model (using docker) then converts it | |||||
| │ └── train_utils.py # utility function used during the export | |||||
| ├── prepare_and_run.sh # main script that creates model, compiles it and send to device for running | |||||
| ├── README.md # this manual | |||||
| ├── scripts | |||||
| │ ├── eval.sh # on-device script that load the train model and evaluates its accuracy | |||||
| │ ├── run_eval.sh # adb script that launches eval.sh | |||||
| │ ├── run_train.sh # adb script that launches train.sh | |||||
| │ └── train.sh # on-device script that load the initial model and train it | |||||
| ├── src | |||||
| │ ├── dataset.cc # dataset handler | |||||
| │ ├── dataset.h # dataset class header | |||||
| │ ├── net_runner.cc # program that runs training/evaluation of models | |||||
| │ └── net_runner.h # net_runner header | |||||
| ``` | |||||
| 在脚本`prepare_and_run.sh`运行前,必须确保以下目录结构正确,这些文件将被传入设备用于训练。 | |||||
| ``` python | |||||
| ├── package | |||||
| │ ├── bin | |||||
| │ │ └── net_runner # the executable that performs the training/evaluation | |||||
| │ ├── dataset | |||||
| │ │ ├── test | |||||
| │ │ │ ├── t10k-images-idx3-ubyte # test images | |||||
| │ │ │ └── t10k-labels-idx1-ubyte # test labels | |||||
| │ │ └── train | |||||
| │ │ ├── train-images-idx3-ubyte # train images | |||||
| │ │ └── train-labels-idx1-ubyte # train labels | |||||
| │ ├── eval.sh # on-device script that load the train model and evaluates its accuracy | |||||
| │ ├── lib | |||||
| │ │ └── libmindspore-lite.so # MindSpore Lite library | |||||
| │ ├── model | |||||
| │ │ └── lenet_tod.ms # model to train | |||||
| │ └── train.sh # on-device script that load the initial model and train it | |||||
| ``` | |||||
| @@ -21,7 +21,6 @@ | |||||
| #include <iostream> | #include <iostream> | ||||
| #include <fstream> | #include <fstream> | ||||
| #include <memory> | #include <memory> | ||||
| #include <filesystem> | |||||
| using LabelId = std::map<std::string, int>; | using LabelId = std::map<std::string, int>; | ||||
| @@ -18,7 +18,7 @@ | |||||
| #define MINDSPORE_LITE_EXAMPLES_TRAIN_LENET_SRC_NET_RUNNER_H_ | #define MINDSPORE_LITE_EXAMPLES_TRAIN_LENET_SRC_NET_RUNNER_H_ | ||||
| #include <tuple> | #include <tuple> | ||||
| #include <filesystem> | |||||
| #include <iomanip> | |||||
| #include <map> | #include <map> | ||||
| #include <vector> | #include <vector> | ||||
| #include <string> | #include <string> | ||||
| @@ -0,0 +1,131 @@ | |||||
| # 目录 | |||||
| <!-- TOC --> | |||||
| - [概述](#概述) | |||||
| - [数据集](#环境要求) | |||||
| - [环境要求](#环境要求) | |||||
| - [快速入门](#快速入门) | |||||
| - [工程目录](#工程目录) | |||||
| <!-- /TOC --> | |||||
| # 概述 | |||||
| 本文主要讲解如何在端侧基于[efficientNet](https://arxiv.org/abs/1905.11946)模型迁移学习训练。首先在服务器或个人笔记本上进行模型转换;然后在安卓设备上训练模型。示例代码中使用efficientNet预训练模型,除最后全连接层外均冻结。这种训练模式能有效降低计算能耗,适用于端侧训练。 | |||||
| # 数据集 | |||||
| [Places dataset](http://places2.csail.mit.edu/)数据集包含不同分辨率的图片,总大小约100GB。本例使用大小仅有500MB的验证集 [validation data of small images](http://places2.csail.mit.edu/download.html)。 | |||||
| - 数据集大小:501M,36, 500, 224*224 images 共365类 | |||||
| - 数据格式:jpeg | |||||
| > 注意 | |||||
| > - 当前发布版本中,数据通过dataset.cc中自定义的`DataSet`类加载。我们使用[ImageMagick convert tool](https://imagemagick.org/)进行数据预处理,包括图像裁剪、转换为BMP格式。 | |||||
| > - 本例将使用10分类而不是365类。 | |||||
| > - 训练、验证和测试数据集的比例分别是3:1:1。 | |||||
| - 验证集数据目录结构如下: | |||||
| ```text | |||||
| places | |||||
| ├── val_256 | |||||
| │ ├── Places365_val_00000001.jpg | |||||
| │ ├── Places365_val_00000002.jpg | |||||
| │ ├── ... | |||||
| │ ├── Places365_val_00036499.jpg | |||||
| │ └── Places365_val_00036500.jpg | |||||
| ``` | |||||
| # 环境要求 | |||||
| - 服务端 | |||||
| - [MindSpore Framework](https://www.mindspore.cn/install/en) - 建议使用安装docker环境 | |||||
| - [MindSpore ToD Framework](https://www.mindspore.cn/tutorial/tod/en/use/prparation.html) | |||||
| - [Android NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) | |||||
| - [Android SDK](https://developer.android.com/studio?hl=zh-cn#cmdline-tools) | |||||
| - [ImageMagick convert tool](https://imagemagick.org/) | |||||
| - Android设备端 | |||||
| # 快速入门 | |||||
| 安装完毕,在`./mindspore/mindspore/lite/examples/transfer_learning`目录下执行脚本,命令如下: | |||||
| ```bash | |||||
| sh ./prepare_and_run.sh -D DATASET_PATH [-d MINDSPORE_DOCKER] [-r RELEASE.tar.gz] [-t arm64|x86] | |||||
| ``` | |||||
| 其中,`DATASET_PATH`是数据集路径;`MINDSPORE_DOCKER`是运行MindSpore的docker镜像,如果没有使用docker环境,则使用本地运行;`REALEASE.tar.gz`为端侧运行时训练工具压缩包绝对路径;`-t`选项为设备处理器架构,默认为`arm64`,如果输入`x86`则本地运行。注意:若在不同平台执行训练,需在先执行脚本前运行`make clean`指令。 | |||||
| # 脚本详述 | |||||
| `prepare_and_run.sh`脚本的功能如下: | |||||
| - 将Python模型文件转换为`.ms`文件。 | |||||
| - 编译训练源码并将相关文件传输到设备端 | |||||
| - 设备端执行训练 | |||||
| 运行命令参见[快速入门](#快速入门) | |||||
| ## 模型准备 | |||||
| 脚本`prepare_model.sh`会基于MIndSpore架构将Python模型转换为`lenet_tod.mindir`模型;然后,使用MindSpore ToD 模型转换工具将`lenet_tod.mindir`文件转换为`lenet_tod.ms`文件。如果没有docker环境,则本地执行转换。 | |||||
| ## 模型训练 | |||||
| 首先编译`/src`文件夹中训练代码源码,生成的二进制文件在`./bin`目录下;然后将`transfer_learning_tod.ms`模型文件、训练脚本、MindSpore ToD库文件、编译生成的`/bin`目录和预处理后的`Places`数据集拷贝到`package`文件夹;最后使用`adb`工具将`package`文件夹传输至设备端并执行训练。 | |||||
| # 工程目录 | |||||
| ```text | |||||
| transfer_learning/ | |||||
| ├── Makefile # Makefile of src code | |||||
| ├── model | |||||
| │ ├── effnet.py # Python implementation of efficientNet | |||||
| │ ├── transfer_learning_export.py # Python script that exports the LeNet model to .mindir | |||||
| │ ├── prepare_model.sh # script that export model (using docker) then converts it | |||||
| │ └── train_utils.py # utility function used during the export | |||||
| ├── prepare_and_run.sh # main script that creates model, compiles it and send to device for running | |||||
| ├── prepare_dataset.sh # prepares the Places dataset (crop/convert/organizing folders) | |||||
| ├── README.md # this manual | |||||
| ├── scripts | |||||
| │ ├── eval.sh # script that load the train model and evaluates its accuracy | |||||
| │ ├── eval_untrained.sh # script that load the untrained model and evaluates its accuracy | |||||
| │ ├── places365_val.txt # association of images to classes withiin the Places 365 dataset | |||||
| │ └── train.sh # script that load the initial model and trains it | |||||
| ├── src | |||||
| │ ├── dataset.cc # dataset handler | |||||
| │ ├── dataset.h # dataset class header | |||||
| │ ├── net_runner.cc # program that runs training/evaluation of models | |||||
| │ └── net_runner.h # net_runner header | |||||
| ``` | |||||
| 在脚本`prepare_and_run.sh`运行前,必须确保以下目录结构正确,这些文件将被传入设备用于训练。 | |||||
| ```text | |||||
| package-arm64/ | |||||
| ├── bin | |||||
| │ └── net_runner # the executable that performs the training/evaluation | |||||
| ├── dataset | |||||
| │ ├── 0 # folder containing images 0-99 belonging to 0'th class | |||||
| │ │ ├── 0.bmp | |||||
| │ │ ├── 1.bmp | |||||
| │ │ ├── .... | |||||
| │ │ ├── 98.bmp | |||||
| │ │ └── 99.bmp | |||||
| │ ├── ... # folders containing images 0-99 belonging to 1'st-8'th classes | |||||
| │ ├── 9 # folder containing images 0-99 belonging to 9'th class | |||||
| │ │ ├── 0.bmp | |||||
| │ │ ├── 1.bmp | |||||
| │ │ ├── .... | |||||
| │ │ ├── 98.bmp | |||||
| │ │ └── 99.bmp | |||||
| ├── lib | |||||
| │ └── libmindspore-lite.so # MindSpore Lite library | |||||
| ├── model | |||||
| │ └── transfer_learning_tod.ms # model to train | |||||
| ├── eval.sh # script that load the train model and evaluates its accuracy | |||||
| ├── eval_untrained.sh # script that load the untrain model and evaluates its accuracy | |||||
| └── train.sh # script that load the initial model and train it | |||||
| ``` | |||||
| @@ -22,7 +22,6 @@ | |||||
| #include <iostream> | #include <iostream> | ||||
| #include <fstream> | #include <fstream> | ||||
| #include <memory> | #include <memory> | ||||
| #include <filesystem> | |||||
| #pragma pack(push, 1) | #pragma pack(push, 1) | ||||
| @@ -18,7 +18,7 @@ | |||||
| #define MINDSPORE_LITE_EXAMPLES_TRANSFER_LEARNING_SRC_NET_RUNNER_H_ | #define MINDSPORE_LITE_EXAMPLES_TRANSFER_LEARNING_SRC_NET_RUNNER_H_ | ||||
| #include <tuple> | #include <tuple> | ||||
| #include <filesystem> | |||||
| #include <iomanip> | |||||
| #include <map> | #include <map> | ||||
| #include <vector> | #include <vector> | ||||
| #include <string> | #include <string> | ||||