add music auto tagging network.

model_zoo/official/audio/music_auto_tagging/README.md

@@ -0,0 +1,203 @@
+# Contents
+
+- [Music Auto Tagging Description](#fcn-4-description)
+- [Model Architecture](#model-architecture)
+- [Features](#features)
+    - [Mixed Precision](#mixed-precision)
+- [Environment Requirements](#environment-requirements)
+- [Quick Start](#quick-start)    
+- [Script Description](#script-description)
+    - [Script and Sample Code](#script-and-sample-code)
+    - [Script Parameters](#script-parameters)
+    - [Training Process](#training-process)
+        - [Training](#training)
+    - [Evaluation Process](#evaluation-process)
+        - [Evaluation](#evaluation)
+- [Model Description](#model-description)
+    - [Performance](#performance)  
+        - [Evaluation Performance](#evaluation-performance)
+- [ModelZoo Homepage](#modelzoo-homepage)
+
+
+# [Music Auto Tagging Description](#contents)
+
+This repository provides a script and recipe to train the Music Auto Tagging model to achieve state-of-the-art accuracy.
+
+[Paper](https://arxiv.org/abs/1606.00298):  `"Keunwoo Choi, George Fazekas, and Mark Sandler, “Automatic tagging using deep convolutional neural networks,” in International Society of Music Information Retrieval Conference. ISMIR, 2016."
+
+
+# [Model Architecture](#contents)
+
+Music Auto Tagging is a convolutional neural network architecture, its name Music Auto Tagging comes from the fact that it has 4 layers. Its layers consists of Convolutional layers, Max Pooling layers, Activation layers, Fully connected layers.
+
+# [Features](#contents)
+
+## Mixed Precision
+
+The [mixed precision](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/mixed_precision.html) training method accelerates the deep learning neural network training process by using both the single-precision and half-precision data formats, and maintains the network precision achieved by the single-precision training at the same time. Mixed precision training can accelerate the computation process, reduce memory usage, and enable a larger model or batch size to be trained on specific hardware. 
+For FP16 operators, if the input data type is FP32, the backend of MindSpore will automatically handle it with reduced precision. Users could check the reduced-precision operators by enabling INFO log and then searching ‘reduce precision’.
+
+
+# [Environment Requirements](#contents)
+
+- Hardware（Ascend
+  - If you want to try Ascend  , please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to ascend@huawei.com. Once approved, you can get the resources. 
+- Framework
+  - [MindSpore](https://www.mindspore.cn/install/en)
+- For more information, please check the resources below：
+  - [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html) 
+  - [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html)
+
+
+
+# [Quick Start](#contents)
+
+After installing MindSpore via the official website, you can start training and evaluation as follows: 
+
+### 1. Download and preprocess the dataset
+
+1. down load the classification dataset (for instance, MagnaTagATune Dataset, Million Song Dataset, etc)
+2. Extract the dataset
+3. The information file of each clip should contain the label and path. Please refer to the annotations_final.csv in MagnaTagATune Dataset.
+4. The provided pre-processing script use MagnaTagATune Dataset as an example. Please modify the code accprding to your own need.
+
+### 2. setup parameters (src/config.py)
+
+### 3. Train
+
+after having your dataset, first convert the audio clip into mindrecord dataset by using the following codes
+```shell
+python pre_process_data.py --device_id 0
+```
+
+Then, you can start training the model by using the following codes
+```shell
+SLOG_PRINT_TO_STDOUT=1 python train.py --device_id 0
+```
+
+### 4. Test
+
+Then you can test your model
+```shell
+SLOG_PRINT_TO_STDOUT=1 python eval.py --device_id 0
+```
+
+# [Script Description](#contents)
+
+## [Script and Sample Code](#contents)
+
+```
+├── model_zoo
+    ├── README.md                          // descriptions about all the models
+    ├── music_auto_tagging        
+        ├── README.md                    // descriptions about googlenet
+        ├── scripts 
+        │   ├──run_train.sh             // shell script for distributed on Ascend
+        │   ├──run_eval.sh              // shell script for evaluation on Ascend
+        │   ├──run_process_data.sh      // shell script for convert audio clips to mindrecord
+        ├── src
+        │   ├──dataset.py                     // creating dataset
+        │   ├──pre_process_data.py            // pre-process dataset
+        │   ├──musictagger.py                 // googlenet architecture
+        │   ├──config.py                      // parameter configuration
+        │   ├──loss.py                        // loss function
+        │   ├──tag.txt                        // tag for each number
+        ├── train.py               // training script 
+        ├── eval.py                //  evaluation script 
+        ├── export.py              //  export model in air format 
+```
+
+## [Script Parameters](#contents)
+
+Parameters for both training and evaluation can be set in config.py
+
+- config for Music Auto tagging
+
+  ```python
+  
+  'num_classes': 50                      # number of tagging classes
+  'num_consumer': 4                      # file number for mindrecord
+  'get_npy': 1 # mode for converting to npy, default 1 in this case
+  'get_mindrecord': 1 # mode for converting npy file into mindrecord file，default 1 in this case
+  'audio_path': "/dev/data/Music_Tagger_Data/fea/" # path to audio clips
+  'npy_path': "/dev/data/Music_Tagger_Data/fea/" # path to numpy
+  'info_path': "/dev/data/Music_Tagger_Data/fea/" # path to info_name, which provide the label of each audio clips
+  'info_name': 'annotations_final.csv'   # info_name
+  'device_target': 'Ascend'              # device running the program
+  'device_id': 0                         # device ID used to train or evaluate the dataset. Ignore it when you use run_train.sh for distributed training
+  'mr_path': '/dev/data/Music_Tagger_Data/fea/' # path to mindrecord
+  'mr_name': ['train', 'val']            # mindrecord name
+  
+  'pre_trained': False                   # whether training based on the pre-trained model
+  'lr': 0.0005                           # learning rate
+  'batch_size': 32                       # training batch size
+  'epoch_size': 10                       # total training epochs
+  'loss_scale': 1024.0                   # loss scale
+  'num_consumer': 4                      # file number for mindrecord
+  'mixed_precision': False               # if use mix precision calculation
+  'train_filename': 'train.mindrecord0'  # file name of the train mindrecord data 
+  'val_filename': 'val.mindrecord0'      # file name of the evaluation mindrecord data 
+  'data_dir': '/dev/data/Music_Tagger_Data/fea/' # directory of mindrecord data
+  'device_target': 'Ascend'              # device running the program
+  'device_id': 0,                        # device ID used to train or evaluate the dataset. Ignore it when you use run_train.sh for distributed training
+  'keep_checkpoint_max': 10,             # only keep the last keep_checkpoint_max checkpoint
+  'save_step': 2000,                     # steps for saving checkpoint
+  'checkpoint_path': '/dev/data/Music_Tagger_Data/model/',  # the absolute full path to save the checkpoint file
+  'prefix': 'MusicTagger',               # prefix of checkpoint
+  'model_name': 'MusicTagger_3-50_543.ckpt', # checkpoint name
+  ```
+
+
+## [Training Process](#contents)
+
+### Training 
+
+- running on Ascend
+
+  ```
+  python train.py > train.log 2>&1 & 
+  ```
+  
+  The python command above will run in the background, you can view the results through the file `train.log`.
+  
+  After training, you'll get some checkpoint files under the script folder by default. The loss value will be achieved as follows:
+  
+  ```
+  # grep "loss is " train.log
+  epoch: 1 step: 100, loss is 0.23264095
+  epoch: 1 step: 200, loss is 0.2013525
+  ...
+  ```
+  
+  The model checkpoint will be saved in the set directory. 
+
+## [Evaluation Process](#contents)
+
+### Evaluation
+
+
+# [Model Description](#contents)
+## [Performance](#contents)
+
+### Evaluation Performance 
+
+| Parameters                 | Ascend                                                      |
+| -------------------------- | ----------------------------------------------------------- |
+| Model Version              | FCN-4                                                       |
+| Resource                   | Ascend 910 ；CPU 2.60GHz，56cores；Memory，314G             |
+| uploaded Date              | 09/11/2020 (month/day/year)                                 |
+| MindSpore Version          | r0.7.0                                                |
+| Training Parameters        | epoch=10, steps=534, batch_size = 32, lr=0.005              |
+| Optimizer                  | Adam                                                        |
+| Loss Function              | Binary cross entropy                                        |
+| outputs                    | probability                                                 |
+| Loss                       | AUC 0.909                                                  |
+| Speed                      | 1pc: 160 samples/sec;                                       |
+| Total time                 | 1pc: 20 mins;                                               |
+| Checkpoint for Fine tuning | 198.73M(.ckpt file)                                         |
+| Scripts                    | [music_auto_tagging script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/audio/music_auto_tagging) |
+
+
+
+# [ModelZoo Homepage](#contents)  
+ Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).  

model_zoo/official/audio/music_auto_tagging/eval.py

@@ -0,0 +1,137 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+'''
+##############evaluate trained models#################
+python eval.py
+'''
+
+import argparse
+import numpy as np
+import mindspore.common.dtype as mstype
+from mindspore import context
+from mindspore import Tensor
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+from src.musictagger import MusicTaggerCNN
+from src.config import music_cfg as cfg
+from src.dataset import create_dataset
+
+
+def calculate_auc(labels_list, preds_list):
+    """
+    The AUC calculation function
+    Input:
+            labels_list: list of true label
+            preds_list:  list of predicted label
+    Outputs
+            Float, means of AUC
+    """
+    auc = []
+    n_bins = labels_list.shape[0] // 2
+    if labels_list.ndim == 1:
+        labels_list = labels_list.reshape(-1, 1)
+        preds_list = preds_list.reshape(-1, 1)
+    for i in range(labels_list.shape[1]):
+        labels = labels_list[:, i]
+        preds = preds_list[:, i]
+        postive_len = labels.sum()
+        negative_len = labels.shape[0] - postive_len
+        total_case = postive_len * negative_len
+        positive_histogram = np.zeros((n_bins))
+        negative_histogram = np.zeros((n_bins))
+        bin_width = 1.0 / n_bins
+
+        for j, _ in enumerate(labels):
+            nth_bin = int(preds[j] // bin_width)
+            if labels[j]:
+                positive_histogram[nth_bin] = positive_histogram[nth_bin] + 1
+            else:
+                negative_histogram[nth_bin] = negative_histogram[nth_bin] + 1
+
+        accumulated_negative = 0
+        satisfied_pair = 0
+        for k in range(n_bins):
+            satisfied_pair += (
+                positive_histogram[k] * accumulated_negative +
+                positive_histogram[k] * negative_histogram[k] * 0.5)
+            accumulated_negative += negative_histogram[k]
+        auc.append(satisfied_pair / total_case)
+
+    return np.mean(auc)
+
+
+def val(net, data_dir, filename, num_consumer=4, batch=32):
+    """
+    Validation function, estimate the performance of trained model
+
+    Input:
+            net:        the trained neural network
+            data_dir:       path to the validation dataset
+            filename:       name of the validation dataset
+            num_consumer:   split number of validation dataset
+            batch:          validation batch size
+    Outputs
+            Float, AUC
+    """
+    data_train = create_dataset(data_dir, filename, 32, ['feature', 'label'],
+                                num_consumer)
+    data_train = data_train.create_tuple_iterator()
+    res_pred = []
+    res_true = []
+    for data, label in data_train:
+        x = net(Tensor(data, dtype=mstype.float32))
+        res_pred.append(x.asnumpy())
+        res_true.append(label.asnumpy())
+    res_pred = np.concatenate(res_pred, axis=0)
+    res_true = np.concatenate(res_true, axis=0)
+    auc = calculate_auc(res_true, res_pred)
+    return auc
+
+
+def validation(net, model_path, data_dir, filename, num_consumer, batch):
+    param_dict = load_checkpoint(model_path)
+    load_param_into_net(net, param_dict)
+
+    auc = val(net, data_dir, filename, num_consumer, batch)
+    return auc
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description='Evaluate model')
+    parser.add_argument('--device_id',
+                        type=int,
+                        help='device ID',
+                        default=None)
+    args = parser.parse_args()
+
+    if args.device_id is not None:
+        context.set_context(device_target=cfg.device_target,
+                            mode=context.GRAPH_MODE,
+                            device_id=args.device_id)
+    else:
+        context.set_context(device_target=cfg.device_target,
+                            mode=context.GRAPH_MODE,
+                            device_id=cfg.device_id)
+
+    network = MusicTaggerCNN(in_classes=[1, 128, 384, 768, 2048],
+                             kernel_size=[3, 3, 3, 3, 3],
+                             padding=[0] * 5,
+                             maxpool=[(2, 4), (4, 5), (3, 8), (4, 8)],
+                             has_bias=True)
+    network.set_train(False)
+    auc_val = validation(network, cfg.checkpoint_path + "/" + cfg.model_name, cfg.data_dir,
+                         cfg.val_filename, cfg.num_consumer, cfg.batch_size)
+
+    print("=" * 10 + "Validation Peformance" + "=" * 10)
+    print("AUC: {:.5f}".format(auc_val))

model_zoo/official/audio/music_auto_tagging/export.py

@@ -0,0 +1,40 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+'''
+##############evaluate trained models#################
+python export.py
+'''
+
+import numpy as np
+from mindspore.train.serialization import export
+from mindspore import Tensor
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+from src.musictagger import MusicTaggerCNN
+from src.config import music_cfg as cfg
+
+if __name__ == "__main__":
+    network = MusicTaggerCNN(in_classes=[1, 128, 384, 768, 2048],
+                             kernel_size=[3, 3, 3, 3, 3],
+                             padding=[0] * 5,
+                             maxpool=[(2, 4), (4, 5), (3, 8), (4, 8)],
+                             has_bias=True)
+    param_dict = load_checkpoint(cfg.checkpoint_path + "/" + cfg.model_name)
+    load_param_into_net(network, param_dict)
+    input_data = np.random.uniform(0.0, 1.0, size=[1, 1, 96, 1366]).astype(np.float32)
+    export(network,
+           Tensor(input_data),
+           filename="{}/{}.air".format(cfg.checkpoint_path,
+                                       cfg.model_name[:-5]),
+           file_format="AIR")

model_zoo/official/audio/music_auto_tagging/scripts/run_eval.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+export SLOG_PRINT_TO_STDOUT=1
+python ../eval.py --device_id 0

model_zoo/official/audio/music_auto_tagging/scripts/run_process_data.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+export SLOG_PRINT_TO_STDOUT=1
+python ../src/pre_process_data.py --device_id 0

model_zoo/official/audio/music_auto_tagging/scripts/run_train.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+export SLOG_PRINT_TO_STDOUT=1
+python ../train.py --device_id 0

model_zoo/official/audio/music_auto_tagging/src/__init__.py

@@ -0,0 +1,23 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""
+__init__.py
+"""
+
+from . import musictagger
+from . import loss
+from . import dataset
+from . import config
+from . import pre_process_data

model_zoo/official/audio/music_auto_tagging/src/config.py

@@ -0,0 +1,53 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""
+network config setting, will be used in train.py, eval.py
+"""
+from easydict import EasyDict as edict
+
+data_cfg = edict({
+    'num_classes': 50,
+    'num_consumer': 4,
+    'get_npy': 1,
+    'get_mindrecord': 1,
+    'audio_path': "/dev/data/Music_Tagger_Data/fea/",
+    'npy_path': "/dev/data/Music_Tagger_Data/fea/",
+    'info_path': "/dev/data/Music_Tagger_Data/fea/",
+    'info_name': 'annotations_final.csv',
+    'device_target': 'Ascend',
+    'device_id': 0,
+    'mr_path': '/dev/data/Music_Tagger_Data/fea/',
+    'mr_name': ['train', 'val'],
+})
+
+music_cfg = edict({
+    'pre_trained': False,
+    'lr': 0.0005,
+    'batch_size': 32,
+    'epoch_size': 10,
+    'loss_scale': 1024.0,
+    'num_consumer': 4,
+    'mixed_precision': False,
+    'train_filename': 'train.mindrecord0',
+    'val_filename': 'val.mindrecord0',
+    'data_dir': '/dev/data/Music_Tagger_Data/fea/',
+    'device_target': 'Ascend',
+    'device_id': 0,
+    'keep_checkpoint_max': 10,
+    'save_step': 2000,
+    'checkpoint_path': '/dev/data/Music_Tagger_Data/model',
+    'prefix': 'MusicTagger',
+    'model_name': 'MusicTagger_3-50_543.ckpt',
+})

model_zoo/official/audio/music_auto_tagging/src/dataset.py

@@ -0,0 +1,30 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+'''python dataset.py'''
+
+import os
+import mindspore.dataset as ds
+
+
+def create_dataset(base_path, filename, batch_size, columns_list,
+                   num_consumer):
+    """Create dataset"""
+
+    path = os.path.join(base_path, filename)
+    dtrain = ds.MindDataset(path, columns_list, num_consumer)
+    dtrain = dtrain.shuffle(buffer_size=dtrain.get_dataset_size())
+    dtrain = dtrain.batch(batch_size, drop_remainder=True)
+
+    return dtrain

model_zoo/official/audio/music_auto_tagging/src/loss.py

@@ -0,0 +1,41 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""
+define loss
+"""
+from mindspore import nn
+import mindspore.common.dtype as mstype
+from mindspore.ops import operations as P
+
+
+
+class BCELoss(nn.Cell):
+    """
+    BCELoss
+    """
+    def __init__(self, record=None):
+        super(BCELoss, self).__init__(record)
+        self.sm_scalar = P.ScalarSummary()
+        self.cast = P.Cast()
+        self.record = record
+        self.weight = None
+        self.bce = P.BinaryCrossEntropy()
+
+    def construct(self, input_data, target):
+        target = self.cast(target, mstype.float32)
+        loss = self.bce(input_data, target, self.weight)
+        if self.record:
+            self.sm_scalar("loss", loss)
+        return loss

model_zoo/official/audio/music_auto_tagging/src/musictagger.py

@@ -0,0 +1,83 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+'''model'''
+
+from mindspore import nn
+from mindspore.ops import operations as P
+
+
+class MusicTaggerCNN(nn.Cell):
+    """
+    Music Tagger CNN
+    """
+    def __init__(self, in_classes, kernel_size, padding, maxpool, has_bias):
+        super(MusicTaggerCNN, self).__init__()
+        self.in_classes = in_classes
+        self.kernel_size = kernel_size
+        self.maxpool = maxpool
+        self.padding = padding
+        self.has_bias = has_bias
+        # build model
+        self.conv1 = nn.Conv2d(self.in_classes[0], self.in_classes[1],
+                               self.kernel_size[0])
+        self.conv2 = nn.Conv2d(self.in_classes[1], self.in_classes[2],
+                               self.kernel_size[1])
+        self.conv3 = nn.Conv2d(self.in_classes[2], self.in_classes[3],
+                               self.kernel_size[2])
+        self.conv4 = nn.Conv2d(self.in_classes[3], self.in_classes[4],
+                               self.kernel_size[3])
+
+        self.bn1 = nn.BatchNorm2d(self.in_classes[1])
+        self.bn2 = nn.BatchNorm2d(self.in_classes[2])
+        self.bn3 = nn.BatchNorm2d(self.in_classes[3])
+        self.bn4 = nn.BatchNorm2d(self.in_classes[4])
+
+        self.pool1 = nn.MaxPool2d(maxpool[0], maxpool[0])
+        self.pool2 = nn.MaxPool2d(maxpool[1], maxpool[1])
+        self.pool3 = nn.MaxPool2d(maxpool[2], maxpool[2])
+        self.pool4 = nn.MaxPool2d(maxpool[3], maxpool[3])
+        self.poolreduce = P.ReduceMax(keep_dims=False)
+        self.Act = nn.ReLU()
+        self.flatten = nn.Flatten()
+        self.dense = nn.Dense(2048, 50, activation='sigmoid')
+        self.sigmoid = nn.Sigmoid()
+
+    def construct(self, input_data):
+        """
+        Music Tagger CNN
+        """
+        x = self.conv1(input_data)
+        x = self.bn1(x)
+        x = self.Act(x)
+        x = self.pool1(x)
+
+        x = self.conv2(x)
+        x = self.bn2(x)
+        x = self.Act(x)
+        x = self.pool2(x)
+
+        x = self.conv3(x)
+        x = self.bn3(x)
+        x = self.Act(x)
+        x = self.pool3(x)
+
+        x = self.conv4(x)
+        x = self.bn4(x)
+        x = self.Act(x)
+        x = self.poolreduce(x, (2, 3))
+        x = self.flatten(x)
+        x = self.dense(x)
+
+        return x

model_zoo/official/audio/music_auto_tagging/src/pre_process_data.py

@@ -0,0 +1,226 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+'''python dataset.py'''
+
+import os
+import argparse
+import pandas as pd
+import numpy as np
+import librosa
+from mindspore.mindrecord import FileWriter
+from mindspore import context
+from src.config import data_cfg as cfg
+
+
+def compute_melgram(audio_path, save_path='', filename='', save_npy=True):
+    """
+    extract melgram feature from the audio and save as numpy array
+
+    Args:
+        audio_path (str): path to the audio clip.
+        save_path (str): path to save the numpy array.
+        filename (str): filename of the audio clip.
+
+    Returns:
+        numpy array.
+
+    """
+    SR = 12000
+    N_FFT = 512
+    N_MELS = 96
+    HOP_LEN = 256
+    DURA = 29.12  # to make it 1366 frame..
+
+    src, _ = librosa.load(audio_path, sr=SR)  # whole signal
+    n_sample = src.shape[0]
+    n_sample_fit = int(DURA * SR)
+
+    if n_sample < n_sample_fit:  # if too short
+        src = np.hstack((src, np.zeros((int(DURA * SR) - n_sample,))))
+    elif n_sample > n_sample_fit:  # if too long
+        src = src[(n_sample - n_sample_fit) // 2:(n_sample + n_sample_fit) //
+                  2]
+    logam = librosa.core.amplitude_to_db
+    melgram = librosa.feature.melspectrogram
+    ret = logam(
+        melgram(y=src, sr=SR, hop_length=HOP_LEN, n_fft=N_FFT, n_mels=N_MELS))
+    ret = ret[np.newaxis, np.newaxis, :]
+    if save_npy:
+
+        save_path = save_path + filename[:-4] + '.npy'
+        np.save(save_path, ret)
+    return ret
+
+
+def get_data(features_data, labels_data):
+    data_list = []
+    for i, (label, feature) in enumerate(zip(labels_data, features_data)):
+        data_json = {"id": i, "feature": feature, "label": label}
+        data_list.append(data_json)
+    return data_list
+
+
+def convert(s):
+    if s.isdigit():
+        return int(s)
+    return s
+
+
+def GetLabel(info_path, info_name):
+    """
+    separate dataset into training set and validation set
+
+    Args:
+        info_path (str): path to the information file.
+        info_name (str): name of the information file.
+
+    """
+    T = []
+    with open(info_path + '/' + info_name, 'rb') as info:
+        data = info.readline()
+        while data:
+            T.append([
+                convert(i[1:-1])
+                for i in data.strip().decode('utf-8').split("\t")
+            ])
+            data = info.readline()
+
+    annotation = pd.DataFrame(T[1:], columns=T[0])
+    count = []
+    for i in annotation.columns[1:-2]:
+        count.append([annotation[i].sum() / len(annotation), i])
+    count = sorted(count)
+    full_label = []
+    for i in count[-50:]:
+        full_label.append(i[1])
+    out = []
+    for i in T[1:]:
+        index = [k for k, x in enumerate(i) if x == 1]
+        label = [T[0][k] for k in index]
+        L = [str(0) for k in range(50)]
+        L.append(i[-1])
+        for j in label:
+            if j in full_label:
+                ind = full_label.index(j)
+                L[ind] = '1'
+        out.append(L)
+    out = np.array(out)
+
+    Train = []
+    Val = []
+
+    for i in out:
+        if np.random.rand() > 0.2:
+            Train.append(i)
+        else:
+            Val.append(i)
+    np.savetxt("{}/music_tagging_train_tmp.csv".format(info_path),
+               np.array(Train),
+               fmt='%s',
+               delimiter=',')
+    np.savetxt("{}/music_tagging_val_tmp.csv".format(info_path),
+               np.array(Val),
+               fmt='%s',
+               delimiter=',')
+
+
+def generator_md(info_name, file_path, num_classes):
+    """
+    generate numpy array from features of all audio clips
+
+    Args:
+        info_path (str): path to the information file.
+        file_path (str): path to the npy files.
+
+    Returns:
+        2 numpy array.
+
+    """
+    df = pd.read_csv(info_name, header=None)
+    df.columns = [str(i) for i in range(num_classes)] + ["mp3_path"]
+    data = []
+    label = []
+    for i in range(len(df)):
+        try:
+            data.append(
+                np.load(file_path + df.mp3_path.values[i][:-4] +
+                        '.npy').reshape(1, 96, 1366))
+            label.append(np.array(df[df.columns[:-1]][i:i + 1])[0])
+        except FileNotFoundError:
+            print("Exception occurred in generator_md.")
+    return np.array(data), np.array(label, dtype=np.int32)
+
+
+def convert_to_mindrecord(info_name, file_path, store_path, mr_name,
+                          num_classes):
+    """ convert dataset to mindrecord """
+    num_shard = 4
+    data, label = generator_md(info_name, file_path, num_classes)
+    schema_json = {
+        "id": {
+            "type": "int32"
+        },
+        "feature": {
+            "type": "float32",
+            "shape": [1, 96, 1366]
+        },
+        "label": {
+            "type": "int32",
+            "shape": [num_classes]
+        }
+    }
+
+    writer = FileWriter(
+        os.path.join(store_path, '{}.mindrecord'.format(mr_name)), num_shard)
+    datax = get_data(data, label)
+    writer.add_schema(schema_json, "music_tagger_schema")
+    writer.add_index(["id"])
+    writer.write_raw_data(datax)
+    writer.commit()
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description='get feature')
+    parser.add_argument('--device_id',
+                        type=int,
+                        help='device ID',
+                        default=None)
+    args = parser.parse_args()
+
+    if cfg.get_npy:
+        GetLabel(cfg.info_path, cfg.info_name)
+        dirname = os.listdir(cfg.audio_path)
+        for d in dirname:
+            file_name = os.listdir("{}/{}".format(cfg.audio_path, d))
+            if not os.path.isdir("{}/{}".format(cfg.npy_path, d)):
+                os.mkdir("{}/{}".format(cfg.npy_path, d))
+            for f in file_name:
+                compute_melgram("{}/{}/{}".format(cfg.audio_path, d, f),
+                                "{}/{}/".format(cfg.npy_path, d), f)
+
+    if cfg.get_mindrecord:
+        if args.device_id is not None:
+            context.set_context(device_target='Ascend',
+                                mode=context.GRAPH_MODE,
+                                device_id=args.device_id)
+        else:
+            context.set_context(device_target='Ascend',
+                                mode=context.GRAPH_MODE,
+                                device_id=cfg.device_id)
+        for cmn in cfg.mr_nam:
+            if cmn in ['train', 'val']:
+                convert_to_mindrecord('music_tagging_{}_tmp.csv'.format(cmn),
+                                      cfg.npy_path, cfg.mr_path, cmn,
+                                      cfg.num_classes)

model_zoo/official/audio/music_auto_tagging/src/tag.txt

@@ -0,0 +1,50 @@
+choral
+female voice
+metal
+country
+weird
+no voice
+cello
+harp
+beats
+female vocal
+male voice
+dance
+new age
+voice
+choir
+classic
+man
+solo
+sitar
+soft
+no vocal
+pop
+male vocal
+woman
+flute
+quiet
+loud
+harpsichord
+no vocals
+vocals
+singing
+male
+opera
+indian
+female
+synth
+vocal
+violin
+beat
+ambient
+piano
+fast
+rock
+electronic
+drums
+strings
+techno
+slow
+classical
+guitar

model_zoo/official/audio/music_auto_tagging/train.py

@@ -0,0 +1,109 @@
+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+'''
+##############train models#################
+python train.py
+'''
+import argparse
+from mindspore import context, nn
+from mindspore.train import Model
+from mindspore.common import set_seed
+from mindspore.train.loss_scale_manager import FixedLossScaleManager
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor
+from src.dataset import create_dataset
+from src.musictagger import MusicTaggerCNN
+from src.loss import BCELoss
+from src.config import music_cfg as cfg
+
+def train(model, dataset_direct, filename, columns_list, num_consumer=4,
+          batch=16, epoch=50, save_checkpoint_steps=2172, keep_checkpoint_max=50,
+          prefix="model", directory='./'):
+    """
+    train network
+    """
+    config_ck = CheckpointConfig(save_checkpoint_steps=save_checkpoint_steps,
+                                 keep_checkpoint_max=keep_checkpoint_max)
+    ckpoint_cb = ModelCheckpoint(prefix=prefix,
+                                 directory=directory,
+                                 config=config_ck)
+    data_train = create_dataset(dataset_direct, filename, batch, columns_list,
+                                num_consumer)
+
+
+    model.train(epoch,
+                data_train,
+                callbacks=[
+                    ckpoint_cb,
+                    LossMonitor(per_print_times=181),
+                    TimeMonitor()
+                ],
+                dataset_sink_mode=True)
+
+
+if __name__ == "__main__":
+    set_seed(1)
+    parser = argparse.ArgumentParser(description='Train model')
+    parser.add_argument('--device_id',
+                        type=int,
+                        help='device ID',
+                        default=None)
+
+    args = parser.parse_args()
+
+    if args.device_id is not None:
+        context.set_context(device_target='Ascend',
+                            mode=context.GRAPH_MODE,
+                            device_id=args.device_id)
+    else:
+        context.set_context(device_target='Ascend',
+                            mode=context.GRAPH_MODE,
+                            device_id=cfg.device_id)
+
+    context.set_context(enable_auto_mixed_precision=cfg.mixed_precision)
+    network = MusicTaggerCNN(in_classes=[1, 128, 384, 768, 2048],
+                             kernel_size=[3, 3, 3, 3, 3],
+                             padding=[0] * 5,
+                             maxpool=[(2, 4), (4, 5), (3, 8), (4, 8)],
+                             has_bias=True)
+
+    if cfg.pre_trained:
+        param_dict = load_checkpoint(cfg.checkpoint_path + '/' +
+                                     cfg.model_name)
+        load_param_into_net(network, param_dict)
+
+    net_loss = BCELoss()
+
+    network.set_train(True)
+    net_opt = nn.Adam(params=network.trainable_params(),
+                      learning_rate=cfg.lr,
+                      loss_scale=cfg.loss_scale)
+
+    loss_scale_manager = FixedLossScaleManager(loss_scale=cfg.loss_scale,
+                                               drop_overflow_update=False)
+    net_model = Model(network, net_loss, net_opt, loss_scale_manager=loss_scale_manager)
+
+    train(model=net_model,
+          dataset_direct=cfg.data_dir,
+          filename=cfg.train_filename,
+          columns_list=['feature', 'label'],
+          num_consumer=cfg.num_consumer,
+          batch=cfg.batch_size,
+          epoch=cfg.epoch_size,
+          save_checkpoint_steps=cfg.save_step,
+          keep_checkpoint_max=cfg.keep_checkpoint_max,
+          prefix=cfg.prefix,
+          directory=cfg.checkpoint_path + "_{}".format(cfg.device_id))
+    print("train success")