@@ -6,46 +6,47 @@ MindSpore Deep Probabilistic Programming (MDP) is a programming library for Baye
The objective of MDP is to integrate deep learning with Bayesian learning. On the one hand, similar to other Deep Probabilistic Programming Languages (DPPL) (e.g., TFP, Pyro), for the professional Bayesian learning researchers, MDP provides probability sampling, inference algorithms, and model building libraries; On the other hand, MDP provides high-level APIs for DNN researchers that are unfamiliar with Bayesian models, making it possible to take advantage of Bayesian models without the need of changing their DNN programming logics.
## Layer 0: High performance kernels for different platforms
**Layer 0: High performance kernels for different platforms**
- Random sampling kernels;
- Mathematical kernels that are used by Bayesian models.
- Random sampling kernels;
- Mathematical kernels that are used by Bayesian models.
## Layer 1: Probabilistic Programming (PP) focuses on professional Bayesian learning
**Layer 1: Probabilistic Programming (PP) focuses on professional Bayesian learning**
**Layer 1-1: Statistical distributions classes used to generate stochastic tensors**
### Layer 1-1: Statistical distributions classes used to generate stochastic tensors
- Distributions ([mindspore.nn.probability.distribution](https://gitee.com/mindspore/mindspore/tree/master/mindspore/nn/probability/distribution)): A large collection of probability distributions.
- Bijectors([mindspore.nn.probability.bijectors](https://gitee.com/mindspore/mindspore/tree/master/mindspore/nn/probability/bijector)): Reversible and composable transformations of random variables.
**Layer 1-2: Probabilistic inference algorithms**
### Layer 1-2: Probabilistic inference algorithms
- SVI([mindspore.nn.probability.infer.variational](https://gitee.com/mindspore/mindspore/tree/master/mindspore/nn/probability/infer/variational)): A unified interface for stochastic variational inference.
- MC: Algorithms for approximating integrals via sampling.
**Layer 2: Deep Probabilistic Programming (DPP) aims to provide composable BNN modules**
## Layer 2: Deep Probabilistic Programming (DPP) aims to provide composable BNN modules
- Layers([mindspore.nn.probability.bnn_layers](https://gitee.com/mindspore/mindspore/tree/master/mindspore/nn/probability/bnn_layers)): BNN layers, which are used to construct BNN.
- Dpn([mindspore.nn.probability.dpn](https://gitee.com/mindspore/mindspore/tree/master/mindspore/nn/probability/dpn)): A bunch of BNN models that allow to be integrated into DNN;
- Transform([mindspore.nn.probability.transforms](https://gitee.com/mindspore/mindspore/tree/master/mindspore/nn/probability/transforms)): Interfaces for the transformation between BNN and DNN;
- Context: context managers for models and layers.
**Layer 3: Toolbox provides a set of BNN tools for some specific applications**
## Layer 3: Toolbox provides a set of BNN tools for some specific applications
- Uncertainty Estimation([mindspore.nn.probability.toolbox.uncertainty_evaluation](https://gitee.com/mindspore/mindspore/tree/master/mindspore/nn/probability/toolbox/uncertainty_evaluation.py)): Interfaces to estimate epistemic uncertainty and aleatoric uncertainty.
- OoD detection: Interfaces to detect out of distribution samples.

MDP requires MindSpore version 0.7.0-beta or later. MDP is actively evolving. Interfaces may change as Mindspore releases are iteratively updated.
### Tutorial
**Bayesian Neural Network**
## Tutorial
### Bayesian Neural Network
1. Process the required dataset. The MNIST dateset is used in the example. Data processing is consistent with [Implementing an Image Classification Application](https://www.mindspore.cn/tutorial/training/en/master/quick_start/quick_start.html) in Tutorial.
2. Define a Bayesian Neural Network. The bayesian LeNet is used in this example.
```
```python
import mindspore.nn as nn
from mindspore.nn.probability import bnn_layers
import mindspore.ops.operations as P
@@ -91,12 +92,14 @@ class BNNLeNet5(nn.Cell):
x = self.fc3(x)
return x
```
The way to construct Bayesian Neural Network by bnn_layers is the same as DNN. It's worth noting that bnn_layers and traditional layers of DNN can be combined with each other.
3. Define the Loss Function and Optimizer
The loss function `SoftmaxCrossEntropyWithLogits` and the optimizer `AdamWeightDecay` are used in the example. Call the loss function and optimizer in the `__main__` function.
```
```python
if __name__ == "__main__":
...
# define the loss function
@@ -106,10 +109,11 @@ if __name__ == "__main__":
```
4. Train the Network
The process of Bayesian network training is basically the same as that of DNN, the only differance is that WithLossCell is replaced with WithBNNLossCell suitable for BNN.
The process of Bayesian network training is basically the same as that of DNN, the only difference is that WithLossCell is replaced with WithBNNLossCell suitable for BNN.
Based on the two parameters `backbone` and `loss_fn` in WithLossCell, WithBNNLossCell adds two parameters of `dnn_factor` and `bnn_factor`. Those two parameters are used to trade off backbone's loss and kl loss to prevent kl loss from being too large to cover backbone's loss.
```
```python
from mindspore.nn import TrainOneStepCell
if __name__ == "__main__":
@@ -133,7 +137,7 @@ if __name__ == "__main__":
The `train_model` and `validate_model` are defined as follows:
3. Process the required dataset. The MNIST dateset is used in the example. Data processing is consistent with [Implementing an Image Classification Application](https://www.mindspore.cn/tutorial/training/en/master/quick_start/quick_start.html) in Tutorial.
4. Use SVI interface to train VAE network. vi.run can return the trained network, get_train_loss can get the loss after training.
```
```python
from mindspore.nn.probability.infer import SVI
vi = SVI(net_with_loss=net_with_loss, optimizer=optimizer)
vae = vi.run(train_dataset=ds_train, epochs=10)
trained_loss = vi.get_train_loss()
```
5. Use the trained VAE network, we can generate new samples or reconstruct the input samples.
For DNN researchers who are unfamiliar with Bayesian models, MDP provides high-level APIs `TransformToBNN` to support one-click conversion of DNN models to BNN models.
1. Define a Deep Neural Network. The LeNet is used in this example.
```
```python
from mindspore.common.initializer import TruncatedNormal
import mindspore.nn as nn
import mindspore.ops.operations as P
@@ -314,7 +323,7 @@ class LeNet5(nn.Cell):
2. Wrap DNN by TrainOneStepCell
```
```python
from mindspore.nn import WithLossCell, TrainOneStepCell
if __name__ == "__main__":
@@ -328,9 +337,10 @@ if __name__ == "__main__":
```
3. Instantiate class `TransformToBNN`
The `__init__` of `TransformToBNN` are as follows:
The arg `trainable_dnn` specifies a trainable DNN model wrapped by TrainOneStepCell, `dnn_factor` is the coefficient of backbone's loss, which is computed by loss function, and `bnn_factor` is the coefficient of kl loss, which is kl divergence of Bayesian layer. `dnn_factor` and `bnn_factor` are used to trade off backbone's loss and kl loss to prevent kl loss from being too large to cover backbone's loss.
```
```python
from mindspore.nn.probability import transforms
if __name__ == "__main__":
@@ -355,7 +366,7 @@ if __name__ == "__main__":
3-1. Transform the whole model
The method `transform_to_bnn_model` can transform both convolutional layer and full connection layer of DNN model to BNN model. Its code is as follows:
@@ -382,9 +393,10 @@ The method `transform_to_bnn_model` can transform both convolutional layer and f
Cell, a trainable BNN model wrapped by TrainOneStepCell.
"""
```
Arg `get_dense_args` specifies which arguments to be gotten from full connection layer of DNN. Its Default value contains arguments common to nn.Dense and DenseReparameterization. Arg `get_conv_args` specifies which arguments to be gotten from convolutional layer of DNN. Its Default value contains arguments common to nn.Con2d and ConvReparameterization. Arg `add_dense_args` and `add_conv_args` specify which arguments to be add to full connection layer and convolutional layer of BNN. Note that the parameters in `add_dense_args` cannot be repeated with `get_dense_args`, so do `add_conv_args` and `get_conv_args`.
The method `transform_to_bnn_layer` can transform a specific type of layers (nn.Dense or nn.Conv2d) in DNN model to corresponding BNN layer. Its code is as follows:
Transform a specific type of layers in DNN model to corresponding BNN layer.
@@ -411,20 +423,23 @@ The method `transform_to_bnn_layer` can transform a specific type of layers (nn.
Cell, a trainable model wrapped by TrainOneStepCell, whose sprcific type of layer is transformed to the corresponding bayesian layer.
"""
```
Arg `dnn_layer` specifies which type of DNN layer to be transformed to BNN layer. The optional values are nn.Dense and nn.Conv2d. Arg `bnn_layer` specifies which type of BNN layer to be transformed to. The value should correspond to dnn_layer. Arg `get_args` and `add_args` specify the arguments gotten from DNN layer and the new arguments added to BNN layer respectively.
The uncertainty estimation toolbox is based on MindSpore Deep Probabilistic Programming (MDP), and it is suitable for mainstream deep learning models, such as regression, classification, target detection and so on. In the inference stage, with the uncertainy estimation toolbox, developers only need to pass in the trained model and training dataset, specify the task and the samples to be estimated, then can obtain the aleatoric uncertainty and epistemic uncertainty. Based the uncertainty information, developers can understand the model and the dataset better.
In classification task, for example, the model is lenet model. The MNIST dateset is used in the example. Data processing is consistent with [Implementing an Image Classification Application](https://www.mindspore.cn/tutorial/training/en/master/quick_start/quick_start.html) in Tutorial. For evaluating the uncertainty of test examples, the use of the toolbox is as follows:
```
```python
from mindspore.nn.probability.toolbox.uncertainty_evaluation import UncertaintyEvaluation
from mindspore.train.serialization import load_checkpoint, load_param_into_net
@@ -448,18 +463,20 @@ for eval_data in ds_eval.create_dict_iterator():
Examples in [mindspore/tests/st/probability](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability) are as follows:
### Examples
Examples in [mindspore/tests/st/probability](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability) are as follows:
- [Bayesian LeNet](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability/bnn_layers/test_bnn_layer.py). How to construct and train a LeNet by bnn layers.
- [Transform whole DNN model to BNN](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability/transforms/test_transform_bnn_model.py): How to transform whole DNN model to BNN.
- [Transform DNN layer to BNN](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability/transforms/test_transform_bnn_layer.py): How to transform one certainty type of layer in DNN model to corresponding Bayesian layer.
- [Variational Auto-Encoder](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability/dpn/test_gpu_svi_vae.py): Variational Auto-Encoder (VAE) model trained with MNIST to generate sample images.
- [Conditional Variational Auto-Encoder](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability/dpn/test_gpu_svi_cvae.py): Conditional Variational Auto-Encoder (CVAE) model trained with MNIST to generate sample images.
- [VAE-GAN](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability/dpn/test_gpu_vae_gan.py): VAE-GAN model trained with MNIST to generate sample images.
- [Uncertainty Estimation](https://gitee.com/mindspore/mindspore/tree/master/tests/st/probability/toobox/test_uncertainty.py): Evaluate uncertainty of model and data..
- [Bayesian LeNet](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability/bnn_layers/test_bnn_layer.py). How to construct and train a LeNet by bnn layers.
- [Transform whole DNN model to BNN](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability/transforms/test_transform_bnn_model.py): How to transform whole DNN model to BNN.
- [Transform DNN layer to BNN](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability/transforms/test_transform_bnn_layer.py): How to transform one certainty type of layer in DNN model to corresponding Bayesian layer.
- [Variational Auto-Encoder](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability/dpn/test_gpu_svi_vae.py): Variational Auto-Encoder (VAE) model trained with MNIST to generate sample images.
- [Conditional Variational Auto-Encoder](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability/dpn/test_gpu_svi_cvae.py): Conditional Variational Auto-Encoder (CVAE) model trained with MNIST to generate sample images.
- [VAE-GAN](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability/dpn/test_gpu_vae_gan.py): VAE-GAN model trained with MNIST to generate sample images.
- [Uncertainty Estimation](https://gitee.com/mindspore/mindspore/blob/master/tests/st/probability/toolbox/test_uncertainty.py): Evaluate uncertainty of model and data.
## Community
### Community
As part of MindSpore, we are committed to creating an open and friendly environment.
- [Gitee](https://gitee.com/mindspore/mindspore/issues): Report bugs or make feature requests.
DeepText is a convolutional neural network architecture for text detection in non-specific scenarios. The DeepText system is based on the elegant framwork of Faster R-CNN. This idea was proposed in the paper "DeepText: A new approach for text proposal generation and text detection in natural images.", published in 2017.
DeepText is a convolutional neural network architecture for text detection in non-specific scenarios. The DeepText system is based on the elegant framework of Faster R-CNN. This idea was proposed in the paper "DeepText: A new approach for text proposal generation and text detection in natural images.", published in 2017.
[Paper](https://arxiv.org/pdf/1605.07314v1.pdf) Zhuoyao Zhong, Lianwen Jin, Shuangping Huang, South China University of Technology (SCUT), Published in ICASSP 2017.
@@ -74,7 +74,7 @@ Here we used 4 datasets for training, and 1 datasets for Evaluation.
├─anchor_genrator.py # anchor generator
├─bbox_assign_sample.py # proposal layer for stage 1
├─bbox_assign_sample_stage2.py # proposal layer for stage 2
├─deeptext_vgg16.py # main network defination
├─deeptext_vgg16.py # main network definition
├─proposal_generator.py # proposal generator
├─rcnn.py # rcnn
├─roi_align.py # roi_align cell wrapper
@@ -83,7 +83,7 @@ Here we used 4 datasets for training, and 1 datasets for Evaluation.
├─config.py # training configuration
├─dataset.py # data proprocessing
├─lr_schedule.py # learning rate scheduler
├─network_define.py # network defination
├─network_define.py # network definition
└─utils.py # some functions which is commonly used