|
|
|
@@ -238,17 +238,16 @@ class SoftmaxCrossEntropyWithLogits(_Loss): |
|
|
|
Measures the distribution error between the probabilities of the input (computed with softmax function) and the |
|
|
|
target where the classes are mutually exclusive (only one class is positive) using cross entropy loss. |
|
|
|
|
|
|
|
Typical input into this function is unnormalized scores and target of each class. |
|
|
|
Scores Tensor :math:`x` is of shape :math:`(N, C)` and target Tensor :math:`t` is a |
|
|
|
Tensor of shape :math:`(N, C)` which contains one-hot labels of length :math:`C`. |
|
|
|
Typical input into this function is unnormalized scores denoted as x whose shape is (N, C), |
|
|
|
and the corresponding targets. |
|
|
|
|
|
|
|
For each instance :math:`N_i`, the loss is given as: |
|
|
|
For each instance :math:`x_i`, i ranges from 0 to N-1, the loss is given as: |
|
|
|
|
|
|
|
.. math:: |
|
|
|
\ell(x_i, t_i) = - \log\left(\frac{\exp(x_{t_i})}{\sum_j \exp(x_j)}\right) |
|
|
|
= -x_{t_i} + \log\left(\sum_j \exp(x_j)\right) |
|
|
|
\ell(x_i, c) = - \log\left(\frac{\exp(x_i[c])}{\sum_j \exp(x_i[j])}\right) |
|
|
|
= -x_i[c] + \log\left(\sum_j \exp(x_i[j])\right) |
|
|
|
|
|
|
|
where :math:`x_i` is a 1D score Tensor, :math:`t_i` is a scalar. |
|
|
|
where :math:`x_i` is a 1D score Tensor, :math:`c` is the index of 1 in one-hot. |
|
|
|
|
|
|
|
Note: |
|
|
|
While the target classes are mutually exclusive, i.e., only one class is positive in the target, the predicted |
|
|
|
|