# dropout¶

paddle.nn.functional. dropout ( x, p=0.5, axis=None, training=True, mode='upscale_in_train', name=None ) [source]

Dropout is a regularization technique for reducing overfitting by preventing neuron co-adaption during training. The dropout operator randomly sets the outputs of some units to zero, while upscale others according to the given dropout probability.

Parameters
• x (Tensor) – The input tensor. The data type is float32 or float64.

• p (float|int) – Probability of setting units to zero. Default 0.5.

• axis (int|list|tuple) – The axis along which the dropout is performed. Default None.

• training (bool) – A flag indicating whether it is in train phrase or not. Default True.

• mode (str) –

[‘upscale_in_train’(default) | ‘downscale_in_infer’].

1. upscale_in_train(default), upscale the output at training time

• train: out = input * mask / ( 1.0 - dropout_prob )

• inference: out = input

2. downscale_in_infer, downscale the output at inference

• train: out = input * mask

• inference: out = input * (1.0 - dropout_prob)

• name (str, optional) – Name for the operation (optional, default is None). For more information, please refer to Name.

Returns

A Tensor representing the dropout, has same shape and data type as x .

Examples

We use `p=0.5` in the following description for simplicity.

1. When `axis=None` , this is commonly used dropout, which dropout each element of x randomly.

```Let's see a simple case when x is a 2d tensor with shape 2*3:
[[1 2 3]
[4 5 6]]
we generate mask with the same shape as x, which is 2*3. The value of mask is
sampled from a Bernoulli distribution randomly. For example, we may get such mask:
[[0 1 0]
[1 0 1]]
So the output is obtained from elementwise multiply of x and mask:
[[0 2 0]
[4 0 6]]
Using default setting, i.e. ``mode='upscale_in_train'`` ,
if in training phase, the final upscale output is:
[[0 4 0 ]
[8 0 12]]
if in test phase, the output is the same as input:
[[1 2 3]
[4 5 6]]
we can also set ``mode='downscale_in_infer'`` , then
if in training phase, the final output is:
[[0 2 0]
[4 0 6]]
if in test phase, the scale output is:
[[0.5 1.  1.5]
[2.  2.5 3. ]]
```
1. When `axis!=None` , this is useful for dropping whole channels from an image or sequence.

```Let's see the simple case when x is a 2d tensor with shape 2*3 again:
[[1 2 3]
[4 5 6]]
(1) If ``axis=0`` , this means the dropout is only performed in axis `0` .
we generate mask with the shape 2*1. Only in axis `0` the value is randomly selected.
For example, we may get such mask:
[
]
The output is obtained from elementwise multiply of x and mask. Doing that the mask will be
[[1 1 1]
[0 0 0]]
and the result after elementwise multiply is:
[[1 2 3]
[0 0 0]]
then we can do upscale or downscale according to the setting of other arguments.
(2) If ``axis=1`` , this means the dropout is only performed in axis `1` .
we generate mask with the shape 1*3. Only in axis `1` the value is randomly selected.
For example, we may get such mask:
[[1 0 1]]
Doing elementwise multiply the mask will be broadcast from 1*3 to 2*3:
[[1 0 1]
[1 0 1]]
and the result after elementwise multiply is:
[[1 0 3]
[4 0 6]]
(3) What about ``axis=[0, 1]`` ? This means the dropout is performed in all axes of x,
which is the same case as default setting ``axis=None`` .
(4) You may note that logically `axis=None` means the dropout is performed in none axis of x,
We generate mask with the shape 1*1. Whole input is randomly selected or dropped.
For example, we may get such mask:
[]
Doing elementwise multiply the mask will be broadcast from 1*1 to 2*3:
[[0 0 0]
[0 0 0]]
and the result after elementwise multiply is:
[[0 0 0]
[0 0 0]]
Actually this is not what we want because all elements may set to zero~
```

When x is a 4d tensor with shape NCHW, we can set `axis=[0,1]` and the dropout will be performed in channel N and C, H and W is tied, i.e. paddle.nn.dropout(x, p, axis=[0,1]) . Please refer to `paddle.nn.functional.dropout2d` for more details. Similarly, when x is a 5d tensor with shape NCDHW, we can set `axis=[0,1]` to perform dropout3d. Please refer to `paddle.nn.functional.dropout3d` for more details.

```import paddle
import numpy as np

x = np.array([[1,2,3], [4,5,6]]).astype('float32')