XavierUniform

class paddle.nn.initializer. XavierUniform ( fan_in=None, fan_out=None, name=None ) [source]

This class implements the Xavier weight initializer from the paper Understanding the difficulty of training deep feedforward neural networks by Xavier Glorot and Yoshua Bengio.

This initializer is designed to keep the scale of the gradients approximately same in all the layers. In case of Uniform distribution, the range is \([-x,x]\), where

\[x = \sqrt{\frac{6.0}{fan\_in + fan\_out}}.\]
Parameters
  • fan_in (float, optional) – fan_in for Xavier initialization, which is inferred from the Tensor. The default value is None.

  • fan_out (float, optional) – fan_out for Xavier initialization, which is inferred from the Tensor. The default value is None.

  • name (str, optional) – For details, please refer to Name. Generally, no setting is required. Default: None.

Returns

A parameter initialized by Xavier weight, using a uniform distribution.

Examples

import paddle

data = paddle.ones(shape=[3, 1, 2], dtype='float32')
weight_attr = paddle.framework.ParamAttr(
    name="linear_weight",
    initializer=paddle.nn.initializer.XavierUniform())
bias_attr = paddle.framework.ParamAttr(
    name="linear_bias",
    initializer=paddle.nn.initializer.XavierUniform())
linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr)
# linear.weight:  [[-0.04229349 -1.1248565 ]
#                  [-0.10789523 -0.5938053 ]]
# linear.bias:  [ 1.1983747  -0.40201235]

res = linear(data)
# res:  [[[ 1.0481861 -2.1206741]]
#        [[ 1.0481861 -2.1206741]]
#        [[ 1.0481861 -2.1206741]]]