XavierNormal

class paddle.nn.initializer. XavierNormal ( fan_in=None, fan_out=None, name=None ) [source]

This class implements the Xavier weight initializer from the paper Understanding the difficulty of training deep feedforward neural networks by Xavier Glorot and Yoshua Bengio, using a normal distribution.

The mean is 0 and the standard deviation is

\[\begin{split}\sqrt{\\frac{2.0}{fan\_in + fan\_out}}\end{split}\]
Parameters
  • fan_in (float, optional) – fan_in for Xavier initialization, It is inferred from the tensor. The default value is None.

  • fan_out (float, optional) – fan_out for Xavier initialization, it is inferred from the tensor. The default value is None.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to Name.

Returns

A parameter initialized by Xavier weight, using a normal distribution.

Examples

import paddle

data = paddle.ones(shape=[3, 1, 2], dtype='float32')
weight_attr = paddle.framework.ParamAttr(
    name="linear_weight",
    initializer=paddle.nn.initializer.XavierNormal())
bias_attr = paddle.framework.ParamAttr(
    name="linear_bias",
    initializer=paddle.nn.initializer.XavierNormal())
linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr)
# inear.weight:  [[ 0.06910077 -0.18103665]
#                 [-0.02546741 -1.0402188 ]]
# linear.bias:  [-0.5012929   0.12418364]

res = linear(data)
# res:  [[[-0.4576595 -1.0970719]]
#        [[-0.4576595 -1.0970719]]
#        [[-0.4576595 -1.0970719]]]