MSRAInitializer

class paddle.fluid.initializer.MSRAInitializer(uniform=True, fan_in=None, seed=0)[source]

Implements the MSRA initializer a.k.a. Kaiming Initializer

This class implements the weight initialization from the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Uniform distribution, the range is [-x, x], where

\[x = \sqrt{\frac{6.0}{fan\_in}}\]

In case of Normal distribution, the mean is 0 and the standard deviation is

\[\sqrt{\frac{2.0}{fan\_in}}\]
Parameters
  • uniform (bool) – whether to use uniform or normal distribution

  • fan_in (float32|None) – fan_in for MSRAInitializer. If None, it is inferred from the variable. default is None.

  • seed (int32) – random seed

Note

It is recommended to set fan_in to None for most cases.

Examples

import paddle.fluid as fluid
x = fluid.data(name="data", shape=[8, 32, 32], dtype="float32")
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.MSRA(uniform=False))