- class paddle.nn.initializer. Orthogonal ( gain=1.0, name=None )
The orthogonal initializer. The initialized tensor is (semi) orthogonal.
It’s only applied to Tensor whose dimension is greater than or equal to 2.
For the Tensor whose dimension is greater than 2, the 0 dimension is seen as
rows, and the >=1 dimension are flattened as
Which can be describe as:
rows = shape cols = shape·shape···shape[N] if rows < cols: The rows are orthogonal vectors elif rows > cols: The columns are orthogonal vectors else rows = cols: Both rows and columns are orthogonal vectors
gain (float, optional) – The multiplication coefficient for initialized tensor. Default: 1.0.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to Name.
A parameter initialized by orthogonal initialized.
import paddle weight_attr = paddle.ParamAttr(initializer=paddle.nn.initializer.Orthogonal()) linear = paddle.nn.Linear(10, 15, weight_attr=weight_attr) # linear.weight: X * X' = I linear = paddle.nn.Linear(15, 10, weight_attr=weight_attr) # linear.weight: X' * X = I