spectral_norm¶

paddle.static.nn. spectral_norm ( weight, dim=0, power_iters=1, eps=1e-12, name=None ) [source]

Api_attr

Static Graph

Spectral Normalization Layer

This operation calculates the spectral normalization value of weight parameters of fc, conv1d, conv2d, conv3d layers which should be 2-D, 3-D, 4-D, 5-D Parameters. Output tensor will be in same shape with input tensor. Calculations are showed as follows.

Step 1: Generate vector U in shape of [H], and V in shape of [W]. While H is the dim th dimension of the input weights, and W is the product result of remaining dimensions.

Step 2: power_iters should be a positive integer, do following calculations with U and V for power_iters rounds. Calculations as follows:

\[ \begin{align}\begin{aligned}\begin{split}\mathbf{v} := \\frac{\mathbf{W}^{T} \mathbf{u}}{\|\mathbf{W}^{T} \mathbf{u}\|_2}\end{split}\\\begin{split}\mathbf{u} := \\frac{\mathbf{W}^{T} \mathbf{v}}{\|\mathbf{W}^{T} \mathbf{v}\|_2}\end{split}\end{aligned}\end{align} \]

Step 3: Calculate \(\sigma(\mathbf{W})\) and normalize weight values.

\[ \begin{align}\begin{aligned}\sigma(\mathbf{W}) = \mathbf{u}^{T} \mathbf{W} \mathbf{v}\\\begin{split}\mathbf{W} = \\frac{\mathbf{W}}{\sigma(\mathbf{W})}\end{split}\end{aligned}\end{align} \]

Refer to Spectral Normalization .

Parameters

• weight (Tensor) – The input weight tensor of spectral_norm operator, This can be a 2-D, 3-D, 4-D, 5-D tensor which is the weights of fc, conv1d, conv2d, conv3d layer. The data type is float32 or float64

• dim (int) – The index of dimension which should be permuted to the first before reshaping Input(Weight) to matrix, it should be set as 0 if Input(Weight) is the weight of fc layer, and should be set as 1 if Input(Weight) is the weight of conv layer, default 0

• power_iters (int) – number of power iterations to calculate spectral norm, default 1

• eps (float) – epsilon for numerical stability in calculating norms, it will be added to the denominator to aviod divide zero. Default 1e-12

• name (str, optional) – For detailed information, please refer to Name. Usually name is no need to set and None by default.

Returns

A tensor of weight parameters after spectral normalization.

The data type and shape is same as input tensor.

Return type

Tensor

Examples

import paddle

paddle.enable_static()
weight = paddle.static.data(name='weight', shape=[2, 8, 32, 32], dtype='float32')
x = paddle.static.nn.spectral_norm(weight=weight, dim=1, power_iters=2)
print(x.shape) # [2, 8, 32, 32]