spectral_norm

api_attr

declarative programming (static graph)

paddle.fluid.layers.spectral_norm(weight, dim=0, power_iters=1, eps=1e-12, name=None)[source]

Spectral Normalization Layer

This operation calculates the spectral normalization value of weight parameters of fc, conv1d, conv2d, conv3d layers which should be 2-D, 3-D, 4-D, 5-D Parameters. Output tensor will be in same shape with input tensor. Calculations are showed as follows.

Step 1: Generate vector U in shape of [H], and V in shape of [W]. While H is the dim th dimension of the input weights, and W is the product result of remaining dimensions.

Step 2: power_iters should be a positive integer, do following calculations with U and V for power_iters rounds. Calculations as follows:

\[ \begin{align}\begin{aligned}\mathbf{v} := \frac{\mathbf{W}^{T} \mathbf{u}}{\|\mathbf{W}^{T} \mathbf{u}\|_2}\\\mathbf{u} := \frac{\mathbf{W}^{T} \mathbf{v}}{\|\mathbf{W}^{T} \mathbf{v}\|_2}\end{aligned}\end{align} \]

Step 3: Calculate \(\sigma(\mathbf{W})\) and normalize weight values.

\[ \begin{align}\begin{aligned}\sigma(\mathbf{W}) = \mathbf{u}^{T} \mathbf{W} \mathbf{v}\\\mathbf{W} = \frac{\mathbf{W}}{\sigma(\mathbf{W})}\end{aligned}\end{align} \]

Refer to Spectral Normalization .

Parameters

  • weight (Variable) – The input weight tensor of spectral_norm operator, This can be a 2-D, 3-D, 4-D, 5-D tensor which is the weights of fc, conv1d, conv2d, conv3d layer. The data type is float32 or float64

  • dim (int) – The index of dimension which should be permuted to the first before reshaping Input(Weight) to matrix, it should be set as 0 if Input(Weight) is the weight of fc layer, and should be set as 1 if Input(Weight) is the weight of conv layer, default 0

  • power_iters (int) – number of power iterations to calculate spectral norm, default 1

  • eps (float) – epsilon for numerical stability in calculating norms, it will be added to the denominator to aviod divide zero. Default 1e-12

  • name (str, optional) – For detailed information, please refer to Name. Usually name is no need to set and None by default.

Returns

A tensor variable of weight parameters after spectral normalization.

The data type and shape is same as input tensor.

Return type

Variable

Examples

import paddle.fluid as fluid

weight = fluid.data(name='weight', shape=[2, 8, 32, 32], dtype='float32')
x = fluid.layers.spectral_norm(weight=weight, dim=1, power_iters=2)