spectral_norm¶
- api_attr
declarative programming (static graph)
-
paddle.fluid.layers.
spectral_norm
(weight, dim=0, power_iters=1, eps=1e-12, name=None)[source] Spectral Normalization Layer
This operation calculates the spectral normalization value of weight parameters of fc, conv1d, conv2d, conv3d layers which should be 2-D, 3-D, 4-D, 5-D Parameters. Output tensor will be in same shape with input tensor. Calculations are showed as follows.
Step 1: Generate vector U in shape of [H], and V in shape of [W]. While H is the
dim
th dimension of the input weights, and W is the product result of remaining dimensions.Step 2:
power_iters
should be a positive integer, do following calculations with U and V forpower_iters
rounds. Calculations as follows:\[ \begin{align}\begin{aligned}\mathbf{v} := \frac{\mathbf{W}^{T} \mathbf{u}}{\|\mathbf{W}^{T} \mathbf{u}\|_2}\\\mathbf{u} := \frac{\mathbf{W}^{T} \mathbf{v}}{\|\mathbf{W}^{T} \mathbf{v}\|_2}\end{aligned}\end{align} \]Step 3: Calculate \(\sigma(\mathbf{W})\) and normalize weight values.
\[ \begin{align}\begin{aligned}\sigma(\mathbf{W}) = \mathbf{u}^{T} \mathbf{W} \mathbf{v}\\\mathbf{W} = \frac{\mathbf{W}}{\sigma(\mathbf{W})}\end{aligned}\end{align} \]Refer to Spectral Normalization .
- Parameters
weight (Variable) – The input weight tensor of spectral_norm operator, This can be a 2-D, 3-D, 4-D, 5-D tensor which is the weights of fc, conv1d, conv2d, conv3d layer. The data type is float32 or float64
dim (int) – The index of dimension which should be permuted to the first before reshaping Input(Weight) to matrix, it should be set as 0 if Input(Weight) is the weight of fc layer, and should be set as 1 if Input(Weight) is the weight of conv layer, default 0
power_iters (int) – number of power iterations to calculate spectral norm, default 1
eps (float) – epsilon for numerical stability in calculating norms, it will be added to the denominator to aviod divide zero. Default 1e-12
name (str, optional) – For detailed information, please refer to Name. Usually name is no need to set and None by default.
- Returns
- A tensor variable of weight parameters after spectral normalization.
The data type and shape is same as input tensor.
- Return type
Variable
Examples
import paddle.fluid as fluid weight = fluid.data(name='weight', shape=[2, 8, 32, 32], dtype='float32') x = fluid.layers.spectral_norm(weight=weight, dim=1, power_iters=2)