SpectralNorm

class paddle.nn. SpectralNorm ( weight_shape: ShapeLike, dim: int = 0, power_iters: int = 1, eps: float = 1e-12, dtype: DTypeLike = 'float32' ) [source]

This interface is used to construct a callable object of the SpectralNorm class. For more details, refer to code examples. It implements the function of the Spectral Normalization Layer. This layer calculates the spectral normalization value of weight parameters of fc, conv1d, conv2d, conv3d layers which should be 2-D, 3-D, 4-D, 5-D Parameters. Calculations are showed as follows.

Step 1: Generate vector U in shape of [H], and V in shape of [W]. While H is the dim th dimension of the input weights, and W is the product result of remaining dimensions.

Step 2: power_iters should be a positive integer, do following calculations with U and V for power_iters rounds.

\[ \begin{align}\begin{aligned}\mathbf{v} := \frac{\mathbf{W}^{T} \mathbf{u}}{\|\mathbf{W}^{T} \mathbf{u}\|_2}\\\mathbf{u} := \frac{\mathbf{W}^{T} \mathbf{v}}{\|\mathbf{W}^{T} \mathbf{v}\|_2}\end{aligned}\end{align} \]

Step 3: Calculate \(\sigma(\mathbf{W})\) and normalize weight values.

\[ \begin{align}\begin{aligned}\sigma(\mathbf{W}) = \mathbf{u}^{T} \mathbf{W} \mathbf{v}\\\mathbf{W} = \frac{\mathbf{W}}{\sigma(\mathbf{W})}\end{aligned}\end{align} \]

Refer to Spectral Normalization .

Parameters

  • weight_shape (list or tuple) – The shape of weight parameter.

  • dim (int, optional) – The index of dimension which should be permuted to the first before reshaping Input(Weight) to matrix, it should be set as 0 if Input(Weight) is the weight of fc layer, and should be set as 1 if Input(Weight) is the weight of conv layer. Default: 0.

  • power_iters (int, optional) – The number of power iterations to calculate spectral norm. Default: 1.

  • eps (float, optional) – The epsilon for numerical stability in calculating norms. Default: 1e-12.

  • dtype (str, optional) – Data type, it can be “float32” or “float64”. Default: “float32”.

Returns

None

Examples

>>> import paddle
>>> x = paddle.rand((2,8,32,32))
>>> spectral_norm = paddle.nn.SpectralNorm(x.shape, dim=1, power_iters=2)
>>> spectral_norm_out = spectral_norm(x)
>>> print(spectral_norm_out.shape)
[2, 8, 32, 32]
forward ( x: Tensor ) → Tensor

forward

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

  • *inputs (tuple) – unpacked tuple arguments

  • **kwargs (dict) – unpacked dict arguments