class paddle.nn. ClipGradByNorm ( clip_norm ) [source]

Limit the l2 norm of multi-dimensional Tensor $$X$$ to clip_norm .

• If the l2 norm of $$X$$ is greater than clip_norm , $$X$$ will be compressed by a ratio.

• If the l2 norm of $$X$$ is less than or equal to clip_norm , nothing will be done.

The multidimensional Tensor $$X$$ is not passed from this class, but the gradients of all parameters set in optimizer. If need_clip of specific param is False in its ParamAttr, then the gradients of this param will not be clipped.

Gradient clip will takes effect after being set in optimizer , see the document optimizer (for example: api_paddle_optimizer_SGD).

The clipping formula is:

\begin{split}Out = \\left \{ \\begin{aligned} & X & & if (norm(X) \\leq clip\_norm) \\\\ & \\frac{clip\_norm*X}{norm(X)} & & if (norm(X) > clip\_norm) \\\\ \\end{aligned} \\right.\end{split}

where $$norm(X)$$ represents the L2 norm of $$X$$.

$\begin{split}norm(X) = ( \\sum_{i=1}^{n}|x\_i|^2)^{ \\frac{1}{2}}\end{split}$

Note

need_clip of ClipGradByNorm HAS BEEN DEPRECATED since 2.0. Please use need_clip in ParamAttr to speficiy the clip scope.

Parameters

clip_norm (float) – The maximum norm value.

Examples

import paddle

x = paddle.uniform([10, 10], min=-1.0, max=1.0, dtype='float32')