fast_rms_norm

paddle.incubate.nn.functional. fast_rms_norm ( x: Tensor, scale: Tensor, epsilon: float = 1e-05 ) → tuple[Tensor, Tensor] [source]

Apply Fast LayerNorm kernel.

Parameters

  • x (Tensor) – the input Tensor..

  • scale (Tensor) – the weight Tensor to affine output.

  • epsilon (float) – a small float number to avoid divide 0.

Returns

the Tensor after performing layernorm. invvar: the invert variance(scaling factor) of y

Return type

y