fast_ln

paddle.incubate.nn.functional. fast_ln ( x: Tensor, scale: Tensor, bias: Tensor, epsilon: float = 1e-05 ) → tuple[Tensor, Tensor, Tensor] [source]

Apply Fast LayerNorm kernel.

Parameters

• x (Tensor) – the input Tensor..

• scale (Tensor) – the weight Tensor to affine output.

• bias (Tensor) – the bias Tensor to affine output.

• epsilon (float) – a small float number to avoid divide 0.

Returns

the Tensor after performing layernorm. mean: the mean of input tensor invvar: the invert variance(scaling factor) of y

Return type

y