rms_norm
- paddle.nn.functional. rms_norm ( input: Tensor, normalized_shape: int | Sequence[int], weight: Tensor | None = None, eps: float = 1e-05, name: str | None = None ) tuple[Tensor, Tensor] [source]
-
Applies Layer Normalization over the last dimension of the input tensor using CUDA implementation. :param input: Input tensor of shape [rows, cols] or higher dimensions (flattened to 2D). :type input: Tensor :param normalized_shape: Input shape from an expected input of
size \([*, normalized_shape[0], normalized_shape[1], ..., normalized_shape[-1]]\). If it is a single integer, this module will normalize over the last dimension which is expected to be of that specific size.
- Parameters
-
weight (Tensor, optional) – The weight tensor of rms_norm. Default: None.
eps (float, optional) – The small value added to the variance to prevent division by zero. Default: 1e-05.
name (str, optional) – Name of the operator.
- Returns
-
Normalized tensor of same shape as input. invvar (Tensor): Tensor of shape [rows], the inverse standard deviation of each row.
- Return type
-
out (Tensor)