gelu

paddle.nn.functional. gelu ( x: Tensor, approximate: Literal['tanh', 'none'] | bool = False, name: str | None = None ) → Tensor [source]

gelu activation.

The activation function of Gelu is calculated element by element. More information refers to :ref: Gaussian Error Linear Units.

The approximate parameter must be True, False, “tanh”, or “none”.

If approximate is True or “tanh”:

\[\begin{split}gelu(x) = 0.5 * x * (1 + tanh(\\sqrt{\\frac{2}{\\pi}} * (x + 0.044715x^{3})))\end{split}\]

else:

\[\begin{split}gelu(x) = 0.5 * x * (1 + erf(\\frac{x}{\\sqrt{2}}))\end{split}\]

Parameters

• x (Tensor) – The input Tensor with data type float32, float64. alias: input.

• approximate (str|bool, optional) – Whether to enable approximation. Default is False.

• name (str|None, optional) – For details, please refer to api_guide_Name. Generally, no setting is required. Default: None.

Returns

A Tensor with the same data type and shape as x .

Examples

>>> import paddle
>>> import paddle.nn.functional as F

>>> x = paddle.to_tensor([[-1, 0.5], [1, 1.5]])
>>> out1 = F.gelu(x)
>>> print(out1)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15865529,  0.34573123],
 [ 0.84134471,  1.39978933]])
>>> out2 = F.gelu(x, True)
>>> print(out2)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15880796,  0.34571400],
 [ 0.84119201,  1.39957154]])
>>> out3 = F.gelu(x, "none")
>>> print(out3)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15865529,  0.34573123],
 [ 0.84134471,  1.39978933]])
>>> out4 = F.gelu(x, "tanh")
>>> print(out4)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15880796,  0.34571400],
 [ 0.84119201,  1.39957154]])