gelu¶

paddle.nn.functional. gelu ( x, approximate=False, name=None ) [source]

gelu activation.

The activation function of Gelu is calculated element by element. More information refers to :ref: Gaussian Error Linear Units.

if approximate is True

\[gelu(x) = 0.5 * x * (1 + tanh(\sqrt{\frac{2}{\pi}} * (x + 0.044715x^{3})))\]

else

\[gelu(x) = 0.5 * x * (1 + erf(\frac{x}{\sqrt{2}}))\]

Parameters

• x (Tensor) – The input Tensor with data type float32, float64.

• approximate (bool, optional) – Whether to enable approximation. Default is False.

• name (str, optional) – For details, please refer to Name. Generally, no setting is required. Default: None.

Returns

A Tensor with the same data type and shape as x .

Examples

import paddle
import paddle.nn.functional as F

x = paddle.to_tensor([[-1, 0.5], [1, 1.5]])
out1 = F.gelu(x)
# [[-0.15865529,  0.34573123],
#  [ 0.84134471,  1.39978933]]
out2 = F.gelu(x, True)
# [[-0.15880799,  0.34571400],
#  [ 0.84119201,  1.39957154]]