gelu

paddle.nn.functional. gelu ( x, approximate=False, name=None ) [source]

gelu activation.

if approximate is True

\[\begin{split}gelu(x) = 0.5 * x * (1 + tanh(\\sqrt{\\frac{2}{\\pi}} * (x + 0.044715x^{3})))\end{split}\]

else

\[\begin{split}gelu(x) = 0.5 * x * (1 + erf(\\frac{x}{\\sqrt{2}}))\end{split}\]
Parameters
  • x (Tensor) – The input Tensor with data type float32, float64.

  • approximate (bool, optional) – Wether to enable approximation. Default is False.

  • name (str, optional) – Name for the operation (optional, default is None). For more information, please refer to Name.

Returns

A Tensor with the same data type and shape as x .

Examples

import paddle
import paddle.nn.functional as F

x = paddle.to_tensor([[-1, 0.5], [1, 1.5]])
out1 = F.gelu(x)
# [[-0.15865529,  0.34573123],
#  [ 0.84134471,  1.39978933]]
out2 = F.gelu(x, True)
# [[-0.15880799,  0.34571400],
#  [ 0.84119201,  1.39957154]]