hardswish

paddle.nn.functional. hardswish ( x, name=None ) [source]

hardswish activation

hardswish is proposed in MobileNetV3, and performs better in computational stability and efficiency compared to swish function. For more details please refer to: https://arxiv.org/pdf/1905.02244.pdf

\[\begin{split}hardswish(x)= \\left\\{ \\begin{aligned} &0, & & \\text{if } x \\leq -3 \\\\ &x, & & \\text{if } x \\geq 3 \\\\ &\\frac{x(x+3)}{6}, & & \\text{otherwise} \\end{aligned} \\right.\end{split}\]
Parameters
  • x (Tensor) – The input Tensor with data type float32, float64.

  • name (str, optional) – Name for the operation (optional, default is None). For more information, please refer to Name.

Returns

A Tensor with the same data type and shape as x .

Examples

import paddle
import paddle.nn.functional as F

x = paddle.to_tensor([-4., 5., 1.])
out = F.hardswish(x) # [0., 5., 0.666667]