sequence_softmax

paddle.fluid.layers.sequence_softmax(input, use_cudnn=False, name=None)[source]

Note:

The input type of the OP must be LoDTensor. For Tensor, use: softmax

A LoD-tensor can be regarded as several sequences, and this op apply softmax algo on each sequence. The shape of input Tensor can be \([N, 1]\) or \([N]\), where \(N\) is the sum of the length of all sequences. Recommended usage: \([N]\).

For i-th sequence in a mini-batch:

\[Out(X[lod[i]:lod[i+1]], :) = \frac{\exp(X[lod[i]:lod[i+1], :])}{\sum(\exp(X[lod[i]:lod[i+1], :]))}\]

For example, for a LoD-Tensor with 6 sequences ([3, 2, 4, 1, 2, 3] - sequence length list in order), the lod in the runtime is [[0, 3, 5, 9, 10, 12, 15]], then softmax will be computed among \(X[0:3,:],X[3:5,:],X[5:9,:],X[9:10,:],X[10:12,:],X[12:15,:]\), and \(N\) turns out to be 15.

*Case 1:

    Given:
        input.data = [0.7, 1, 0.6,
                      1.5, 1.1,
                      1.2, 0.2, 0.6, 1.9,
                      3.1,
                      2.5, 0.8,
                      0.1, 2.4, 1.3]
        input.lod = [[0, 3, 5, 9, 10, 12, 15]]
    then:
         output.data = [0.30724832, 0.41474187, 0.2780098,
                        0.59868765, 0.40131235,
                        0.2544242, 0.09359743, 0.13963096, 0.5123474,
                        1.,
                        0.84553474, 0.15446526,
                        0.06995796, 0.69777346, 0.23226859]
         output.lod = [[0, 3, 5, 9, 10, 12, 15]]
Parameters
  • input (Variable) – A LoDTensor with shape of \([N, 1]\) or \([N]\), Recommended usage: \([N]\). Supported data types: float32, float64.

  • use_cudnn (bool, optional) – Use cudnn kernel or not. Effective only when the cudnn version of the paddle library is installed and GPU is used for training or reasoning. Default: False.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to Name

Returns

A LoD-Tensor which has the same shape and data type with input.

Return type

Variable

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[7, 1],
                 dtype='float32', lod_level=1)
x_sequence_softmax_1 = fluid.layers.sequence_softmax(input=x)

y = fluid.data(name='y', shape=[7],
    dtype='float32', lod_level=1)
x_sequence_softmax_2 = fluid.layers.sequence_softmax(input=y)