lstm

paddle.fluid.layers. lstm ( input, init_h, init_c, max_len, hidden_size, num_layers, dropout_prob=0.0, is_bidirec=False, is_test=False, name=None, default_initializer=None, seed=- 1 ) [source]
Warning: API “paddle.fluid.layers.rnn.lstm” is deprecated since 2.0.0, and will be removed in future versions. Please use “paddle.nn.LSTM” instead.

reason: This API may occur CUDNN errors.

api_attr

Static Graph

Note:

This OP only supports running on GPU devices.

This OP implements LSTM operation - Hochreiter, S., & Schmidhuber, J. (1997) .

The implementation of this OP does not include diagonal/peephole connections. Please refer to Gers, F. A., & Schmidhuber, J. (2000) . If you need peephole connections, please use api_fluid_layers_dynamic_lstm .

This OP computes each timestep as follows:

\[i_t = \sigma(W_{ix}x_{t} + W_{ih}h_{t-1} + b_{x_i} + b_{h_i})\]
\[f_t = \sigma(W_{fx}x_{t} + W_{fh}h_{t-1} + b_{x_f} + b_{h_f})\]
\[o_t = \sigma(W_{ox}x_{t} + W_{oh}h_{t-1} + b_{x_o} + b_{h_o})\]
\[\widetilde{c_t} = tanh(W_{cx}x_t + W_{ch}h_{t-1} + b{x_c} + b_{h_c})\]
\[c_t = f_t \odot c_{t-1} + i_t \odot \widetilde{c_t}\]
\[h_t = o_t \odot tanh(c_t)\]

The symbolic meanings in the formula are as follows:

  • \(x_{t}\) represents the input at timestep \(t\)

  • \(h_{t}\) represents the hidden state at timestep \(t\)

  • \(h_{t-1}, c_{t-1}\) represent the hidden state and cell state at timestep \(t-1\) , respectively

  • \(\widetilde{c_t}\) represents the candidate cell state

  • \(i_t\) , \(f_t\) and \(o_t\) represent input gate, forget gate, output gate, respectively

  • \(W\) represents weight (e.g., \(W_{ix}\) is the weight of a linear transformation of input \(x_{t}\) when calculating input gate \(i_t\) )

  • \(b\) represents bias (e.g., \(b_{i}\) is the bias of input gate)

  • \(\sigma\) represents nonlinear activation function for gate, default sigmoid

  • \(\odot\) represents the Hadamard product of a matrix, i.e. multiplying the elements of the same position for two matrices with the same dimension to get another matrix with the same dimension

Parameters:

input ( Variable ): LSTM input tensor, 3-D Tensor of shape \([batch\_size, seq\_len, input\_dim]\) . Data type is float32 or float64 init_h( Variable ): The initial hidden state of the LSTM, 3-D Tensor of shape \([num\_layers, batch\_size, hidden\_size]\) .

System Message: ERROR/3 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 45)

Unexpected indentation.

If is_bidirec = True, shape should be \([num\_layers*2, batch\_size, hidden\_size]\) . Data type is float32 or float64.

System Message: WARNING/2 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 46)

Block quote ends without a blank line; unexpected unindent.

max_len (int): This parameter has no effect and will be discarded. init_c( Variable ): The initial cell state of the LSTM, 3-D Tensor of shape \([num\_layers, batch\_size, hidden\_size]\) .

System Message: ERROR/3 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 48)

Unexpected indentation.

If is_bidirec = True, shape should be \([num\_layers*2, batch\_size, hidden\_size]\) . Data type is float32 or float64.

System Message: WARNING/2 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 49)

Block quote ends without a blank line; unexpected unindent.

hidden_size (int): hidden size of the LSTM. num_layers (int): total layers number of the LSTM. dropout_prob(float, optional): dropout prob, dropout ONLY work between rnn layers, NOT between time steps

System Message: ERROR/3 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 52)

Unexpected indentation.

There is NO dropout work on rnn output of the last RNN layers. Default: 0.0.

System Message: WARNING/2 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 54)

Block quote ends without a blank line; unexpected unindent.

is_bidirec (bool, optional): If it is bidirectional. Default: False. is_test (bool, optional): If it is in test phrase. Default: False. name (str, optional): A name for this layer. If set None, the layer

System Message: ERROR/3 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 57)

Unexpected indentation.

will be named automatically. Default: None.

System Message: WARNING/2 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 58)

Block quote ends without a blank line; unexpected unindent.

default_initializer(Initializer, optional): Where use initializer to initialize the Weight

If set None, default initializer will be used. Default: None.

System Message: WARNING/2 (/usr/local/lib/python3.8/site-packages/paddle/fluid/layers/rnn.py:docstring of paddle.fluid.layers.rnn.lstm, line 60)

Definition list ends without a blank line; unexpected unindent.

seed(int, optional): Seed for dropout in LSTM, If it’s -1, dropout will use random seed. Default: 1.

Returns:

tuple ( Variable , Variable , Variable ) :

Three tensors, rnn_out, last_h, last_c:

  • rnn_out is result of LSTM hidden, shape is \([seq\_len, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([seq\_len, batch\_size, hidden\_size*2]\)

  • last_h is the hidden state of the last step of LSTM shape is \([num\_layers, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([num\_layers*2, batch\_size, hidden\_size]\)

  • last_c(Tensor): the cell state of the last step of LSTM shape is \([num\_layers, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([num\_layers*2, batch\_size, hidden\_size]\)

Examples:
import paddle.fluid as fluid
import paddle.fluid.layers as layers

emb_dim = 256
vocab_size = 10000
data = fluid.data(name='x', shape=[None, 100], dtype='int64')
emb = fluid.embedding(input=data, size=[vocab_size, emb_dim], is_sparse=True)
batch_size = 20
dropout_prob = 0.2
input_size = 100
hidden_size = 150
num_layers = 1
init_h = layers.fill_constant( [num_layers, batch_size, hidden_size], 'float32', 0.0 )
init_c = layers.fill_constant( [num_layers, batch_size, hidden_size], 'float32', 0.0 )
rnn_out, last_h, last_c = layers.lstm( emb, init_h, init_c, \
        max_len, hidden_size, num_layers, \
        dropout_prob=dropout_prob)
rnn_out.shape  # (-1, 100, 150)
last_h.shape  # (1, 20, 150)
last_c.shape  # (1, 20, 150)