LSTMCell

Note: This API is only avaliable in [Static Graph] mode

class paddle.fluid.layers.LSTMCell(hidden_size, param_attr=None, bias_attr=None, gate_activation=None, activation=None, forget_bias=1.0, dtype='float32', name='LSTMCell')[source]

Long-Short Term Memory cell. It is a wrapper for fluid.contrib.layers.rnn_impl.BasicLSTMUnit to make it adapt to RNNCell.

The formula used is as follow:

\[ \begin{align}\begin{aligned}i_{t} & = act_g(W_{x_{i}}x_{t} + W_{h_{i}}h_{t-1} + b_{i})\\f_{t} & = act_g(W_{x_{f}}x_{t} + W_{h_{f}}h_{t-1} + b_{f} + forget\_bias)\\c_{t} & = f_{t}c_{t-1} + i_{t} act_c (W_{x_{c}}x_{t} + W_{h_{c}}h_{t-1} + b_{c})\\o_{t} & = act_g(W_{x_{o}}x_{t} + W_{h_{o}}h_{t-1} + b_{o})\\h_{t} & = o_{t} act_c (c_{t})\end{aligned}\end{align} \]

For more details, please refer to RECURRENT NEURAL NETWORK REGULARIZATION

Examples

import paddle.fluid.layers as layers
cell = layers.LSTMCell(hidden_size=256)
call(inputs, states)

Perform calculations of LSTM.

Parameters
  • inputs (Variable) – A tensor with shape [batch_size, input_size], corresponding to \(x_t\) in the formula. The data type should be float32.

  • states (Variable) – A list of containing two tensers, each shaped [batch_size, hidden_size], corresponding to \(h_{t-1}, c_{t-1}\) in the formula. The data type should be float32.

Returns

A tuple( (outputs, new_states) ), where outputs is a tensor with shape [batch_size, hidden_size], corresponding to \(h_{t}\) in the formula; new_states is a list containing two tenser variables shaped [batch_size, hidden_size], corresponding to \(h_{t}, c_{t}\) in the formula. The data type of these tensors all is same as that of states.

Return type

tuple

state_shape

[[hidden_size], [hidden_size]] (-1 for batch size would be automatically inserted into shape). These two shapes correspond to \(h_{t-1}\) and \(c_{t-1}\) separately.

Type

The state_shape of LSTMCell is a list with two shapes

get_initial_states(batch_ref, shape=None, dtype=None, init_value=0)

Generate initialized states according to provided shape, data type and value.

Parameters
  • batch_ref – A (possibly nested structure of) tensor variable[s]. The first dimension of the tensor will be used as batch size to initialize states.

  • shape – A (possiblely nested structure of) shape[s], where a shape is represented as a list/tuple of integer). -1(for batch size) will beautomatically inserted if shape is not started with it. If None, property state_shape will be used. The default value is None.

  • dtype – A (possiblely nested structure of) data type[s]. The structure must be same as that of shape, except when all tensors’ in states has the same data type, a single data type can be used. If None and property cell.state_shape is not available, float32 will be used as the data type. The default value is None.

  • init_value – A float value used to initialize states.

Returns

tensor variable[s] packed in the same structure provided by shape, representing the initialized states.

Return type

Variable

state_dtype

Used to initialize states. A (possiblely nested structure of) data types[s]. The structure must be same as that of shape, except when all tensors’ in states has the same data type, a signle data type can be used. Not necessary to be implemented if states are not initialized by get_initial_states or the dtype argument is provided when using get_initial_states.