LSTMCell¶

class paddle.fluid.layers. LSTMCell ( hidden_size, param_attr=None, bias_attr=None, gate_activation=None, activation=None, forget_bias=1.0, dtype='float32', name='LSTMCell' ) [source]

api_attr

Static Graph

Long-Short Term Memory cell. It is a wrapper for fluid.contrib.layers.rnn_impl.BasicLSTMUnit to make it adapt to RNNCell.

The formula used is as follow:

$$\begin{align}\begin{aligned}i_{t} & = act_g(W_{x_{i}}x_{t} + W_{h_{i}}h_{t-1} + b_{i})\\\begin{split}f_{t} & = act_g(W_{x_{f}}x_{t} + W_{h_{f}}h_{t-1} + b_{f} + forget\\_bias)\end{split}\\c_{t} & = f_{t}c_{t-1} + i_{t} act_c (W_{x_{c}}x_{t} + W_{h_{c}}h_{t-1} + b_{c})\\o_{t} & = act_g(W_{x_{o}}x_{t} + W_{h_{o}}h_{t-1} + b_{o})\\h_{t} & = o_{t} act_c (c_{t})\end{aligned}\end{align}$$

For more details, please refer to RECURRENT NEURAL NETWORK REGULARIZATION

Examples

import paddle.fluid.layers as layers
cell = layers.LSTMCell(hidden_size=256)

call ( inputs, states )

call¶

Perform calculations of LSTM.

Parameters

• inputs (Variable) – A tensor with shape [batch_size, input_size], corresponding to $x_t$ in the formula. The data type should be float32 or float64.

• states (Variable) – A list of containing two tensors, each shaped [batch_size, hidden_size], corresponding to $h_{t-1}, c_{t-1}$ in the formula. The data type should be float32 or float64.

Returns

A tuple( (outputs, new_states) ), where outputs is

a tensor with shape [batch_size, hidden_size], corresponding to $h_{t}$ in the formula; new_states is a list containing two tenser variables shaped [batch_size, hidden_size], corresponding to $h_{t}, c_{t}$ in the formula. The data type of these tensors all is same as that of states.

Return type

tuple

property state_shape

[[hidden_size], [hidden_size]] (-1 for batch size would be automatically inserted into shape). These two shapes correspond to $h_{t-1}$ and $c_{t-1}$ separately.

Type

The state_shape of LSTMCell is a list with two shapes

get_initial_states ( batch_ref, shape=None, dtype='float32', init_value=0, batch_dim_idx=0 )

get_initial_states¶

Generate initialized states according to provided shape, data type and value.

Parameters

• batch_ref – A (possibly nested structure of) tensor variable[s]. The first dimension of the tensor will be used as batch size to initialize states.

• shape – A (possibly nested structure of) shape[s], where a shape is represented as a list/tuple of integer). -1(for batch size) will beautomatically inserted if shape is not started with it. If None, property state_shape will be used. The default value is None.

• dtype – A (possibly nested structure of) data type[s]. The structure must be same as that of shape, except when all tensors’ in states has the same data type, a single data type can be used. If property cell.state_shape is not available, float32 will be used as the data type. The default value is float32.

• init_value – A float value used to initialize states.

• batch_dim_idx – An integer indicating which dimension of the tensor in inputs represents batch size. The default value is 0.

Returns

tensor variable[s] packed in the same structure provided

by shape, representing the initialized states.

Return type

Variable

property state_dtype

Abstract method (property). Used to initialize states. A (possibly nested structure of) data types[s]. The structure must be same as that of shape, except when all tensors’ in states has the same data type, a single data type can be used. Not necessary to be implemented if states are not initialized by get_initial_states or the dtype argument is provided when using get_initial_states.