SimpleRNNCell¶

class paddle.nn. SimpleRNNCell ( input_size, hidden_size, activation='tanh', weight_ih_attr=None, weight_hh_attr=None, bias_ih_attr=None, bias_hh_attr=None, name=None ) [source]

Elman RNN (SimpleRNN) cell. Given the inputs and previous states, it computes the outputs and updates states.

The formula used is as follows:

$$\begin{align}\begin{aligned}h_{t} & = act(W_{ih}x_{t} + b_{ih} + W_{hh}h_{t-1} + b_{hh})\\y_{t} & = h_{t}\end{aligned}\end{align}$$

where $act$ is for activation.

Please refer to Finding Structure in Time for more details.

Parameters

• input_size (int) – The input size.

• hidden_size (int) – The hidden size.

• activation (str, optional) – The activation in the SimpleRNN cell. It can be tanh or relu. Defaults to tanh.

• weight_ih_attr (ParamAttr, optional) – The parameter attribute for $weight_ih$. Default: None.

• weight_hh_attr (ParamAttr, optional) – The parameter attribute for $weight_hh$. Default: None.

• bias_ih_attr (ParamAttr, optional) – The parameter attribute for the $bias_ih$. Default: None.

• bias_hh_attr (ParamAttr, optional) – The parameter attribute for the $bias_hh$. Default: None.

• name (str, optional) – Name for the operation (optional, default is None). For more information, please refer to Name.

Variables:

• weight_ih (Parameter): shape (hidden_size, input_size), input to hidden weight, corresponding to $W_{ih}$ in the formula.

• weight_hh (Parameter): shape (hidden_size, hidden_size), hidden to hidden weight, corresponding to $W_{hh}$ in the formula.

• bias_ih (Parameter): shape (hidden_size, ), input to hidden bias, corresponding to $b_{ih}$ in the formula.

• bias_hh (Parameter): shape (hidden_size, ), hidden to hidden bias, corresponding to $b_{hh}$ in the formula.

Inputs:

• inputs (Tensor): shape [batch_size, input_size], the input, corresponding to $x_{t}$ in the formula.

• states (Tensor, optional): shape [batch_size, hidden_size], the previous hidden state, corresponding to $h_{t-1}$ in the formula. When states is None, zero state is used. Defaults to None.

Returns

shape [batch_size, hidden_size], the output, corresponding to $h_{t}$ in the formula. - states (Tensor): shape [batch_size, hidden_size], the new hidden state, corresponding to $h_{t}$ in the formula.

Return type

• outputs (Tensor)

Notes

All the weights and bias are initialized with Uniform(-std, std) by default. Where std = $\frac{1}{\sqrt{hidden\_size}}$. For more information about parameter initialization, please refer to ParamAttr.

Examples

>>> import paddle

>>> x = paddle.randn((4, 16))
>>> prev_h = paddle.randn((4, 32))

>>> cell = paddle.nn.SimpleRNNCell(16, 32)
>>> y, h = cell(x, prev_h)
>>> print(y.shape)
[4, 32]

forward ( inputs, states=None )

forward¶

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

property state_shape

Abstract method (property). Used to initialize states. A (possiblely nested structure of) shape[s], where a shape is a list/tuple of integers (-1 for batch size would be automatically inserted into a shape if shape is not started with it). Not necessary to be implemented if states are not initialized by get_initial_states or the shape argument is provided when using get_initial_states.

extra_repr ( )

extra_repr¶

Extra representation of this layer, you can have custom implementation of your own layer.