nn

adaptive_pool2d

paddle.fluid.layers.adaptive_pool2d(input, pool_size, pool_type='max', require_index=False, name=None)[source]

This operation calculates the output based on the input, pool_size, pool_type parameters. Input(X) and output(Out) are in NCHW format, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Parameters(pool_size) should contain two elements which represent height and width, respectively. Also the H and W dimensions of output(Out) is same as Parameter(pool_size). The output tensor shape will be [N, C, pool_size[0], pool_size[1]]

For average adaptive pool2d:

\[ \begin{align}\begin{aligned}hstart &= floor(i * H_{in} / H_{out})\\hend &= ceil((i + 1) * H_{in} / H_{out})\\wstart &= floor(j * W_{in} / W_{out})\\wend &= ceil((j + 1) * W_{in} / W_{out})\\Output(i ,j) &= \frac{sum(Input[hstart:hend, wstart:wend])}{(hend - hstart) * (wend - wstart)}\end{aligned}\end{align} \]
Parameters
  • input (Variable) – The input tensor of pooling operator, which is a 4-D tensor with shape [N, C, H, W]. The format of input tensor is NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. The data type is float32 or float64.

  • pool_size (int|list|tuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain two integers, (pool_size_Height, pool_size_Width).

  • pool_type – (string), pooling type, can be “max” for max-pooling and “avg” for average-pooling

  • require_index (bool) – If true, the index of max pooling point will be returned along with outputs. It cannot be set in average pooling type. Default False.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

The output tensor of adaptive pooling result. The data type is same

as input tensor.

Return type

Variable

Raises
  • ValueError – ‘pool_type’ is not ‘max’ nor ‘avg’.

  • ValueError – invalid setting ‘require_index’ true when ‘pool_type’ is ‘avg’.

  • ValueError – ‘pool_size’ should be a list or tuple with length as 2.

Examples

# average adaptive pool2d
# suppose input data in shape of [N, C, H, W], `pool_size` is [m, n],
# output shape is [N, C, m, n], adaptive pool divide H and W dimentions
# of input data into m * n grids averagely and performs poolings in each
# grid to get output.
# adaptive average pool performs calculations as follow:
#
#     for i in range(m):
#         for j in range(n):
#             hstart = floor(i * H / m)
#             hend = ceil((i + 1) * H / m)
#             wstart = floor(i * W / n)
#             wend = ceil((i + 1) * W / n)
#             output[:, :, i, j] = avg(input[:, :, hstart: hend, wstart: wend])
#
import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32')
pool_out = fluid.layers.adaptive_pool2d(
                  input=data,
                  pool_size=[3, 3],
                  pool_type='avg')

# max adaptive pool2d
# suppose input data in shape of [N, C, H, W], `pool_size` is [m, n],
# output shape is [N, C, m, n], adaptive pool divide H and W dimentions
# of input data into m * n grids averagely and performs poolings in each
# grid to get output.
# adaptive average pool performs calculations as follow:
#
#     for i in range(m):
#         for j in range(n):
#             hstart = floor(i * H / m)
#             hend = ceil((i + 1) * H / m)
#             wstart = floor(i * W / n)
#             wend = ceil((i + 1) * W / n)
#             output[:, :, i, j] = max(input[:, :, hstart: hend, wstart: wend])
#
import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32')
pool_out = fluid.layers.adaptive_pool2d(
                  input=data,
                  pool_size=[3, 3],
                  pool_type='max')

adaptive_pool3d

paddle.fluid.layers.adaptive_pool3d(input, pool_size, pool_type='max', require_index=False, name=None)[source]

This operation calculates the output based on the input, pool_size, pool_type parameters. Input(X) and output(Out) are in NCDHW format, where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. Parameters(pool_size) should contain three elements which represent height and width, respectively. Also the D, H and W dimensions of output(Out) is same as Parameter(pool_size). The output tensor shape will be [N, C, pool_size[0], pool_size[1], pool_size[2]]

For average adaptive pool3d:

\[ \begin{align}\begin{aligned}dstart &= floor(i * D_{in} / D_{out})\\dend &= ceil((i + 1) * D_{in} / D_{out})\\hstart &= floor(j * H_{in} / H_{out})\\hend &= ceil((j + 1) * H_{in} / H_{out})\\wstart &= floor(k * W_{in} / W_{out})\\wend &= ceil((k + 1) * W_{in} / W_{out})\\Output(i ,j, k) &= \frac{sum(Input[dstart:dend, hstart:hend, wstart:wend])}{(dend - dstart) * (hend - hstart) * (wend - wstart)}\end{aligned}\end{align} \]
Parameters
  • input (Variable) – The input tensor of pooling operator, which is a 5-D tensor with shape [N, C, D, H, W]. The format of input tensor is NCDHW, where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. The data type is float32 or float64.

  • pool_size (int|list|tuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain three integers, (Depth, Height, Width).

  • pool_type – (string) Pooling type, can be “max” for max-pooling and “avg” for average-pooling

  • require_index (bool) – If true, the index of max pooling point will be returned along with outputs. It cannot be set in average pooling type. Default False.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

The output tensor of adaptive pooling result. The data type is same as input tensor.

Return type

Variable

Raises
  • ValueError – ‘pool_type’ is not ‘max’ nor ‘avg’.

  • ValueError – invalid setting ‘require_index’ true when ‘pool_type’ is ‘avg’.

  • ValueError – ‘pool_size’ should be a list or tuple with length as 2.

Examples

# average adaptive pool3d
# suppose input data in shape of [N, C, D, H, W], `pool_size` is [l, m, n],
# output shape is [N, C, l, m, n], adaptive pool divide D, H and W dimentions
# of input data into l * m * n grids averagely and performs poolings in each
# grid to get output.
# adaptive average pool performs calculations as follow:
#
#     for i in range(l):
#         for j in range(m):
#             for k in range(n):
#                 dstart = floor(i * D / l)
#                 dend = ceil((i + 1) * D / l)
#                 hstart = floor(j * H / m)
#                 hend = ceil((j + 1) * H / m)
#                 wstart = floor(k * W / n)
#                 wend = ceil((k + 1) * W / n)
#                 output[:, :, i, j, k] =
#                     avg(input[:, :, dstart:dend, hstart: hend, wstart: wend])
#

import paddle.fluid as fluid

data = fluid.data(
    name='data', shape=[None, 3, 32, 32, 32], dtype='float32')
pool_out = fluid.layers.adaptive_pool3d(
                  input=data,
                  pool_size=[3, 3, 3],
                  pool_type='avg')

# max adaptive pool3d
# suppose input data in shape of [N, C, D, H, W], `pool_size` is [l, m, n],
# output shape is [N, C, l, m, n], adaptive pool divide D, H and W dimentions
# of input data into l * m * n grids averagely and performs poolings in each
# grid to get output.
# adaptive average pool performs calculations as follow:
#
#     for i in range(l):
#         for j in range(m):
#             for k in range(n):
#                 dstart = floor(i * D / l)
#                 dend = ceil((i + 1) * D / l)
#                 hstart = floor(j * H / m)
#                 hend = ceil((j + 1) * H / m)
#                 wstart = floor(k * W / n)
#                 wend = ceil((k + 1) * W / n)
#                 output[:, :, i, j, k] =
#                     avg(input[:, :, dstart:dend, hstart: hend, wstart: wend])
#

import paddle.fluid as fluid

data = fluid.data(
    name='data', shape=[None, 3, 32, 32, 32], dtype='float32')
pool_out = fluid.layers.adaptive_pool3d(
                  input=data,
                  pool_size=[3, 3, 3],
                  pool_type='max')

add_position_encoding

paddle.fluid.layers.add_position_encoding(input, alpha, beta, name=None)[source]

This operator performs weighted sum of input feature at each position (position in the sequence) and the corresponding position encoding.

For more details of position encoding, please refer to Attention Is All You Need .

The formula is as follows:

\[\begin{split}PE(pos, 2i) &= \sin{(pos / 10000^{2i / P})} \\ PE(pos, 2i + 1) &= \cos{(pos / 10000^{2i / P})} \\ Out(:, pos, i) &= \alpha * input(:, pos, i) + \beta * PE(pos, i)\end{split}\]
Where:
  • \(PE(pos, 2i)\) : the value at even index 2i for encoding of position pos.

  • \(PE(pos, 2i + 1)\) : the value at odd index 2i+1 for encoding of position pos

Parameters
  • input (Variable) – A Tensor or LoDTensor (lod level is 1). If it is a Tensor, the shape should be [N, M, P], where N stands for batch size, M for sequence length, P for the size of feature dimension. If it is a LoDTensor, the shape should be [N, P], where N stands for the total sequence lengths in this mini-batch, P for the size of feature. The data type should be float32 or float64.

  • alpha (float) – Indicate the weight coefficient for input when performing weighted sum.

  • beta (float) – Indicate the weight coefficient for position encoding when performing weighted sum.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

A Tensor or LoDTensor. It has the same shape, data type and lod as input.

Return type

Variable

Examples

import paddle.fluid as fluid

tensor = fluid.data(
    name='tensor',
    shape=[None, 64, 512],
    dtype='float32')
position_tensor = fluid.layers.add_position_encoding(
    input=tensor, alpha=1.0, beta=1.0)

affine_channel

paddle.fluid.layers.affine_channel(x, scale=None, bias=None, data_layout='NCHW', name=None, act=None)[source]

Applies a separate affine transformation to each channel of the input. Useful for replacing spatial batch norm with its equivalent fixed transformation. The input also can be 2D tensor and applies a affine transformation in second dimension.

Parameters
  • x (Variable) – Feature map input can be a 4D tensor with order NCHW or NHWC. It also can be a 2D tensor and the affine transformation is applied in the second dimension.The data type is float32 or float64.

  • scale (Variable) – 1D input of shape (C), the c-th element is the scale factor of the affine transformation for the c-th channel of the input.The data type is float32 or float64.

  • bias (Variable) – 1D input of shape (C), the c-th element is the bias of the affine transformation for the c-th channel of the input. The data type is float32 or float64.

  • data_layout (str, default NCHW) – NCHW or NHWC. If input is 2D tensor, you can ignore data_layout.

  • name (str, default None) – The name of this layer. For more information, please refer to api_guide_Name .

  • act (str, default None) – Activation to be applied to the output of this layer.

Returns

A tensor which has the same shape, data layout and data type with x.

Return type

Variable

Examples

import numpy as np
import paddle.fluid as fluid

use_gpu = False
place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
exe = fluid.Executor(place)

data = fluid.data(name='data', shape=[None, 1, 2, 2], dtype='float32')
input_scale = fluid.layers.create_parameter(shape=[1], dtype="float32",
                        default_initializer=fluid.initializer.Constant(2.0))
input_bias = fluid.layers.create_parameter(shape=[1],dtype="float32",
                        default_initializer=fluid.initializer.Constant(0.5))
out = fluid.layers.affine_channel(data,scale=input_scale,
                        bias=input_bias)

exe.run(fluid.default_startup_program())
test_program = fluid.default_main_program().clone(for_test=True)

[out_array] = exe.run(test_program,
                      fetch_list=out,
                      feed={'data': np.ones([1,1,2,2]).astype('float32')})
# out_array is [[[[2.5, 2.5],
#                [2.5, 2.5]]]] with shape: [1, 1, 2, 2]

affine_grid

paddle.fluid.layers.affine_grid(theta, out_shape, name=None)[source]

It generates a grid of (x,y) coordinates using the parameters of the affine transformation that correspond to a set of points where the input feature map should be sampled to produce the transformed output feature map.

Parameters
  • theta (Variable) – The data type can be float32 or float64.

  • out_shape (Variable | list | tuple) – The shape of target output with format [batch_size, channel, height, width]. out_shape can be a Tensor or a list or tuple. The data type must be int32.

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.

Returns

A Tensor with shape [batch_size, H, W, 2] while ‘H’ and ‘W’ are the height and width of feature map in affine transformation. The data type is the same as theta.

Return type

Variable

Raises

ValueError – If the type of arguments is not supported.

Examples

import paddle.fluid as fluid
import numpy as np
place = fluid.CPUPlace()
theta = fluid.data(name="x", shape=[None, 2, 3], dtype="float32")
out_shape = fluid.data(name="y", shape=[4], dtype="int32")
grid_0 = fluid.layers.affine_grid(theta, out_shape)
grid_1 = fluid.layers.affine_grid(theta, [5, 3, 28, 28])
batch_size=2
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
output= exe.run(feed={"x": np.random.rand(batch_size,2,3).astype("float32"),
                      "y": np.array([5, 3, 28, 28]).astype("int32")},
                      fetch_list=[grid_0.name, grid_1.name])
print(output[0])
print(output[1])

autoincreased_step_counter

paddle.fluid.layers.autoincreased_step_counter(counter_name=None, begin=1, step=1)[source]

Create an auto-increase variable. which will be automatically increased by 1 in every iteration. By default, the first return of this counter is 1, and the step size is 1.

Parameters
  • counter_name (str, optional) – The counter name. Default ‘@STEP_COUNTER@’.

  • begin (int, optional) – The first return value of this counter. Default 1.

  • step (int, optional) – The step size. Default 1.

Returns

The auto-increased Variable with data type int64.

Return type

Variable

Examples

import paddle.fluid as fluid
global_step = fluid.layers.autoincreased_step_counter(
    counter_name='@LR_DECAY_COUNTER@', begin=0, step=1)

batch_norm

paddle.fluid.layers.batch_norm(input, act=None, is_test=False, momentum=0.9, epsilon=1e-05, param_attr=None, bias_attr=None, data_layout='NCHW', in_place=False, name=None, moving_mean_name=None, moving_variance_name=None, do_model_average_for_mean_and_var=True, fuse_with_relu=False, use_global_stats=False)[source]

Batch Normalization Layer

Can be used as a normalizer function for convolution or fully_connected operations. The required data format for this layer is one of the following:

  1. NHWC [batch, in_height, in_width, in_channels]

  2. NCHW [batch, in_channels, in_height, in_width]

Refer to Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift for more details.

\(input\) is the input features over a mini-batch.

\[ \begin{align}\begin{aligned}\begin{split}\mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\ \ mini-batch\ mean \\ \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \ \mu_{\beta})^2 \qquad &//\ mini-batch\ variance \\ \hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\ \sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift\end{split}\\\begin{split}moving\_mean = moving\_mean * momentum + mini-batch\_mean * (1. - momentum) \\ moving\_var = moving\_var * momentum + mini-batch\_var * (1. - momentum)\end{split}\end{aligned}\end{align} \]

moving_mean is global mean and moving_var is global variance.

When use_global_stats = True, the \(\mu_{\beta}\) and \(\sigma_{\beta}^{2}\) are not the statistics of one mini-batch. They are global (or running) statistics. (It usually got from the pre-trained model.) The training and testing (or inference) have the same behavior:

\[\begin{split}\hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\ \sigma_{\beta}^{2} + \epsilon}} \\ y_i &\gets \gamma \hat{x_i} + \beta\end{split}\]

Note

if build_strategy.sync_batch_norm=True, the batch_norm in network will use sync_batch_norm automatically.

Parameters
  • input (variable) – The rank of input variable can be 2, 3, 4, 5. The data type is float16 or float32 or float64.

  • act (string, Default None) – Activation type, linear|relu|prelu|…

  • is_test (bool, Default False) – A flag indicating whether it is in test phrase or not.

  • momentum (float, Default 0.9) – The value used for the moving_mean and moving_var computation. The updated formula is: \(moving\_mean = moving\_mean * momentum + new\_mean * (1. - momentum)\) \(moving\_var = moving\_var * momentum + new\_var * (1. - momentum)\) Default is 0.9.

  • epsilon (float, Default 1e-05) – A value added to the denominator for numerical stability. Default is 1e-5.

  • param_attr (ParamAttr|None) – The parameter attribute for Parameter scale of batch_norm. If it is set to None or one attribute of ParamAttr, batch_norm will create ParamAttr as param_attr, the name of scale can be set in ParamAttr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.

  • bias_attr (ParamAttr|None) – The parameter attribute for the bias of batch_norm. If it is set to None or one attribute of ParamAttr, batch_norm will create ParamAttr as bias_attr, the name of bias can be set in ParamAttr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

  • data_layout (str, default NCHW) – the data_layout of input, is NCHW or NHWC.

  • in_place (bool, Default False) – Make the input and output of batch norm reuse memory.

  • name (str|None) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

  • moving_mean_name (str, Default None) – The name of moving_mean which store the global Mean. If it is set to None, batch_norm will save global mean with a random name, otherwise, batch_norm will save global mean with the string.

  • moving_variance_name (str, Default None) – The name of the moving_variance which store the global Variance. If it is set to None, batch_norm will save global variance with a random name, otherwise, batch_norm will save global variance with the string.

  • do_model_average_for_mean_and_var (bool, Default True) – Whether parameter mean and variance should do model average when model average is enabled.

  • fuse_with_relu (bool) – if True, this OP performs relu after batch norm.

  • use_global_stats (bool, Default False) – Whether to use global mean and variance. In inference or test mode, set use_global_stats to true or is_test to true, and the behavior is equivalent. In train mode, when setting use_global_stats True, the global mean and variance are also used during train period.

Returns

A Variable holding Tensor which is the result after applying batch normalization on the input, has same shape and data type with input.

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[3, 7, 3, 7], dtype='float32')
hidden1 = fluid.layers.fc(input=x, size=200, param_attr='fc1.w')
hidden2 = fluid.layers.batch_norm(input=hidden1)

beam_search_decode

paddle.fluid.layers.beam_search_decode(ids, scores, beam_size, end_id, name=None)[source]

This operator is used after beam search has completed. It constructs the full predicted sequences for each sample by walking back along the search paths stored in lod of ids . The result sequences are stored in a LoDTensor, which uses the following way to parse:

If lod = [[0, 3, 6], [0, 12, 24, 40, 54, 67, 82]]

The first level of lod stands for: There are 2 samples each having 3
(beam width) predicted sequence.

The second level of lod stands for: The lengths of the first sample's
3 predicted sequences are 12, 12, 16; The lengths of the second sample's
3 predicted sequences are 14, 13, 15.
Please see the following demo for a fully beam search usage example:

fluid/tests/book/test_machine_translation.py

Parameters
  • ids (Variable) – The LoDTensorArray variable containing the selected ids of all steps. Each LoDTensor in it has int64 data type and 2 level lod which can be used to get the search paths.

  • scores (Variable) – The LodTensorArray variable containing the accumulated scores corresponding to selected ids of all steps. It has the same size as ids . Each LoDTensor in it has the same shape and lod as the counterpart in ids , and has a float32 data type.

  • beam_size (int) – The beam width used in beam search.

  • end_id (int) – The id of end token.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

The tuple contains two LodTensor variables. The two LodTensor, containing the full sequences of ids and the correspongding accumulated scores, have the same shape flattened to 1D and have the same 2 level lod. The lod can be used to get how many predicted sequences each sample has and how many ids each predicted sequence has.

Return type

tuple

Examples

import paddle.fluid as fluid

# Suppose `ids` and `scores` are LodTensorArray variables reserving
# the selected ids and scores of all steps
ids = fluid.layers.create_array(dtype='int64')
scores = fluid.layers.create_array(dtype='float32')
finished_ids, finished_scores = fluid.layers.beam_search_decode(
    ids, scores, beam_size=5, end_id=0)

bilinear_tensor_product

paddle.fluid.layers.bilinear_tensor_product(x, y, size, act=None, name=None, param_attr=None, bias_attr=None)[source]

Bilinear Tensor Product Layer

This layer performs bilinear tensor product on two inputs. For example:

\[out_{i} = x * W_{i} * {y^\mathrm{T}}, i=0,1,...,size-1\]
In this formula:
  • \(x\): the first input contains M elements, shape is [batch_size, M].

  • \(y\): the second input contains N elements, shape is [batch_size, N].

  • \(W_{i}\): the i-th learned weight, shape is [M, N].

  • \(out_{i}\): the i-th element of out, shape is [batch_size, size].

  • \(y^\mathrm{T}\): the transpose of \(y_{2}\).

Parameters
  • x (Variable) – 2-D input tensor with shape [batch_size, M]. Data type is float32 or float64.

  • y (Variable) – 2-D input tensor with shape [batch_size, N]. Data type should be same as x.

  • size (int) – The dimension of this layer.

  • act (str|None) – Activation to be applied to the output of this layer. Default None.

  • name (str|None) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.

  • param_attr (ParamAttr|None) – To specify the weight parameter attribute. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • bias_attr (ParamAttr|None) – To specify the bias parameter attribute. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .

Returns

A 2-D Tensor of shape [batch_size, size]. Data type is the same as input x.

Return type

Variable

Examples

import paddle.fluid as fluid
layer1 = fluid.data("t1", shape=[-1, 5], dtype="float32")
layer2 = fluid.data("t2", shape=[-1, 4], dtype="float32")
tensor = fluid.layers.bilinear_tensor_product(x=layer1, y=layer2, size=1000)

bpr_loss

paddle.fluid.layers.bpr_loss(input, label, name=None)[source]

Bayesian Personalized Ranking Loss Operator

This operator belongs to pairwise ranking loss. Label is the desired item. The loss at a given point in one session is defined as:

\[Y[i] = 1/(N[i] - 1) * \sum_j{\log(\sigma(X[i, Label[i]]-X[i, j]))}\]

Learn more details by reading paper <session-based recommendations with recurrent neural networks>.

Parameters
  • input (Variable|list) – a 2-D tensor with shape [N x D], where N is the batch size and D is the number of positive classes and negative classes This input is not probability but logits.

  • label (Variable|list) – the ground truth which is a 2-D tensor. label is a tensor<int64> with shape [N x 1].

  • name (str|None) – A name for this layer(optional). If set None, the layer will be named automatically. Default: None.

Returns

A 2-D tensor with shape [N x 1], the bpr loss.

Examples

import paddle.fluid as fluid

neg_size = 10
label = fluid.data(
          name="label", shape=[3, 1], dtype="int64")
predict = fluid.data(
          name="predict", shape=[3, neg_size + 1], dtype="float32")
cost = fluid.layers.bpr_loss(input=predict, label=label)

brelu

paddle.fluid.layers.brelu(x, t_min=0.0, t_max=24.0, name=None)[source]

BRelu Activation Operator.

\(out = \min(\max(x, t_{min}), t_{max})\)

Parameters
  • x (Variable) – The input is a multi-dimensional Tensor. The data type is float32, float64

  • t_min (FLOAT|0.0) – The min marginal value of BRelu

  • t_max (FLOAT|24.0) – The max marginal value of BRelu

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.

Returns

The output is a multi-dimensional Tensor which has same dimension and data type as the X

Return type

Variable

Examples:

import paddle.fluid as fluid
import numpy as np

input_brelu = np.array([[-1,6],[1,15.6]])
with fluid.dygraph.guard():
    x = fluid.dygraph.to_variable(input_brelu)
    y = fluid.layers.brelu(x, t_min=1.0, t_max=10.0)
    print(y.numpy())
    #[[ 1.  6.]
    #[ 1. 10.]]

chunk_eval

paddle.fluid.layers.chunk_eval(input, label, chunk_scheme, num_chunk_types, excluded_chunk_types=None, seq_length=None)[source]

This operator computes the precision, recall and F1-score for chunk detection. It is often used in sequence tagging tasks, such as Named Entity Recognition(NER).

For some basics of chunking, please refer to Chunking with Support Vector Machines .

This operator supports IOB, IOE, IOBES and IO (also known as plain) tagging schemes. Here is a NER example for the usage of these tagging schemes:

====== ====== ======  =====  ==  ============   =====  ===== =====  ==  =========
       Li     Ming    works  at  Agricultural   Bank   of    China  in  Beijing.
====== ====== ======  =====  ==  ============   =====  ===== =====  ==  =========
IO     I-PER  I-PER   O      O   I-ORG          I-ORG  I-ORG I-ORG  O   I-LOC
IOB    B-PER  I-PER   O      O   B-ORG          I-ORG  I-ORG I-ORG  O   B-LOC
IOE    I-PER  E-PER   O      O   I-ORG          I-ORG  I-ORG E-ORG  O   E-LOC
IOBES  B-PER  E-PER   O      O   I-ORG          I-ORG  I-ORG E-ORG  O   S-LOC
====== ====== ======  =====  ==  ============   =====  ===== =====  ==  =========

There are three chunk types(named entity types) including PER(person), ORG(organization) and LOC(location), and we can see that the labels have the form <tag type>-<chunk type> .

Since the implementation of this operator actually uses label ids rather than label strings, to make it work, there should be a way to map label ids to tag types and chunk types. This operator uses the following way to do mapping:

tag_type = label % num_tag_type
chunk_type = label / num_tag_type

where num_tag_type is the num of tag types in the tagging scheme, num_chunk_type is the num of chunk types, and tag_type get its value from the following table.

Scheme Begin Inside End   Single
 plain   0     -      -     -
 IOB     0     1      -     -
 IOE     -     0      1     -
 IOBES   0     1      2     3

Accordingly, in the above NER example, if the tagging scheme is IOB and chunk types are ORG, PER and LOC, then the label ids would be as follows:

B-ORG  0
I-ORG  1
B-PER  2
I-PER  3
B-LOC  4
I-LOC  5
O      6

With which we can map each label id to the corresponding tag type and chunk type correctly.

Parameters
  • input (Variable) – A Tensor or LoDTensor, representing the predicted labels from the network. When it is a Tensor, its shape would be [N, M, 1], where N stands for batch size, M for sequence length; When it is a LoDTensor, its shape would be [N, 1] where N stands for the total sequence lengths in this mini-batch. The data type should be int64.

  • label (Variable) – A Tensor or LoDTensor representing the ground-truth labels. It shoud have the same shape, lod and data type as input .

  • chunk_scheme (str) – Indicate the tagging schemes used here. The value must be IOB, IOE, IOBES or plain.

  • num_chunk_types (int) – The number of chunk types.

  • excluded_chunk_types (list, optional) – Indicate the chunk types shouldn’t be taken into account. It should be a list of chunk type ids(integer). Default None.

  • seq_length (Variable, optional) – A 1D Tensor containing the length of each sequence when input and label are Tensor. It needn’t be provided if input and label are LoDTensor. Default None.

Returns

A tuple including precision, recall, F1-score, chunk number detected, chunk number in ground-truth, chunk number correctly detected. Each is a Tensor with shape [1]. The data type of precision, recall and F1-score all is float32, and the others’ data type all is int64.

Return type

tuple

Examples

import paddle.fluid as fluid

dict_size = 10000
label_dict_len = 7
sequence = fluid.data(
    name='id', shape=[-1, 1], lod_level=1, dtype='int64')
embedding = fluid.embedding(
    input=sequence, size=[dict_size, 512])
hidden = fluid.layers.fc(input=embedding, size=512)
label = fluid.layers.data(
    name='label', shape=[1], lod_level=1, dtype='int32')
crf = fluid.layers.linear_chain_crf(
    input=hidden, label=label, param_attr=fluid.ParamAttr(name="crfw"))
crf_decode = fluid.layers.crf_decoding(
    input=hidden, param_attr=fluid.ParamAttr(name="crfw"))
fluid.layers.chunk_eval(
    input=crf_decode,
    label=label,
    chunk_scheme="IOB",
    num_chunk_types=(label_dict_len - 1) / 2)

clip

paddle.fluid.layers.clip(x, min, max, name=None)[source]

Clip Operator.

The clip operator limits the value of given input within an interval [min, max], just as the following equation,

$$ Out = MIN(MAX(x, min), max) $$

Parameters
  • x (Variable) – Tensor, the input of clip op, data type should be float32 or float64

  • min (float) – float number, the minimum value to clip by

  • max (float) – float number, the maximum value to clip by

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Tensor, the clipped tensor, with the same shape and data type as input(x)

Return Type:

Variable

Examples

import paddle.fluid as fluid
input = fluid.data(
    name='data', shape=[1], dtype='float32')
reward = fluid.layers.clip(x=input, min=-1.0, max=1.0)

clip_by_norm

paddle.fluid.layers.clip_by_norm(x, max_norm, name=None)[source]

ClipByNorm Operator.

This operator limits the L2 norm of the input \(X\) within \(max\_norm\). If the L2 norm of \(X\) is less than or equal to \(max\_norm\), \(Out\) will be the same as \(X\). If the L2 norm of \(X\) is greater than \(max\_norm\), \(X\) will be linearly scaled to make the L2 norm of \(Out\) equal to \(max\_norm\), as shown in the following formula:

$$ Out = \frac{max\_norm * X}{norm(X)}, $$

where \(norm(X)\) represents the L2 norm of \(X\).

Parameters
  • x (Variable) – (Tensor) The input of clip_by_norm op and data type is float32.The number of dimensions must be between [1, 9]

  • max_norm (FLOAT) – (float) The maximum norm value

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

out(Variable): (Tensor) The output of clip_by_norm op with shape as input(X)The data type is float32

Return type

Variable

Examples

import paddle.fluid as fluid
input = fluid.data(
    name='data', shape=[None, 1], dtype='float32')
reward = fluid.layers.clip_by_norm(x=input, max_norm=1.0)

continuous_value_model

paddle.fluid.layers.continuous_value_model(input, cvm, use_cvm=True)[source]

continuous_value_model layers

Now, this OP is used in CTR project to remove or dispose show and click value in input.

input is an embedding vector including show and click value, whose shape is \([N, D]\) (N is batch size. D is 2 + embedding dim ). Show and click at first two dims of embedding vector D. If use_cvm is True, it will caculate \(log(show)\) and \(log(click)\) , and output shape is \([N, D]\) . If use_cvm is False, it will remove show and click from input , and output shape is \([N, D - 2]\) . cvm is show_click info, whose shape is \([N, 2]\) .

Parameters
  • input (Variable) – The input variable. A 2-D LoDTensor with shape \([N, D]\) , where N is the batch size, D is 2 + the embedding dim . lod level = 1 .

  • Tensor with type float32, float64. (A) –

  • cvm (Variable) – Show and click variable. A 2-D Tensor with shape \([N, 2]\) , where N is the batch size, 2 is show and click.

  • Tensor with type float32, float64.

  • use_cvm (bool) – Use show_click or not. if use, the output dim is the same as input. if not use, the output dim is input dim - 2 (remove show and click)

Returns

A 2-D LodTensor with shape \([N, M]\) . if use_cvm = True, M is equal to input dim D. if False, M is equal to D - 2. A Tensor with same type as input.

Return type

Variable

Examples

import paddle.fluid as fluid
input = fluid.data(name="input", shape=[64, 1], dtype="int64")
label = fluid.data(name="label", shape=[64, 1], dtype="int64")
embed = fluid.layers.embedding(
                  input=input,
                  size=[100, 11],
                  dtype='float32')
ones = fluid.layers.fill_constant_batch_size_like(input=label, shape=[-1, 1], dtype="int64", value=1)
show_clk = fluid.layers.cast(fluid.layers.concat([ones, label], axis=1), dtype='float32')
show_clk.stop_gradient = True
input_with_cvm = fluid.layers.continuous_value_model(embed, show_clk, True)

conv2d

paddle.fluid.layers.conv2d(input, num_filters, filter_size, stride=1, padding=0, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCHW')[source]

The convolution2D layer calculates the output based on the input, filter and strides, paddings, dilations, groups parameters. Input and Output are in NCHW or NHWC format, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Filter is in MCHW format, where M is the number of output image channels, C is the number of input image channels, H is the height of the filter, and W is the width of the filter. If the groups is greater than 1, C will equal the number of input image channels divided by the groups. Please refer to UFLDL’s convolution for more details. If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.

For each input \(X\), the equation is:

\[Out = \sigma (W \ast X + b)\]

Where:

  • \(X\): Input value, a tensor with NCHW or NHWC format.

  • \(W\): Filter value, a tensor with MCHW format.

  • \(\ast\): Convolution operation.

  • \(b\): Bias value, a 2-D tensor with shape [M, 1].

  • \(\sigma\): Activation function.

  • \(Out\): Output value, the shape of \(Out\) and \(X\) may be different.

Example

  • Input:

    Input shape: \((N, C_{in}, H_{in}, W_{in})\)

    Filter shape: \((C_{out}, C_{in}, H_f, W_f)\)

  • Output:

    Output shape: \((N, C_{out}, H_{out}, W_{out})\)

Where

\[\begin{split}H_{out}&= \frac{(H_{in} + 2 * paddings[0] - (dilations[0] * (H_f - 1) + 1))}{strides[0]} + 1 \\ W_{out}&= \frac{(W_{in} + 2 * paddings[1] - (dilations[1] * (W_f - 1) + 1))}{strides[1]} + 1\end{split}\]
Parameters
  • input (Variable) – The input is 4-D Tensor with shape [N, C, H, W], the data type of input is float16 or float32 or float64.

  • num_filters (int) – The number of filter. It is as same as the output image channel.

  • filter_size (int|tuple) – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_height, filter_size_width). Otherwise, filter_size_height = filter_size_width = filter_size.

  • stride (int|tuple) – The stride size. It means the stride in convolution. If stride is a tuple, it must contain two integers, (stride_height, stride_width). Otherwise, stride_height = stride_width = stride. Default: stride = 1.

  • padding (string|int|list|tuple) – The padding size. It means the number of zero-paddings on both sides for each dimention.If padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If padding size is a tuple or list, it could be in three forms: [pad_height, pad_width] or [pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCHW”, padding can be in the form [[0,0], [0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NHWC”, pool_padding can be in the form [[0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.

  • dilation (int|tuple) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain two integers, (dilation_height, dilation_width). Otherwise, dilation_height = dilation_width = dilation. Default: dilation = 1.

  • groups (int) – The groups number of the Conv2d Layer. According to grouped convolution in Alex Krizhevsky’s Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1.

  • param_attr (ParamAttr|None) – The parameter attribute for learnable parameters/weights of conv2d. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with \(Normal(0.0, std)\), and the \(std\) is \((\frac{2.0 }{filter\_elem\_num})^{0.5}\). Default: None.

  • bias_attr (ParamAttr|bool|None) – The parameter attribute for the bias of conv2d. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

  • use_cudnn (bool) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True

  • act (str) – Activation type, if it is set to None, activation is not appended. Default: None

  • name (str|None) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

  • data_format (str) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. The default is “NCHW”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width].

Returns

A Variable holding Tensor representing the conv2d, whose data type is the same with input. If act is None, the tensor variable storing the convolution result, and if act is not None, the tensor variable storing convolution and non-linearity activation result.

Examples

import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32')
conv2d = fluid.layers.conv2d(input=data, num_filters=2, filter_size=3, act="relu")

conv2d_transpose

paddle.fluid.layers.conv2d_transpose(input, num_filters, output_size=None, filter_size=None, padding=0, stride=1, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCHW')[source]

The convolution2D transpose layer calculates the output based on the input, filter, and dilations, strides, paddings. Input(Input) and output(Output) are in NCHW or NHWC format. Where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Parameters(dilations, strides, paddings) are two elements. These two elements represent height and width, respectively. The details of convolution transpose layer, please refer to the following explanation and references therein. If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.

For each input \(X\), the equation is:

\[Out = \sigma (W \ast X + b)\]

Where:

  • \(X\): Input value, a 4-D Tensor with NCHW or NHWC format.

  • \(W\): Filter value, a 4-D Tensor with MCHW format.

  • \(\ast\): Convolution operation.

  • \(b\): Bias value, a 2-D Tensor with shape [M, 1].

  • \(\sigma\): Activation function.

  • \(Out\): Output value, a 4-D Tensor with data format ‘NCHW’ or ‘NHWC’, the shape of \(Out\) and \(X\) may be different.

Example

  • Input:

    Input shape: \((N, C_{in}, H_{in}, W_{in})\)

    Filter shape: \((C_{in}, C_{out}, H_f, W_f)\)

  • Output:

    Output shape: \((N, C_{out}, H_{out}, W_{out})\)

Where

\[\begin{split}H^\prime_{out} &= (H_{in} - 1) * strides[0] - pad_height_top - pad_height_bottom + dilations[0] * (H_f - 1) + 1 \\ W^\prime_{out} &= (W_{in} - 1) * strides[1] - pad_width_left - pad_width_right + dilations[1] * (W_f - 1) + 1 \\ H_{out} &\in [ H^\prime_{out}, H^\prime_{out} + strides[0] ] \\ W_{out} &\in [ W^\prime_{out}, W^\prime_{out} + strides[1] ]\end{split}\]

Note

The conv2d_transpose can be seen as the backward of the conv2d. For conv2d, when stride > 1, conv2d maps multiple input shape to the same output shape, so for conv2d_transpose, when stride > 1, input shape maps multiple output shape. If output_size is None, \(H_{out} = H^\prime_{out}, W_{out} = W^\prime_{out}\); else, the \(H_{out}\) of the output size must between \(H^\prime_{out}\) and \(H^\prime_{out} + strides[0]\), and the \(W_{out}\) of the output size must between \(W^\prime_{out}\) and \(W^\prime_{out} + strides[1]\), conv2d_transpose can compute the kernel size automatically.

Parameters
  • input (Variable) – 4-D Tensor with [N, C, H, W] or [N, H, W, C] format, its data type is float32 or float64.

  • num_filters (int) – The number of the filter. It is as same as the output image channel.

  • output_size (int|tuple, optional) – The output image size. If output size is a tuple, it must contain two integers, (image_height, image_width). None if use filter_size, padding, and stride to calculate output_size. If output_size and filter_size are specified at the same time, They should follow the formula above. Default: None. output_size and filter_size should not be None at the same time.

  • filter_size (int|tuple, optional) – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_height, filter_size_width). Otherwise, filter_size_height = filter_size_width = filter_size. None if use output size to calculate filter_size. Default: None. filter_size and output_size should not be None at the same time.

  • stride (int|tuple, optional) – The stride size. It means the stride in transposed convolution. If stride is a tuple, it must contain two integers, (stride_height, stride_width). Otherwise, stride_height = stride_width = stride. Default: stride = 1.

  • padding (int|list|str|tuple, optional) –

    The padding size. The padding argument effectively adds

    dilation * (kernel - 1) amount of zero-padding on both sides of input. If padding is a string, either ‘VALID’ or ‘SAME’ supported, which is the padding algorithm. If padding is a tuple or list, it could be in three forms: [pad_height, pad_width] or

    [pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is ‘NCHW’, padding can be in the form [[0,0], [0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is ‘NHWC’, padding can be in the form [[0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.

  • dilation (int|tuple, optional) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain two integers, (dilation_height, dilation_width). Otherwise, dilation_height = dilation_width = dilation. Default: dilation = 1.

  • filter_size – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_height, filter_size_width). Otherwise, filter_size_height = filter_size_width = filter_size. None if use output size to calculate filter_size. Default: None.

  • groups (int, optional) – The groups number of the Conv2d transpose layer. Inspired by grouped convolution in Alex Krizhevsky’s Deep CNN paper, in which when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups = 1.

  • param_attr (ParamAttr, optional) – The parameter attribute for learnable parameters/weights of conv2d_transpose. If it is set to None or one attribute of ParamAttr, conv2d_transpose will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.

  • bias_attr (ParamAttr|bool, optional) – The parameter attribute for the bias of conv2d_transpose. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv2d_transpose will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

  • use_cudnn (bool, optional) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True.

  • act (str, optional) – Activation type, if it is set to None, activation is not appended. Default: None.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

  • data_format (str, optional) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width]. Default: ‘NCHW’.

Returns

A Variable holding Tensor representing the conv2d_transpose, whose data type is the same with input and shape is (num_batches, channels, out_h, out_w) or (num_batches, out_h, out_w, channels). If act is None, the tensor variable storing the transposed convolution result, and if act is not None, the tensor variable storing transposed convolution and non-linearity activation result.

Raises

ValueError – If the shapes of output, input, filter_size, stride, padding and groups mismatch.

Examples

import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32')
conv2d_transpose = fluid.layers.conv2d_transpose(input=data, num_filters=2, filter_size=3)

conv3d

paddle.fluid.layers.conv3d(input, num_filters, filter_size, stride=1, padding=0, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCDHW')[source]

The convolution3D layer calculates the output based on the input, filter and strides, paddings, dilations, groups parameters. Input(Input) and Output(Output) are in NCDHW or NDHWC format. Where N is batch size C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. Convlution3D is similar with Convlution2D but adds one dimension(depth). If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.

For each input \(X\), the equation is:

\[Out = \sigma (W \ast X + b)\]

In the above equation:

  • \(X\): Input value, a tensor with NCDHW or NDHWC format.

  • \(W\): Filter value, a tensor with MCDHW format.

  • \(\ast\): Convolution operation.

  • \(b\): Bias value, a 2-D tensor with shape [M, 1].

  • \(\sigma\): Activation function.

  • \(Out\): Output value, the shape of \(Out\) and \(X\) may be different.

Example

  • Input:

    Input shape: \((N, C_{in}, D_{in}, H_{in}, W_{in})\)

    Filter shape: \((C_{out}, C_{in}, D_f, H_f, W_f)\)

  • Output: Output shape: \((N, C_{out}, D_{out}, H_{out}, W_{out})\)

Where

\[\begin{split}D_{out}&= \frac{(D_{in} + 2 * paddings[0] - (dilations[0] * (D_f - 1) + 1))}{strides[0]} + 1 \\ H_{out}&= \frac{(H_{in} + 2 * paddings[1] - (dilations[1] * (H_f - 1) + 1))}{strides[1]} + 1 \\ W_{out}&= \frac{(W_{in} + 2 * paddings[2] - (dilations[2] * (W_f - 1) + 1))}{strides[2]} + 1\end{split}\]
Parameters
  • input (Variable) – The input is 5-D Tensor with shape [N, C, D, H, W], the data type of input is float16 or float32 or float64.

  • num_filters (int) – The number of filter. It is as same as the output image channel.

  • filter_size (int|tuple) – The filter size. If filter_size is a tuple, it must contain three integers, (filter_size_depth, filter_size_height, filter_size_width). Otherwise, filter_size_depth = filter_size_height = filter_size_width = filter_size.

  • stride (int|tuple) – The stride size. It means the stride in convolution. If stride is a tuple, it must contain three integers, (stride_depth, stride_height, stride_width). Otherwise, stride_depth = stride_height = stride_width = stride. Default: stride = 1.

  • padding (string|int|list|tuple) – The padding size. It means the number of zero-paddings on both sides for each dimention. If padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If padding size is a tuple or list, it could be in three forms: [pad_depth, pad_height, pad_width] or [pad_depth_front, pad_depth_back, pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCDHW”, pool_padding can be in the form [[0,0], [0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NDHWC”, pool_padding can be in the form [[0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.

  • dilation (int|tuple) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain three integers, (dilation_depth, dilation_height, dilation_width). Otherwise, dilation_depth = dilation_height = dilation_width = dilation. Default: dilation = 1.

  • groups (int) – The groups number of the Conv3d Layer. According to grouped convolution in Alex Krizhevsky’s Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1

  • param_attr (ParamAttr|None) – The parameter attribute for learnable parameters/weights of conv3d. If it is set to None or one attribute of ParamAttr, conv3d will create ParamAttr as param_attr. If it is set to None, the parameter is initialized with \(Normal(0.0, std)\), and the \(std\) is \((\frac{2.0 }{filter\_elem\_num})^{0.5}\). Default: None.

  • bias_attr (ParamAttr|bool|None) – The parameter attribute for the bias of conv3d. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv3d will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

  • use_cudnn (bool) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True

  • act (str) – Activation type, if it is set to None, activation is not appended. Default: None.

  • name (str|None) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

  • data_format (str) – The data format of the input and output data. An optional string from: “NCDHW”, “NDHWC”. The default is “NCDHW”. When it is “NCDHW”, the data is stored in the order of: [batch_size, input_channels, input_depth, input_height, input_width].

Returns

A Variable holding Tensor representing the conv3d, whose data type is the same with input. If act is None, the tensor variable storing the convolution result, and if act is not None, the tensor variable storing convolution and non-linearity activation result.

Examples

import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 12, 32, 32], dtype='float32')
conv3d = fluid.layers.conv3d(input=data, num_filters=2, filter_size=3, act="relu")

conv3d_transpose

paddle.fluid.layers.conv3d_transpose(input, num_filters, output_size=None, filter_size=None, padding=0, stride=1, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCDHW')[source]

The convolution3D transpose layer calculates the output based on the input, filter, and dilations, strides, paddings. Input(Input) and output(Output) are in NCDHW or NDHWC format. Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. Parameters(dilations, strides, paddings) are two elements. These two elements represent height and width, respectively. The details of convolution transpose layer, please refer to the following explanation and references therein. If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.

For each input \(X\), the equation is:

\[Out = \sigma (W \ast X + b)\]

In the above equation:

  • \(X\): Input value, a Tensor with NCDHW or NDHWC format.

  • \(W\): Filter value, a Tensor with MCDHW format.

  • \(\ast\): Convolution operation.

  • \(b\): Bias value, a 2-D Tensor with shape [M, 1].

  • \(\sigma\): Activation function.

  • \(Out\): Output value, the shape of \(Out\) and \(X\) may be different.

Example

  • Input:

    Input shape: \((N, C_{in}, D_{in}, H_{in}, W_{in})\)

    Filter shape: \((C_{in}, C_{out}, D_f, H_f, W_f)\)

  • Output:

    Output shape: \((N, C_{out}, D_{out}, H_{out}, W_{out})\)

Where

\[\begin{split}D^\prime_{out} &= (D_{in} - 1) * strides[0] - 2 * paddings[0] + dilations[0] * (D_f - 1) + 1 \\ H^\prime_{out} &= (H_{in} - 1) * strides[1] - 2 * paddings[1] + dilations[1] * (H_f - 1) + 1 \\ W^\prime_{out} &= (W_{in} - 1) * strides[2] - 2 * paddings[2] + dilations[2] * (W_f - 1) + 1 \\ D_{out} &\in [ D^\prime_{out}, D^\prime_{out} + strides[0] ] \\ H_{out} &\in [ H^\prime_{out}, H^\prime_{out} + strides[1] ] \\ W_{out} &\in [ W^\prime_{out}, W^\prime_{out} + strides[2] ]\end{split}\]

Note

The conv3d_transpose can be seen as the backward of the conv3d. For conv3d, when stride > 1, conv3d maps multiple input shape to the same output shape, so for conv3d_transpose, when stride > 1, input shape maps multiple output shape. If output_size is None, \(H_{out} = H^\prime_{out}, :math:\); else, the \(D_{out}\) of the output size must between \(D^\prime_{out}\) and \(D^\prime_{out} + strides[0]\), the \(H_{out}\) of the output size must between \(H^\prime_{out}\) and \(H^\prime_{out} + strides[1]\), and the \(W_{out}\) of the output size must between \(W^\prime_{out}\) and \(W^\prime_{out} + strides[2]\), conv3d_transpose can compute the kernel size automatically.

Parameters
  • input (Variable) – The input is 5-D Tensor with shape [N, C, D, H, W] or [N, D, H, W, C], the data type of input is float32 or float64.

  • num_filters (int) – The number of the filter. It is as same as the output image channel.

  • output_size (int|tuple, optional) – The output image size. If output size is a tuple, it must contain three integers, (image_depth, image_height, image_width). This parameter only works when filter_size is None. If output_size and filter_size are specified at the same time, They should follow the formula above. Default: None. Output_size and filter_size should not be None at the same time.

  • filter_size (int|tuple, optional) – The filter size. If filter_size is a tuple, it must contain three integers, (filter_size_depth, filter_size_height, filter_size_width). Otherwise, filter_size_depth = filter_size_height = filter_size_width = filter_size. None if use output size to calculate filter_size. Default: None. filter_size and output_size should not be None at the same time.

  • padding (int|list|str|tuple, optional) –

    The padding size. The padding argument effectively

    adds dilation * (kernel - 1) amount of zero-padding on both sides of input. If padding is a string, either ‘VALID’ or ‘SAME’ supported, which is the padding algorithm. If padding is a tuple or list, it could be in three forms: [pad_depth, pad_height, pad_width] or

    [pad_depth_front, pad_depth_back, pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is ‘NCDHW’, padding can be in the form [[0,0], [0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is ‘NDHWC’, padding can be in the form [[0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.

  • stride (int|tuple, optional) – The stride size. It means the stride in transposed convolution. If stride is a tuple, it must contain three integers, (stride_depth, stride_height, stride_width). Otherwise, stride_depth = stride_height = stride_width = stride. Default: stride = 1.

  • dilation (int|tuple, optional) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain three integers, (dilation_depth, dilation_height, dilation_width). Otherwise, dilation_depth = dilation_height = dilation_width = dilation. Default: dilation = 1.

  • groups (int, optional) – The groups number of the Conv3d transpose layer. Inspired by grouped convolution in Alex Krizhevsky’s Deep CNN paper, in which when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1

  • param_attr (ParamAttr, optional) – The parameter attribute for learnable parameters/weights of conv3d_transpose. If it is set to None or one attribute of ParamAttr, conv3d_transpose will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.

  • bias_attr (ParamAttr|bool, optional) – The parameter attribute for the bias of conv3d_transpose. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv3d_transpose will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

  • use_cudnn (bool, optional) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True

  • act (str, optional) – Activation type, if it is set to None, activation is not appended. Default: None.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

  • data_format (str, optional) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width]. Default: ‘NCDHW’.

Returns

A Variable holding Tensor representing the conv3d_transpose, whose data type is the same with input and shape is (num_batches, channels, out_d, out_h, out_w) or (num_batches, out_d, out_h, out_w, channels). If act is None, the tensor variable storing the transposed convolution result, and if act is not None, the tensor variable storing transposed convolution and non-linearity activation result.

Raises

ValueError – If the shapes of output, input, filter_size, stride, padding and groups mismatch.

Examples

import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 12, 32, 32], dtype='float32')
conv3d_transpose = fluid.layers.conv3d_transpose(input=data, num_filters=2, filter_size=3)

cos_sim

paddle.fluid.layers.cos_sim(X, Y)[source]

Cosine Similarity Operator

\(Out = \frac{X^T * Y}{(\sqrt{X^T * X} * \sqrt{Y^T * Y})}\)

The input X and Y must have the same shape, except that the 1st dimension of input Y could be just 1 (different from input X), which will be broadcasted to match the shape of input X before computing their cosine similarity.

Both the input X and Y can carry the LoD (Level of Details) information, or not. But the output only shares the LoD information with input X.

Parameters
  • X (Variable) – The 1st input of cos_sim op, LoDTensor with shape [N_1, N_2, ..., N_k], the data type is float32.

  • Y (Variable) – The 2nd input of cos_sim op, Tensor with shape [N_1 or 1, N_2, ..., N_k], the data type is float32.

Returns

A Variable holding LoDTensor representing the output of cosine(X, Y).

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[3, 7], dtype='float32')
y = fluid.data(name='y', shape=[1, 7], dtype='float32')
out = fluid.layers.cos_sim(x, y)

crf_decoding

paddle.fluid.layers.crf_decoding(input, param_attr, label=None, length=None)[source]

The crf_decoding operator reads the emission feature weights and the transition feature weights learned by the linear_chain_crf operator and performs decoding. It implements the Viterbi algorithm which is a dynamic programming algorithm for finding the most likely sequence of hidden states, called the Viterbi path, that results in a sequence of observed tags.

The output of this operator changes according to whether Input(Label) is given:

  1. Input(Label) is given: This happens in training. This operator is used to co-work with the chunk_eval operator. When Input(Label) is given, the crf_decoding operator returns tensor with the sampe shape as Input(Label) whose values are fixed to be 0, indicating an incorrect prediction, or 1 indicating a tag is correctly predicted. Such an output is the input to chunk_eval operator.

  2. Input(Label) is not given: This is the standard decoding process.

The crf_decoding operator returns a row vector with shape [N x 1]/[B x S], here the shape depends on the inputs are LoDTensors or common tensors, whose values range from 0 to maximum tag number - 1, Each element indicates an index of a predicted tag.

Parameters
  • input (Variable) – (Tensor/LoDTensor). For a LoDTensor input, its shape is [N x D] where N is the total sequence length of the mini-batch and D is the total tag number. While for a tensor input, its shape is [B X S X D] with B the batch size and S the sequence length of each sample after padding. This input is the unscaled emission weight matrix of the linear_chain_crf operator. The data type is float32 or float64

  • param_attr (ParamAttr|None) – To specify the weight parameter attribute. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • label (Variable, optional) – (Tensor/LoDTensor). The ground truth with shape [N x 1] (for LoDTensor) or [B x S] (for Tensor). This input is optional. See more details in the operator’s comments. The data type is int64

  • length (Variable, optional) – (Tensor). The actual length of each sample before padding with shape [B x 1]. It means the Input(Emission), Input(Label) and Output(ViterbiPath) are common tensors with padding when this input is given. The data type is int64

Returns

(Tensor/LoDTensor). The decoding results. What to return changes depending on whether the Input(Label) (the ground truth) is given. See more details in the operator’s comment. The data type is int64

Return type

Variable

Examples

import paddle.fluid as fluid

# LoDTensor-based example
num_labels = 10
feature = fluid.data(name='word_emb', shape=[-1, 784], dtype='float32', lod_level=1)
label = fluid.data(name='label', shape=[-1, 1], dtype='int64', lod_level=1)
emission = fluid.layers.fc(input=feature, size=num_labels)

crf_cost = fluid.layers.linear_chain_crf(input=emission, label=label,
          param_attr=fluid.ParamAttr(name="crfw"))
crf_decode = fluid.layers.crf_decoding(input=emission,
          param_attr=fluid.ParamAttr(name="crfw"))

# Common tensor example
num_labels, max_len = 10, 20
feature = fluid.data(name='word_emb_pad', shape=[-1, max_len, 784], dtype='float32')
label = fluid.data(name='label_pad', shape=[-1, max_len, 1], dtype='int64')
length = fluid.data(name='length', shape=[-1, 1], dtype='int64')
emission = fluid.layers.fc(input=feature, size=num_labels,
                           num_flatten_dims=2)

crf_cost = fluid.layers.linear_chain_crf(input=emission, label=label, length=length,
          param_attr=fluid.ParamAttr(name="crfw_pad"))
crf_decode = fluid.layers.crf_decoding(input=emission, length=length,
          param_attr=fluid.ParamAttr(name="crfw_pad"))

crop

paddle.fluid.layers.crop(x, shape=None, offsets=None, name=None)[source]

Crop input into output, as specified by offsets and shape.

Warning: THIS OP IS DEPRECATED. It will be removed in the future version. Instructions for updating: Use api_fluid_layers_crop_tensor instead.

* Case 1:
    Given
        X = [[0, 1, 2, 0, 0]
             [0, 3, 4, 0, 0]
             [0, 0, 0, 0, 0]],
    and
        shape = [2, 2],
        offsets = [0, 1],
    output is:
        Out = [[1, 2],
               [3, 4]].
* Case 2:
    Given
        X = [[0, 1, 2, 5, 0]
             [0, 3, 4, 6, 0]
             [0, 0, 0, 0, 0]],
    and shape is tensor
        shape = [[0, 0, 0]
                 [0, 0, 0]]
    and
        offsets = [0, 1],

    output is:
        Out = [[1, 2, 5],
               [3, 4, 6]].
Parameters
  • x (Variable) – Tensor, data type can be float32 or float64.

  • shape (Variable|list/tuple of integers) – The output shape is specified by shape, which can be a Tensor or a list/tuple of integers. If it is a Tensor, it’s rank must be the same as x , only it’s shape will be used, and the value of it will be ignored. This way is suitable for the case that the output shape may be changed each iteration. If it is a list/tuple of integers, it’s length must be the same as the rank of x

  • offsets (Variable|list/tuple of integers|None) – Specifies the cropping offsets at each dimension. It can be a Tensor or a list/tuple of integers. If it is a Tensor, it’s rank must be the same as x. This way is suitable for the case that the offsets may be changed each iteration. If it is a list/tuple of integers, it’s length must be the same as the rank of x. If None, the offsets are 0 at each dimension.

  • name (str, optional) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.

Returns

The cropped Tensor, which has the same rank and data type with x

Return Type:

Variable

Raises

ValueError – If shape is not a list, tuple or Variable.

Examples

import paddle.fluid as fluid
x = fluid.data(name="x", shape=[3, 3, 5], dtype="float32")
y = fluid.data(name="y", shape=[2, 2, 3], dtype="float32")
crop = fluid.layers.crop(x, shape=y)

# or
z = fluid.data(name="z", shape=[3, 3, 5], dtype="float32")
crop = fluid.layers.crop(z, shape=[2, 2, 3])

cross_entropy

paddle.fluid.layers.cross_entropy(input, label, soft_label=False, ignore_index=-100)[source]

This operator computes the cross entropy between input and label. It supports both hard-label and and soft-label cross entropy computation.

  1. Hard-label cross entropy: if soft_label=False, \(label[i_1, i_2, ..., i_k]\) is the hard label of each sample.

    \[output[i_1, i_2, ..., i_k]=-log(input[i_1, i_2, ..., i_k, j]), label[i_1, i_2, ..., i_k] = j, j != ignore\_index\]
  2. Soft-label cross entropy: if soft_label=True, \(label[i_1, i_2, ..., i_k, j]\) is the soft label of each sample corresponding to the j-th class.

    \[output[i_1, i_2, ..., i_k]= -\sum_{j}label[i_1,i_2,...,i_k,j]*log(input[i_1, i_2, ..., i_k,j])\]
Parameters
  • input (Variable) – a multidimensional Tensor with shape \([N_1, N_2, ..., N_k, D]\), where the last dimension D is the class number. The data type should be float32 or float64.

  • label (Variable) – label value corresponding to input. If soft_label=False, the dimension of label should be \([N_1, N_2, ..., N_k]\) or \([N_1, N_2, ..., N_k, 1]\) , and its data type should be int64, and the value must be inside [0, D). If soft_label=True, the shape, data type of label should be the same with input, and the sum of soft label value of each sample should be 1.

  • soft_label (bool) – indicate whether label is soft. Default False, meaning that the label is hard. If soft_label=True, the label is soft.

  • ignore_index (int) – specify an ignorable label value. The ignored label would be omitted when computing. If it is a negative integer, no label would be ignored. Only valid when soft_label=False. Default -100.

Returns

A Variable holding Tensor representing the cross entropy, whose data type is the same with input. If soft_label=False, the shape of output is the same with label. If soft_label=True, the shape of output is \([N_1, N_2, ..., N_k, 1]\) .

Examples

import paddle.fluid as fluid
class_num = 7
x = fluid.data(name='x', shape=[None, 3, 10], dtype='float32')
label = fluid.data(name='label', shape=[None, 1], dtype='int64')
predict = fluid.layers.fc(input=x, size=class_num, act='softmax')
cost = fluid.layers.cross_entropy(input=predict, label=label)

ctc_greedy_decoder

paddle.fluid.layers.ctc_greedy_decoder(input, blank, input_length=None, padding_value=0, name=None)[source]

This op is used to decode sequences by greedy policy by the following steps:

  1. Get the indexes of maximum value for each row in input. a.k.a. numpy.argmax(input, axis=0).

  2. For each sequence in result of step1, merge repeated tokens between two blanks and delete all blanks.

This op is implemented in two modes: lod and padding, either of them can be used. The input can be either LoDTensor or Tensor, corresponding to lod and padding mode respectively.

A simple example as below:

Given:
(1) for lod mode:

input.data = [[0.6, 0.1, 0.3, 0.1],
              [0.3, 0.2, 0.4, 0.1],
              [0.1, 0.5, 0.1, 0.3],
              [0.5, 0.1, 0.3, 0.1],

              [0.5, 0.1, 0.3, 0.1],
              [0.2, 0.2, 0.2, 0.4],
              [0.2, 0.2, 0.1, 0.5],
              [0.5, 0.1, 0.3, 0.1]]

input.lod = [[4, 4]]

Computation:

step1: Apply argmax to first input sequence which is input.data[0:4]. Then we get:
       [[0], [2], [1], [0]]
step2: merge repeated tokens and remove blank which is 0. Then we get first output sequence:
       [[2], [1]]

Finally:

output.data = [[2],
               [1],
               [3]]

output.lod = [[2, 1]]

(2) for padding mode:

 input.data = [[[0.6, 0.1, 0.3, 0.1],
                [0.3, 0.2, 0.4, 0.1],
                [0.1, 0.5, 0.1, 0.3],
                [0.5, 0.1, 0.3, 0.1]],

               [[0.5, 0.1, 0.3, 0.1],
                [0.2, 0.2, 0.2, 0.4],
                [0.2, 0.2, 0.1, 0.5],
                [0.5, 0.1, 0.3, 0.1]]]

input_length.data = [[4], [4]]
input.shape = [2, 4, 4]

step1: Apply argmax to first input sequence which is input.data[0:4]. Then we get:
       [[0], [2], [1], [0]], for input.data[4:8] is [[0], [3], [3], [0]], shape is [2,4,1]
step2: Change the argmax result to use padding mode, then argmax result is
        [[0, 2, 1, 0], [0, 3, 3, 0]], shape is [2, 4], lod is [], input_length is [[4], [4]]
step3: Apply ctc_align to padding argmax result, padding_value is 0

Finally:
output.data = [[2, 1, 0, 0],
               [3, 0, 0, 0]]
output_length.data = [[2], [1]]
Parameters
  • input (Variable) – the probabilities of variable-length sequences. When in lod mode, it is a 2-D LoDTensor with LoD information. It’s shape is [Lp, num_classes + 1] where Lp is the sum of all input sequences’ length and num_classes is the true number of classes. When in padding mode, it is a 3-D Tensor with padding, It’s shape is [batch_size, N, num_classes + 1]. (not including the blank label). The data type can be float32 or float64.

  • blank (int) – the blank label index of Connectionist Temporal Classification (CTC) loss, which is in the half-opened interval [0, num_classes + 1).

  • input_length (Variable, optional) – 2-D LoDTensor, shape is [batch_size, 1], data type is int64. It is used for padding mode. In lod mode, input_length is None.

  • padding_value (int) – padding value.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

For lod mode, returns the result of CTC greedy decoder, 2-D LoDTensor, shape is [Lp, 1], data type is int64. ‘Lp’ is the sum of all output sequences’ length. If all the sequences in result were empty, the result LoDTensor will be [-1] with empty LoD [[]].

For padding mode, returns a tuple of (output, output_length), which was describled as below:

output, 2-D Tensor, shape is [batch_size, N], data type is int64.

output_length, 2-D Tensor, shape is [batch_size, 1], data type is int64. It is the length of each sequence of output for padding mode.

Return type:

For lod mode: Variable

For padding mode: tuple of two Variables (output, output_length).

Examples

# for lod mode
import paddle.fluid as fluid
x = fluid.data(name='x', shape=[None, 8], dtype='float32', lod_level=1)
cost = fluid.layers.ctc_greedy_decoder(input=x, blank=0)

# for padding mode
x_pad = fluid.data(name='x_pad', shape=[10, 4, 8], dtype='float32')
x_pad_len = fluid.data(name='x_pad_len', shape=[10, 1], dtype='int64')
out, out_len = fluid.layers.ctc_greedy_decoder(input=x_pad, blank=0,
                input_length=x_pad_len)

data_norm

paddle.fluid.layers.data_norm(input, act=None, epsilon=1e-05, param_attr=None, data_layout='NCHW', in_place=False, name=None, moving_mean_name=None, moving_variance_name=None, do_model_average_for_mean_and_var=True)[source]

Data Normalization Layer

This op can be used as a normalizer function for conv2d and fully_connected operations. The required data format for this layer is one of the following:

  1. NHWC [batch, in_height, in_width, in_channels]

  2. NCHW [batch, in_channels, in_height, in_width]

\(input\) is the input features over a mini-batch.

\[\begin{split}\mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\ \ mini-batch\ mean \\ \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \ \mu_{\beta})^2 \qquad &//\ mini-batch\ variance \\ \hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\ \sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift\end{split}\]
Parameters
  • input (variable) – The input variable which is a LoDTensor.

  • act (string, Default None) – Activation type, linear|relu|prelu|…

  • epsilon (float, Default 1e-05) –

  • param_attr (ParamAttr) – The parameter attribute for Parameter scale.

  • data_layout (string, default NCHW) – NCHW|NHWC

  • in_place (bool, Default False) – Make the input and output of batch norm reuse memory.

  • name (string, Default None) – A name for this layer(optional). If set None, the layer will be named automatically.

  • moving_mean_name (string, Default None) – The name of moving_mean which store the global Mean.

  • moving_variance_name (string, Default None) – The name of the moving_variance which store the global Variance.

  • do_model_average_for_mean_and_var (bool, Default True) – Whether parameter mean and variance should do model average when model average is enabled.

Returns

A tensor variable which is the result after applying data normalization on the input.

Return type

Variable

Examples

import paddle.fluid as fluid

hidden1 = fluid.data(name="hidden1", shape=[64, 200])
hidden2 = fluid.layers.data_norm(name="hidden2", input=hidden1)

deformable_conv

paddle.fluid.layers.deformable_conv(input, offset, mask, num_filters, filter_size, stride=1, padding=0, dilation=1, groups=None, deformable_groups=None, im2col_step=None, param_attr=None, bias_attr=None, modulated=True, name=None)[source]

Deformable Convolution op

Compute 2-D deformable convolution on 4-D input. Given input image x, output feature map y, the deformable convolution operation can be expressed as follow:

Deformable Convolution v2:

\[y(p) = \sum_{k=1}^{K}{w_k * x(p + p_k + \Delta p_k) * \Delta m_k}\]

Deformable Convolution v1:

\[y(p) = \sum_{k=1}^{K}{w_k * x(p + p_k + \Delta p_k)}\]

Where \(\Delta p_k\) and \(\Delta m_k\) are the learnable offset and modulation scalar for the k-th location, Which \(\Delta m_k\) is one in deformable convolution v1. Please refer to Deformable ConvNets v2: More Deformable, Better Results and Deformable Convolutional Networks.

Example

  • Input:

    Input shape: \((N, C_{in}, H_{in}, W_{in})\)

    Filter shape: \((C_{out}, C_{in}, H_f, W_f)\)

    Offset shape: \((N, 2 * deformable\_groups * H_f * H_w, H_{in}, W_{in})\)

    Mask shape: \((N, deformable\_groups * H_f * H_w, H_{in}, W_{in})\)

  • Output:

    Output shape: \((N, C_{out}, H_{out}, W_{out})\)

Where

\[\begin{split}H_{out}&= \frac{(H_{in} + 2 * paddings[0] - (dilations[0] * (H_f - 1) + 1))}{strides[0]} + 1 \\ W_{out}&= \frac{(W_{in} + 2 * paddings[1] - (dilations[1] * (W_f - 1) + 1))}{strides[1]} + 1\end{split}\]
Parameters
  • input (Variable) – The input image with [N, C, H, W] format. A Tensor with type float32, float64.

  • offset (Variable) – The input coordinate offset of deformable convolution layer. A Tensor with type float32, float64.

  • Mask (Variable, Optional) – The input mask of deformable covolution layer. A Tensor with type float32, float64.It should be None when you use deformable_conv_v2.

  • num_filters (int) – The number of filter. It is as same as the output image channel.

  • filter_size (int|tuple) – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_H, filter_size_W). Otherwise, the filter will be a square.

  • stride (int|tuple) – The stride size. If stride is a tuple, it must contain two integers, (stride_H, stride_W). Otherwise, the stride_H = stride_W = stride. Default: stride = 1.

  • padding (int|tuple) – The padding size. If padding is a tuple, it must contain two integers, (padding_H, padding_W). Otherwise, the padding_H = padding_W = padding. Default: padding = 0.

  • dilation (int|tuple) – The dilation size. If dilation is a tuple, it must contain two integers, (dilation_H, dilation_W). Otherwise, the dilation_H = dilation_W = dilation. Default: dilation = 1.

  • groups (int) – The groups number of the deformable conv layer. According to grouped convolution in Alex Krizhevsky’s Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1.

  • deformable_groups (int) – The number of deformable group partitions. Default: deformable_groups = 1.

  • im2col_step (int) – Maximum number of images per im2col computation; The total batch size should be divisable by this value or smaller than this value; if you face out of memory problem, you can try to use a smaller value here. Default: im2col_step = 64.

  • param_attr (ParamAttr, Optional) – The parameter attribute for learnable parameters/weights of deformable conv. If it is set to None or one attribute of ParamAttr, deformable conv will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with \(Normal(0.0, std)\), and the \(std\) is \((\frac{2.0 }{filter\_elem\_num})^{0.5}\). Default: None.

  • bias_attr (ParamAttr|bool, Optional) – The parameter attribute for the bias of deformable conv layer. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

  • modulated (bool) – Make sure which version should be used between v1 and v2, where v2 is used while True. Default: True.

  • name (str, Optional) – For details, please refer to api_guide_Name. Generally, no setting is required. Default: None.

Returns

The tensor variable storing the deformable convolution result. A Tensor with type float32, float64.

Return type

Variable

Raises

ValueError – If the shapes of input, filter_size, stride, padding and groups mismatch.

Examples

#deformable conv v2:

import paddle.fluid as fluid
C_in, H_in, W_in = 3, 32, 32
filter_size, deformable_groups = 3, 1
data = fluid.data(name='data', shape=[None, C_in, H_in, W_in], dtype='float32')
offset = fluid.data(name='offset', shape=[None, 2*deformable_groups*filter_size**2, H_in, W_in], dtype='float32')
mask = fluid.data(name='mask', shape=[None, deformable_groups*filter_size**2, H_in, W_in], dtype='float32')
out = fluid.layers.deformable_conv(input=data, offset=offset, mask=mask,
                                   num_filters=2, filter_size=filter_size, padding=1, modulated=True)

#deformable conv v1:

import paddle.fluid as fluid
C_in, H_in, W_in = 3, 32, 32
filter_size, deformable_groups = 3, 1
data = fluid.data(name='data', shape=[None, C_in, H_in, W_in], dtype='float32')
offset = fluid.data(name='offset', shape=[None, 2*deformable_groups*filter_size**2, H_in, W_in], dtype='float32')
out = fluid.layers.deformable_conv(input=data, offset=offset, mask=None,
                                   num_filters=2, filter_size=filter_size, padding=1, modulated=False)

deformable_roi_pooling

paddle.fluid.layers.deformable_roi_pooling(input, rois, trans, no_trans=False, spatial_scale=1.0, group_size=[1, 1], pooled_height=1, pooled_width=1, part_size=None, sample_per_part=1, trans_std=0.1, position_sensitive=False, name=None)[source]

Deformable ROI Pooling Layer

Performs deformable region-of-interest pooling on inputs. As described in Deformable Convolutional Networks, it will get offset for each bin after roi pooling so that pooling at correct region. Batch_size will change to the number of region bounding boxes after deformable_roi_pooling.

The operation has three steps:

  1. Dividing each region proposal into equal-sized sections with the pooled_width and pooled_height.

  2. Add offset to pixel in ROI to get new location and the new value which are computed directly through bilinear interpolation with four nearest pixel.

  3. Sample several points in each bin to get average values as output.

Parameters
  • input (Variable) – The input of deformable roi pooling and it is tensor which value type is float32. The shape of input is [N, C, H, W]. Where N is batch size, C is number of input channels, H is height of the feature, and W is the width of the feature.

  • rois (Variable) – ROIs (Regions of Interest) with type float32 to pool over. It should be a 2-D LoDTensor of shape (num_rois, 4), and the lod level is 1. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates, which value type is float32.

  • trans (Variable) – Offset of features on ROIs while pooling which value type is float32. The format is [N, C, H, W], where N is number of ROIs, C is number of channels, which indicate the offset distance in the x and y directions, H is pooled height, and W is pooled width.

  • no_trans (bool) – Whether to add offset to get new value or not while roi pooling, which value with type bool is True or False. If value is True, no offset will be added in operation. Default: False.

  • spatial_scale (float) – Ratio of input feature map height (or width) to raw image height (or width), which value type is float32. Equals the reciprocal of total stride in convolutional layers, Default: 1.0.

  • group_size (list|tuple) – The number of groups which input channels are divided and the input is list or tuple, which value type is int32. (eg.number of input channels is k1 * k2 * (C + 1), which k1 and k2 are group width and height and C+1 is number of output chanels.) eg.(4, 6), which 4 is height of group and 6 is width of group. Default: [1, 1].

  • pooled_height (int) – The pooled output height which value type is int32. Default: 1.

  • pooled_width (int) – The pooled output width which value type is int32. Default: 1.

  • part_size (list|tuple) – The height and width of offset which values in list or tuple is int32, eg.(4, 6), which height is 4 and width is 6, and values always equal to pooled_height and pooled_width. Default: if None, default value is [pooled_height, pooled_width].

  • sample_per_part (int) – The number of samples in each bin which value type is int32. If value is bigger, it will consume more performance. Default: 1.

  • trans_std (float) – Coefficient of offset which value type is float32. It controls weight of offset. Default: 0.1.

  • position_sensitive (bool) – Whether to choose deformable psroi pooling mode or not, and value type is bool(True or False). If value is False, input dimension equals to output dimension. If value is True, input dimension shoule be output dimension * pooled_height * pooled_width. Default: False.

  • name (str|None) – Name of layer. Default: None.

Returns

Output of deformable roi pooling is that, if position sensitive is False, input dimension equals to output dimension. If position sensitive is True, input dimension should be the result of output dimension divided by pooled height and pooled width.

Return type

Variable

Examples

# position_sensitive=True
import paddle.fluid as fluid
input = fluid.data(name="input",
                   shape=[2, 192, 64, 64],
                   dtype='float32')
rois = fluid.data(name="rois",
                  shape=[-1, 4],
                  dtype='float32',
                  lod_level=1)
trans = fluid.data(name="trans",
                   shape=[2, 384, 64, 64],
                   dtype='float32')
x = fluid.layers.deformable_roi_pooling(input=input,
                                        rois=rois,
                                        trans=trans,
                                        no_trans=False,
                                        spatial_scale=1.0,
                                        group_size=(1, 1),
                                        pooled_height=8,
                                        pooled_width=8,
                                        part_size=(8, 8),
                                        sample_per_part=4,
                                        trans_std=0.1,
                                        position_sensitive=True)

# position_sensitive=False
import paddle.fluid as fluid
input = fluid.data(name="input",
                   shape=[2, 192, 64, 64],
                   dtype='float32')
rois = fluid.data(name="rois",
                  shape=[-1, 4],
                  dtype='float32',
                  lod_level=1)
trans = fluid.data(name="trans",
                   shape=[2, 384, 64, 64],
                   dtype='float32')
x = fluid.layers.deformable_roi_pooling(input=input,
                                        rois=rois,
                                        trans=trans,
                                        no_trans=False,
                                        spatial_scale=1.0,
                                        group_size=(1, 1),
                                        pooled_height=8,
                                        pooled_width=8,
                                        part_size=(8, 8),
                                        sample_per_part=4,
                                        trans_std=0.1,
                                        position_sensitive=False)

dice_loss

paddle.fluid.layers.dice_loss(input, label, epsilon=1e-05, name=None)[source]

Dice loss for comparing the similarity between the input predictions and the label. This implementation is for binary classification, where the input is sigmoid predictions of each pixel, usually used for segmentation task. The dice loss can be defined as the following equation:

\[\begin{split}dice\_loss &= 1 - \frac{2 * intersection\_area}{total\_area} \\ &= \frac{(total\_area - intersection\_area) - intersection\_area}{total\_area} \\ &= \frac{(union\_area - intersection\_area)}{total\_area}\end{split}\]
Parameters
  • input (Variable) – Tensor, rank>=2, shape is \([N_1, N_2, ..., N_D]\), where \(N_1\) is the batch_size, \(N_D\) is 1. It is usually the output predictions of sigmoid activation. The data type can be float32 or float64.

  • label (Variable) – Tensor, the groud truth with the same rank as input, shape is \([N_1, N_2, ..., N_D]\). where \(N_1\) is the batch_size, \(N_D\) is 1. The data type can be float32 or float64.

  • epsilon (float) – The epsilon will be added to the numerator and denominator. If both input and label are empty, it makes sure dice is 1. Default: 0.00001

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

The dice loss with shape [1], data type is the same as input .

Return Type:

Varaible

Example

import paddle.fluid as fluid
x = fluid.data(name='data', shape = [3, 224, 224, 1], dtype='float32')
label = fluid.data(name='label', shape=[3, 224, 224, 1], dtype='float32')
predictions = fluid.layers.sigmoid(x)
loss = fluid.layers.dice_loss(input=predictions, label=label)

dropout

paddle.fluid.layers.dropout(x, dropout_prob, is_test=False, seed=None, name=None, dropout_implementation='downgrade_in_infer')[source]

Computes dropout.

Drop or keep each element of x independently. Dropout is a regularization technique for reducing overfitting by preventing neuron co-adaption during training. The dropout operator randomly sets (according to the given dropout probability) the outputs of some units to zero, while others are remain unchanged.

dropout op can be removed from the program to make the program more efficient.

Parameters
  • x (Variable) – The input tensor variable. The data type is float16 or float32 or float64.

  • dropout_prob (float) – Probability of setting units to zero.

  • is_test (bool) – A flag indicating whether it is in test phrase or not.

  • seed (int) – A Python integer used to create random seeds. If this parameter is set to None, a random seed is used. NOTE: If an integer seed is given, always the same output units will be dropped. DO NOT use a fixed seed in training.Default: None.

  • name (str|None) – A name for this layer(optional). If set None, the layer will be named automatically.

  • dropout_implementation (string) –

    [‘downgrade_in_infer’(default)|’upscale_in_train’]

    1. downgrade_in_infer(default), downgrade the outcome at inference

      • train: out = input * mask

      • inference: out = input * (1.0 - dropout_prob)

      (mask is a tensor same shape with input, value is 0 or 1 ratio of 0 is dropout_prob)

    2. upscale_in_train, upscale the outcome at training time

      • train: out = input * mask / ( 1.0 - dropout_prob )

      • inference: out = input

      (mask is a tensor same shape with input, value is 0 or 1 ratio of 0 is dropout_prob)

Returns

A Variable holding Tensor representing the dropout, has same shape and data type with x.

Examples

import paddle.fluid as fluid
x = fluid.data(name="data", shape=[None, 32, 32], dtype="float32")
droped = fluid.layers.dropout(x, dropout_prob=0.5)

dynamic_gru

paddle.fluid.layers.dynamic_gru(input, size, param_attr=None, bias_attr=None, is_reverse=False, gate_activation='sigmoid', candidate_activation='tanh', h_0=None, origin_mode=False)[source]

Note: The input type of this must be LoDTensor. If the input type to be processed is Tensor, use StaticRNN .

This operator is used to perform the calculations for a single layer of Gated Recurrent Unit (GRU) on full sequences step by step. The calculations in one time step support these two modes:

If origin_mode is True, then the formula used is from paper Learning Phrase Representations using RNN Encoder Decoder for Statistical Machine Translation .

\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t-1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t-1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t-1}) + b_c)\\h_t & = u_t \odot h_{t-1} + (1-u_t) \odot \tilde{h_t}\end{aligned}\end{align} \]

if origin_mode is False, then the formula used is from paper Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t-1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t-1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t-1}) + b_c)\\h_t & = (1-u_t) \odot h_{t-1} + u_t \odot \tilde{h_t}\end{aligned}\end{align} \]

\(x_t\) is the input of current time step, but it is not from input . This operator does not include the calculations \(W_{ux}x_{t}, W_{rx}x_{t}, W_{cx}x_{t}\) , Note thus a fully-connect layer whose size is 3 times of size should be used before this operator, and the output should be used as input here. \(h_{t-1}\) is the hidden state from previous time step. \(u_t\) , \(r_t\) , \(\tilde{h_t}\) and \(h_t\) stand for update gate, reset gate, candidate hidden and hidden output separately. \(W_{uh}, b_u\) , \(W_{rh}, b_r\) and \(W_{ch}, b_c\) stand for the weight matrix and bias used in update gate, reset gate, candidate hidden calculations. For implementation, the three weight matrix are merged into a tensor shaped \([D, D \times 3]\) , the three bias are concatenated as a tensor shaped \([1, D \times 3]\) , where \(D\) stands for the hidden size; The data layout of weight tensor is: \(W_{uh}\) and \(W_{rh}\) are concatenated with shape \([D, D \times 2]\) lying on the first part, and \(W_{ch}\) lying on the latter part with shape \([D, D]\) .

Parameters
  • input (Variable) – A LoDTensor whose lod level is 1, representing the input after linear projection. Its shape should be \([T, D \times 3]\) , where \(T\) stands for the total sequence lengths in this mini-batch, \(D\) for the hidden size. The data type should be float32 or float64.

  • size (int) – Indicate the hidden size.

  • param_attr (ParamAttr, optional) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • bias_attr (ParamAttr, optional) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .

  • is_reverse (bool, optional) – Whether to compute in the reversed order of input sequences. Default False.

  • gate_activation (str, optional) – The activation fuction corresponding to \(act_g\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “sigmoid”.

  • candidate_activation (str, optional) – The activation fuction corresponding to \(act_c\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “tanh”.

  • h_0 (Variable, optional) – A Tensor representing the initial hidden state. It not provided, the default initial hidden state is 0. The shape is \([N, D]\) , where \(N\) is the number of sequences in the mini-batch, \(D\) for the hidden size. The data type should be same as input . Default None.

Returns

A LoDTensor whose lod level is 1 and shape is \([T, D]\) , where \(T\) stands for the total sequence lengths in this mini-batch \(D\) for the hidden size. It represents GRU transformed sequence output, and has the same lod and data type with input .

Return type

Variable

Examples

import paddle.fluid as fluid

dict_dim, emb_dim = 128, 64
data = fluid.data(name='sequence',
          shape=[None],
          dtype='int64',
          lod_level=1)
emb = fluid.embedding(input=data, size=[dict_dim, emb_dim])
hidden_dim = 512
x = fluid.layers.fc(input=emb, size=hidden_dim * 3)
hidden = fluid.layers.dynamic_gru(input=x, size=hidden_dim)

dynamic_lstm

paddle.fluid.layers.dynamic_lstm(input, size, h_0=None, c_0=None, param_attr=None, bias_attr=None, use_peepholes=True, is_reverse=False, gate_activation='sigmoid', cell_activation='tanh', candidate_activation='tanh', dtype='float32', name=None)[source]
Note:
  1. This OP only supports LoDTensor as inputs. If you need to deal with Tensor, please use lstm .

  2. In order to improve efficiency, users must first map the input of dimension [T, hidden_size] to input of [T, 4 * hidden_size], and then pass it to this OP.

The implementation of this OP include diagonal/peephole connections. Please refer to Gers, F. A., & Schmidhuber, J. (2000) . If you do not need peephole connections, please set use_peepholes to False .

This OP computes each timestep as follows:

\[i_t = \sigma(W_{ix}x_{t} + W_{ih}h_{t-1} + b_{x_i} + b_{h_i})\]
\[f_t = \sigma(W_{fx}x_{t} + W_{fh}h_{t-1} + b_{x_f} + b_{h_f})\]
\[o_t = \sigma(W_{ox}x_{t} + W_{oh}h_{t-1} + b_{x_o} + b_{h_o})\]
\[\widetilde{c_t} = tanh(W_{cx}x_t + W_{ch}h_{t-1} + b{x_c} + b_{h_c})\]
\[c_t = f_t \odot c_{t-1} + i_t \odot \widetilde{c_t}\]
\[h_t = o_t \odot tanh(c_t)\]

The symbolic meanings in the formula are as follows:

  • \(x_{t}\) represents the input at timestep \(t\)

  • \(h_{t}\) represents the hidden state at timestep \(t\)

  • \(h_{t-1}, c_{t-1}\) represent the hidden state and cell state at timestep \(t-1\) , respectively

  • \(\widetilde{c_t}\) represents the candidate cell state

  • \(i_t\) , \(f_t\) and \(o_t\) represent input gate, forget gate, output gate, respectively

  • \(W\) represents weight (e.g., \(W_{ix}\) is the weight of a linear transformation of input \(x_{t}\) when calculating input gate \(i_t\) )

  • \(b\) represents bias (e.g., \(b_{i}\) is the bias of input gate)

  • \(\sigma\) represents nonlinear activation function for gate, default sigmoid

  • \(\odot\) represents the Hadamard product of a matrix, i.e. multiplying the elements of the same position for two matrices with the same dimension to get another matrix with the same dimension

Parameters
  • input (api_guide_Variable_en) – LSTM input tensor, multi-dimensional LODTensor of shape \([T, 4*hidden\_size]\) . Data type is float32 or float64.

  • size (int) – must be 4 * hidden_size.

  • h_0 (api_guide_Variable_en , optional) – The initial hidden state of the LSTM, multi-dimensional Tensor of shape \([batch\_size, hidden\_size]\) . Data type is float32 or float64. If set to None, it will be a vector of all 0. Default: None.

  • c_0 (api_guide_Variable_en , optional) – The initial hidden state of the LSTM, multi-dimensional Tensor of shape \([batch\_size, hidden\_size]\) . Data type is float32 or float64. If set to None, it will be a vector of all 0. h_0 and c_0 can be None but only at the same time. Default: None.

  • param_attr (ParamAttr, optional) –

    Parameter attribute of weight. If it is None, the default weight parameter attribute is used. Please refer to ref:api_fluid_ParamAttr’ . If the user needs to set this parameter, the dimension must be :math:`[hidden_size, 4*hidden_size] . Default: None.

    • Weights = \(\{ W_{cr},W_{ir},W_{fr},W_{or} \}\) , the shape is [hidden_size, 4*hidden_size].

  • bias_attr (ParamAttr, optional) –

    The bias attribute for the learnable bias weights, which contains two parts, input-hidden bias weights and peephole connections weights if setting use_peepholes to True. Please refer to ref:`api_fluid_ParamAttr’ . Default: None.

    1. use_peepholes = False - Biases = {\(b_c, b_i, b_f, b_o\)}. - The shape is [1, 4*hidden_size].

    2. use_peepholes = True - Biases = { \(b_c, b_i, b_f, b_o, W_{ic}, W_{fc}, W_{oc}\)}. - The shape is [1, 7*hidden_size].

  • use_peepholes (bool, optional) – Whether to use peephole connection or not. Default: True.

  • is_reverse (bool, optional) – Whether to calculate reverse LSTM. Default: False.

  • gate_activation (str, optional) – The activation for input gate, forget gate and output gate. Default: “sigmoid”.

  • cell_activation (str, optional) – The activation for cell output. Default: “tanh”.

  • candidate_activation (str, optional) – The activation for candidate hidden state. Default: “tanh”.

  • dtype (str, optional) – Data type, can be “float32” or “float64”. Default: “float32”.

  • name (str, optional) – A name for this layer. Please refer to api_guide_Name . Default: None.

Returns

The hidden state and cell state of LSTM

  • hidden: LoDTensor with shape of \([T, hidden\_size]\) , and its lod and dtype is the same as the input.

  • cell: LoDTensor with shape of \([T, hidden\_size]\) , and its lod and dtype is the same as the input.

Return type

tuple ( api_guide_Variable , api_guide_Variable )

Examples

import paddle.fluid as fluid
emb_dim = 256
vocab_size = 10000
hidden_dim = 512

data = fluid.data(name='x', shape=[None], dtype='int64', lod_level=1)
emb = fluid.embedding(input=data, size=[vocab_size, emb_dim], is_sparse=True)

forward_proj = fluid.layers.fc(input=emb, size=hidden_dim * 4,
                               bias_attr=False)

forward, cell = fluid.layers.dynamic_lstm(
    input=forward_proj, size=hidden_dim * 4, use_peepholes=False)
forward.shape  # (-1, 512)
cell.shape  # (-1, 512)

dynamic_lstmp

paddle.fluid.layers.dynamic_lstmp(input, size, proj_size, param_attr=None, bias_attr=None, use_peepholes=True, is_reverse=False, gate_activation='sigmoid', cell_activation='tanh', candidate_activation='tanh', proj_activation='tanh', dtype='float32', name=None, h_0=None, c_0=None, cell_clip=None, proj_clip=None)[source]
Note:
  1. In order to improve efficiency, users must first map the input of dimension [T, hidden_size] to input of [T, 4 * hidden_size], and then pass it to this OP.

This OP implements the LSTMP (LSTM Projected) layer. The LSTMP layer has a separate linear mapping layer behind the LSTM layer. – Sak, H., Senior, A., & Beaufays, F. (2014) .

Compared with the standard LSTM layer, LSTMP has an additional linear mapping layer, which is used to map from the original hidden state \(h_t\) to the lower dimensional state \(r_t\) . This reduces the total number of parameters and computational complexity, especially when the output unit is relatively large.

The default implementation of the OP contains diagonal/peephole connections, please refer to Gers, F. A., & Schmidhuber, J. (2000) . If you need to disable the peephole connections, set use_peepholes to False.

This OP computes each timestep as follows:

\[i_t = \sigma(W_{ix}x_{t} + W_{ir}r_{t-1} + W_{ic}c_{t-1} + b_i)\]
\[f_t = \sigma(W_{fx}x_{t} + W_{fr}r_{t-1} + W_{fc}c_{t-1} + b_f)\]
\[o_t = \sigma(W_{ox}x_{t} + W_{or}r_{t-1} + W_{oc}c_{t-1} + b_o)\]
\[\widetilde{c_t} = act_g(W_{cx}x_t + W_{cr}r_{t-1} + b_c)\]
\[c_t = f_t \odot c_{t-1} + i_t \odot \widetilde{c_t}\]
\[h_t = o_t \odot act_h(c_t)\]
\[r_t = \overline{act_h}(W_{rh}h_t)\]

The symbolic meanings in the formula are as follows:

  • \(x_{t}\) represents the input at timestep \(t\)

  • \(h_{t}\) represents the hidden state at timestep \(t\)

  • \(r_{t}\) : represents the state of the projected output of the hidden state \(h_{t}\)

  • \(h_{t-1}, c_{t-1}, r_{t-1}\) represent the hidden state, cell state and projected output at timestep \(t-1\) , respectively

  • \(\widetilde{c_t}\) represents the candidate cell state

  • \(i_t\) , \(f_t\) and \(o_t\) represent input gate, forget gate, output gate, respectively

  • \(W\) represents weight (e.g., \(W_{ix}\) is the weight of a linear transformation of input \(x_{t}\) when calculating input gate \(i_t\) )

  • \(b\) represents bias (e.g., \(b_{i}\) is the bias of input gate)

  • \(\sigma\) represents nonlinear activation function for gate, default sigmoid

  • \(\odot\) represents the Hadamard product of a matrix, i.e. multiplying the elements of the same position for two matrices with the same dimension to get another matrix with the same dimension

Parameters
  • input (api_guide_Variable_en) – The input of dynamic_lstmp layer, which supports variable-time length input sequence. It is a multi-dimensional LODTensor of shape \([T, 4*hidden\_size]\) . Data type is float32 or float64.

  • size (int) – must be 4 * hidden_size.

  • proj_size (int) – The size of projection output.

  • param_attr (ParamAttr, optional) –

    Parameter attribute of weight. If it is None, the default weight parameter attribute is used. Please refer to ref:api_fluid_ParamAttr’ . If the user needs to set this parameter, the dimension must be :math:`[hidden_size, 4*hidden_size] . Default: None.

    • Weights = \(\{ W_{cr},W_{ir},W_{fr},W_{or} \}\) , the shape is [P, 4*hidden_size] , where P is the projection size.

    • Projection weight = \(\{ W_{rh} \}\) , the shape is [hidden_size, P].

  • bias_attr (ParamAttr, optional) –

    The bias attribute for the learnable bias weights, which contains two parts, input-hidden bias weights and peephole connections weights if setting use_peepholes to True. Please refer to ref:`api_fluid_ParamAttr’ . Default: None.

    1. use_peepholes = False - Biases = {\(b_c, b_i, b_f, b_o\)}. - The shape is [1, 4*hidden_size].

    2. use_peepholes = True - Biases = { \(b_c, b_i, b_f, b_o, W_{ic}, W_{fc}, W_{oc}\)}. - The shape is [1, 7*hidden_size].

  • use_peepholes (bool, optional) – Whether to use peephole connection or not. Default True.

  • is_reverse (bool, optional) – Whether to calculate reverse LSTM. Default False.

  • gate_activation (str, optional) – The activation for input gate, forget gate and output gate. Default “sigmoid”.

  • cell_activation (str, optional) – The activation for cell output. Default “tanh”.

  • candidate_activation (str, optional) – The activation for candidate hidden state. Default “tanh”.

  • proj_activation (str, optional) – The activation for projection output. Default “tanh”.

  • dtype (str, optional) – Data type, can be “float32” or “float64”. Default “float32”.

  • name (str, optional) – A name for this layer. Please refer to api_guide_Name . Default: None.

  • h_0 (api_guide_Variable , optional) – The initial hidden state is an optional input, default is zero. This is a tensor with shape \([batch\_size, P]\) , where P is the projection size. Default: None.

  • c_0 (api_guide_Variable , optional) – The initial cell state is an optional input, default is zero. This is a tensor with shape \([batch\_size, P]\) , where P is the projection size. h_0 and c_0 can be None but only at the same time. Default: None.

  • cell_clip (float, optional) – If not None, the cell state is clipped by this value prior to the cell output activation. Default: None.

  • proj_clip (float, optional) – If num_proj > 0 and proj_clip is provided, then the projected values are clipped elementwise to within [-proj_clip, proj_clip]. Default: None.

Returns

The hidden state and cell state of LSTMP

  • hidden: LoDTensor with shape of \([T, P]\) , and its lod and dtype is the same as the input.

  • cell: LoDTensor with shape of \([T, hidden\_size]\) , and its lod and dtype is the same as the input.

Return type

tuple ( api_guide_Variable , api_guide_Variable )

Examples

import paddle.fluid as fluid
dict_dim, emb_dim = 128, 64
data = fluid.data(name='sequence', shape=[None], dtype='int64', lod_level=1)
emb = fluid.embedding(input=data, size=[dict_dim, emb_dim])
hidden_dim, proj_dim = 512, 256
fc_out = fluid.layers.fc(input=emb, size=hidden_dim * 4,
                        act=None, bias_attr=None)
proj_out, last_c = fluid.layers.dynamic_lstmp(input=fc_out,
                                        size=hidden_dim * 4,
                                        proj_size=proj_dim,
                                        use_peepholes=False,
                                        is_reverse=True,
                                        cell_activation="tanh",
                                        proj_activation="tanh")
proj_out.shape  # (-1, 256)
last_c.shape  # (-1, 512)

edit_distance

paddle.fluid.layers.edit_distance(input, label, normalized=True, ignored_tokens=None, input_length=None, label_length=None)[source]

This op computes the edit distances between a batch of hypothesis strings and their references. Edit distance, also called Levenshtein distance, measures how dissimilar two strings are by counting the minimum number of operations to transform one string into anthor. Here the operations include insertion, deletion, and substitution.

For example, given hypothesis string A = “kitten” and reference B = “sitting”, the edit distance is 3 for A will be transformed into B at least after two substitutions and one insertion:

“kitten” -> “sitten” -> “sittin” -> “sitting”

The input is a LoDTensor/Tensor consisting of all the hypothesis strings with the total number denoted by batch_size, and the separation is specified by the LoD information or input_length. And the batch_size reference strings are arranged in order in the same way as input.

The output contains the batch_size results and each stands for the edit distance for a pair of strings respectively. If Attr(normalized) is true, the edit distance will be divided by the length of reference string.

Parameters
  • input (Variable) – The indices for hypothesis strings, its rank should equals to 2 and its data type should be int64.

  • label (Variable) – The indices for reference strings, its rank should equals to 2 and its data type should be int64.

  • normalized (bool, default True) – Indicated whether to normalize the edit distance by the length of reference string.

  • ignored_tokens (list<int>, default None) – Tokens that should be removed before calculating edit distance.

  • input_length (Variable) – The length for each sequence in input if it’s of Tensor type, it should have shape [batch_size] and dtype int64.

  • label_length (Variable) – The length for each sequence in label if it’s of Tensor type, it should have shape [batch_size] and dtype int64.

Returns

edit_distance_out(Variable): edit distance result in shape [batch_size, 1]. sequence_num(Variable): sequence number in shape [].

Return type

Tuple

Examples

import paddle.fluid as fluid

# using LoDTensor
x_lod = fluid.data(name='x_lod', shape=[None,1], dtype='int64', lod_level=1)
y_lod = fluid.data(name='y_lod', shape=[None,1], dtype='int64', lod_level=1)
distance_lod, seq_num_lod = fluid.layers.edit_distance(input=x_lod, label=y_lod)

# using Tensor
x_seq_len = 5
y_seq_len = 6
x_pad = fluid.data(name='x_pad', shape=[None,x_seq_len], dtype='int64')
y_pad = fluid.data(name='y_pad', shape=[None,y_seq_len], dtype='int64')
x_len = fluid.data(name='x_len', shape=[None], dtype='int64')
y_len = fluid.data(name='y_len', shape=[None], dtype='int64')
distance_pad, seq_num_pad = fluid.layers.edit_distance(input=x_pad, label=y_pad, input_length=x_len, label_length=y_len)

elementwise_add

paddle.fluid.layers.elementwise_add(x, y, axis=-1, act=None, name=None)[source]

Elementwise Add Operator.

Add two tensors element-wise

The equation is:

$$Out = X + Y$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([2, 3, 4]).astype('float32'),
        "y": np.array([1, 5, 2]).astype('float32')
    }

x = fluid.data(name="x", shape=[3], dtype='float32')
y = fluid.data(name="y", shape=[3], dtype='float32')
z = fluid.layers.elementwise_add(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[3., 8., 6.]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.ones((2, 3, 4, 5)).astype('float32'),
        "y": np.zeros((3, 4)).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[3,4], dtype='float32')
z = fluid.layers.elementwise_add(x, y, axis=1)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'),
        "y": np.random.randint(1, 5, size=[5]).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[5], dtype='float32')
z = fluid.layers.elementwise_add(x, y, axis=3)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])
print(z_value) # z.shape=[2,3,4,5]

elementwise_div

paddle.fluid.layers.elementwise_div(x, y, axis=-1, act=None, name=None)[source]

Elementwise Div Operator.

Divide two tensors element-wise

The equation is:

$$Out = X / Y$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([2, 3, 4]).astype('float32'),
        "y": np.array([1, 5, 2]).astype('float32')
    }

x = fluid.data(name="x", shape=[3], dtype='float32')
y = fluid.data(name="y", shape=[3], dtype='float32')
z = fluid.layers.elementwise_div(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[2., 0.6, 2.]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.ones((2, 3, 4, 5)).astype('float32'),
        "y": np.zeros((3, 4)).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[3,4], dtype='float32')
z = fluid.layers.elementwise_div(x, y, axis=1)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'),
        "y": np.random.randint(1, 5, size=[5]).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[5], dtype='float32')
z = fluid.layers.elementwise_div(x, y, axis=3)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])
print(z_value) # z.shape=[2,3,4,5]

elementwise_floordiv

paddle.fluid.layers.elementwise_floordiv(x, y, axis=-1, act=None, name=None)[source]

Elementwise FloorDiv Operator.

Floor divide two tensors element-wise

The equation is:

$$Out = X // Y$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.

  • y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([10, 15, 8]).astype('int32'),
        "y": np.array([3, 7, 5]).astype('int32')
    }

x = fluid.data(name="x", shape=[3], dtype='int32')
y = fluid.data(name="y", shape=[3], dtype='int32')
z = fluid.layers.elementwise_floordiv(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[3, 2, 1]

elementwise_max

paddle.fluid.layers.elementwise_max(x, y, axis=-1, act=None, name=None)[source]

Elementwise Max Operator.

Compare two tensors and returns a new tensor containing the element-wise maxima.

The equation is:

$$Out = max(X, Y)$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), The first tensor holding the elements to be compared.

  • y – (Variable), The second tensor holding the elements to be compared.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([2, 3, 4]).astype('float32'),
        "y": np.array([1, 5, 2]).astype('float32')
    }

x = fluid.data(name="x", shape=[3], dtype='float32')
y = fluid.data(name="y", shape=[3], dtype='float32')
z = fluid.layers.elementwise_max(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[2, 5, 4]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.ones((2, 3, 4, 5)).astype('float32'),
        "y": np.zeros((3, 4)).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[3,4], dtype='float32')
z = fluid.layers.elementwise_max(x, y, axis=1)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value)#[[[[1., 1., 1., 1., 1.] .... [1., 1., 1., 1., 1.]]]]

elementwise_min

paddle.fluid.layers.elementwise_min(x, y, axis=-1, act=None, name=None)[source]

Elementwise Min Operator.

Compare two tensors and returns a new tensor containing the element-wise minima.

The equation is:

$$Out = min(X, Y)$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), The first tensor holding the elements to be compared.

  • y – (Variable), The second tensor holding the elements to be compared.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([2, 3, 4]).astype('float32'),
        "y": np.array([1, 5, 2]).astype('float32')
    }

x = fluid.data(name="x", shape=[3], dtype='float32')
y = fluid.data(name="y", shape=[3], dtype='float32')
z = fluid.layers.elementwise_max(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[1, 3, 2]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.ones((2, 3, 4, 5)).astype('float32'),
        "y": np.zeros((3, 4)).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[3,4], dtype='float32')
z = fluid.layers.elementwise_max(x, y, axis=1)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value)#[[[[0., 0., 0., 0., 0.] .... [0., 0., 0., 0., 0.]]]]

elementwise_mod

paddle.fluid.layers.elementwise_mod(x, y, axis=-1, act=None, name=None)[source]

Elementwise Mod Operator.

Mod two tensors element-wise

The equation is:

$$Out = X \% Y$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.

  • y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([10, 15, 8]).astype('int32'),
        "y": np.array([3, 6, 5]).astype('int32')
    }

x = fluid.data(name="x", shape=[3], dtype='int32')
y = fluid.data(name="y", shape=[3], dtype='int32')
z = fluid.layers.elementwise_mod(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[1, 3, 3]

elementwise_mul

paddle.fluid.layers.elementwise_mul(x, y, axis=-1, act=None, name=None)[source]

Elementwise Mul Operator.

Multiply two tensors element-wise

The equation is:

$$Out = X \odot Y$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([2, 3, 4]).astype('float32'),
        "y": np.array([1, 5, 2]).astype('float32')
    }

x = fluid.data(name="x", shape=[3], dtype='float32')
y = fluid.data(name="y", shape=[3], dtype='float32')
z = fluid.layers.elementwise_mul(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[2., 15., 8.]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.ones((2, 3, 4, 5)).astype('float32'),
        "y": np.zeros((3, 4)).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[3,4], dtype='float32')
z = fluid.layers.elementwise_mul(x, y, axis=1)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'),
        "y": np.random.randint(1, 5, size=[5]).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[5], dtype='float32')
z = fluid.layers.elementwise_mul(x, y, axis=3)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])
print(z_value) # z.shape=[2,3,4,5]

elementwise_pow

paddle.fluid.layers.elementwise_pow(x, y, axis=-1, act=None, name=None)[source]

Elementwise Pow Operator.

First tensor elements raised to powers from the second tensor, element-wise.

The equation is:

$$Out = X ^ Y$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), The Base.

  • y – (Variable), The exponents.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([2, 3, 4]).astype('float32'),
        "y": np.array([1, 5, 2]).astype('float32')
    }

x = fluid.data(name="x", shape=[3], dtype='float32')
y = fluid.data(name="y", shape=[3], dtype='float32')
z = fluid.layers.elementwise_pow(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[2, 243, 16]

elementwise_sub

paddle.fluid.layers.elementwise_sub(x, y, axis=-1, act=None, name=None)[source]

Elementwise Sub Operator.

Substract two tensors element-wise

The equation is:

$$Out = X - Y$$

  • \(X\): a tensor of any dimension.

  • \(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).

There are two cases for this operator:

  1. The shape of \(Y\) is the same with \(X\).

  2. The shape of \(Y\) is a continuous subsequence of \(X\).

For case 2:

  1. Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).

  2. If \(axis\) is -1 (default), \(axis = rank(X) - rank(Y)\).

  3. The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).

For example:

shape(X) = (2, 3, 4, 5), shape(Y) = (,)
shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=-1(default) or axis=2
shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
Parameters
  • x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.

  • axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.

  • act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function

  • name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name

Returns

N-dimension tensor. A location into which the result is stored. It’s dimension equals with x

Examples

import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.array([2, 3, 4]).astype('float32'),
        "y": np.array([1, 5, 2]).astype('float32')
    }

x = fluid.data(name="x", shape=[3], dtype='float32')
y = fluid.data(name="y", shape=[3], dtype='float32')
z = fluid.layers.elementwise_sub(x, y)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) #[1., -2., 2.]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.ones((2, 3, 4, 5)).astype('float32'),
        "y": np.zeros((3, 4)).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[3,4], dtype='float32')
z = fluid.layers.elementwise_sub(x, y, axis=1)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])

print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid
import numpy as np

def gen_data():
    return {
        "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'),
        "y": np.random.randint(1, 5, size=[5]).astype('float32')
    }

x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32')
y = fluid.data(name="y", shape=[5], dtype='float32')
z = fluid.layers.elementwise_sub(x, y, axis=3)

place = fluid.CPUPlace()
exe = fluid.Executor(place)

z_value = exe.run(feed=gen_data(),
                    fetch_list=[z.name])
print(z_value) # z.shape=[2,3,4,5]

elu

paddle.fluid.layers.elu(x, alpha=1.0, name=None)[source]

ELU Activation Operator.

Applies the following element-wise computation on the input according to https://arxiv.org/abs/1511.07289.

\(out = \max(0, x) + \min(0, \alpha * (e^x - 1))\)

Parameters
  • x (Variable) – The input is a multi-dimensional Tensor. The data type is float32 or float64

  • alpha (FLOAT|1.0) – The alpha value of ELU

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.

Returns

The output is a multi-dimensional Tensor which has same dimension and data type as the x

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

input_elu = np.array([[-1,6],[1,15.6]])
with fluid.dygraph.guard():
    x = fluid.dygraph.to_variable(input_elu)
    y = fluid.layers.elu(x, alpha=0.2)
    print(y.numpy())
    # [[-0.12642411  6.        ]
    # [ 1.          15.6       ]]

embedding

paddle.fluid.layers.embedding(input, size, is_sparse=False, is_distributed=False, padding_idx=None, param_attr=None, dtype='float32')[source]

WARING: This OP will be deprecated in a future release. This OP requires the last dimension of Tensor shape must be equal to 1. It is recommended to use fluid. api_fluid_embedding .

The operator is used to lookup embeddings vector of ids provided by input . It automatically constructs a 2D embedding matrix based on the input size (vocab_size, emb_size) and dtype .

This OP requires the last dimension of Tensor shape must be equal to 1. The shape of output Tensor is generated by replacing the last dimension of the input Tensor shape with emb_size.

Note: The id in input must satisfy \(0 =< id < size[0]\) , otherwise the program will throw an exception and exit.

Case 1:

input is a Tensor. padding_idx = -1
    input.data = [[[1], [3]], [[2], [4]], [[4], [127]]]
    input.shape = [3, 2, 1]
Given size = [128, 16]
output is a Tensor:
    out.shape = [3, 2, 16]
    out.data = [[[0.129435295, 0.244512452, ..., 0.436322452],
                [0.345421456, 0.524563927, ..., 0.144534654]],

                [[0.345249859, 0.124939536, ..., 0.194353745],
                [0.945345345, 0.435394634, ..., 0.435345365]],

                [[0.945345345, 0.435394634, ..., 0.435345365],
                [0.0,         0.0,         ..., 0.0        ]]]  # padding data
The input padding_idx is less than 0, it is automatically converted to padding_idx = -1 + 128 = 127
It will pad all-zero data when ids is 127.

Case 2:

input is a LoDTensor with 1-level LoD. padding_idx = 0
    input.lod = [[2, 3]]
    input.data = [[1], [3], [2], [4], [0]]
    input.shape = [5, 1]
Given size = [128, 16]
output is a LoDTensor:
    out.lod = [[2, 3]]
    out.shape = [5, 16]
    out.data = [[0.129435295, 0.244512452, ..., 0.436322452],
                [0.345421456, 0.524563927, ..., 0.144534654],
                [0.345249859, 0.124939536, ..., 0.194353745],
                [0.945345345, 0.435394634, ..., 0.435345365],
                [0.0,         0.0,         ..., 0.0        ]]  # padding data
It will pad all-zero data when ids is 0.
Parameters
  • input (Variable) – A Tensor or LoDTensor with type int64, which contains the id information. The last dimension of Tensor shape must be equal to 1. The value of the input id should satisfy \(0<= id < size[0]\) .

  • size (tuple|list) – The shape of lookup table parameter. It should have two elements which indicates the size of the dictionary of embeddings and the size of each embedding vector respectively.

  • is_sparse (bool) – The flag indicating whether to use sparse update. This parameter only affects the performance of the backwards gradient update. It is recommended to set True because sparse update is faster. But some optimizer does not support sparse update, such as api_fluid_optimizer_AdadeltaOptimizer , AdamaxOptimizer , DecayedAdagradOptimizer , FtrlOptimizer , LambOptimizer and LarsMomentumOptimizer . In these case, is_sparse must be False. Default: False.

  • is_distributed (bool) – Whether to store the embedding matrix in a distributed manner. Only used in multi-machine distributed CPU training. Default: False.

  • padding_idx (int|long|None) – padding_idx needs to be in the interval [-vocab_size, vocab_size). If \(padding\_idx < 0\), the \(padding\_idx\) will automatically be converted to \(vocab\_size + padding\_idx\) . It will output all-zero padding data whenever lookup encounters \(padding\_idx\) in id. And the padding data will not be updated while training. If set None, it makes no effect to output. Default: None.

  • param_attr (ParamAttr) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr . In addition, user-defined or pre-trained word vectors can be loaded with the param_attr parameter. The local word vector needs to be transformed into numpy format, and the shape of local word vector shoud be consistent with size . Then NumpyArrayInitializer is used to load custom or pre-trained word vectors. See code example 2 for details.

  • dtype (str|core.VarDesc.VarType) – It refers to the data type of output Tensor. It must be float32 or float64. Default: float32.

Returns

Embedding Tensor or LoDTensor mapped by input. The data type is the same as dtype .

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np
data = fluid.data(name='x', shape=[None, 1], dtype='int64')

# exampel 1
emb_1 = fluid.embedding(input=data, size=[128, 64])

# example 2: load custom or pre-trained word vectors
weight_data = np.random.random(size=(128, 100))  # word vectors with numpy format
w_param_attrs = fluid.ParamAttr(
    name="emb_weight",
    learning_rate=0.5,
    initializer=fluid.initializer.NumpyArrayInitializer(weight_data),
    trainable=True)
emb_2 = fluid.layers.embedding(input=data, size=(128, 100), param_attr=w_param_attrs, dtype='float32')

expand

paddle.fluid.layers.expand(x, expand_times, name=None)[source]

This operation tiles x multiple times according to the parameter expand_times. The times number for each dimension of x is set by the parameter expand_times. The rank of x should be less than or equal to 6. Please note that size of expand_times must be the same with X’s rank. Following is a using case:

Input(X) is a 3-D tensor with shape [2, 3, 1]:

        [
           [[1], [2], [3]],
           [[4], [5], [6]]
        ]

Attr(expand_times):  [1, 2, 2]

Output(Out) is a 3-D tensor with shape [2, 6, 2]:

        [
            [[1, 1], [2, 2], [3, 3], [1, 1], [2, 2], [3, 3]],
            [[4, 4], [5, 5], [6, 6], [4, 4], [5, 5], [6, 6]]
        ]
Parameters
  • x (Variable) – A Tensor or LoDTensor with dimension in [1, 6]. The data type is bool, float32, float64 or int32 .

  • expand_times (list|tuple|Variable) – The data type is int32 . If expand_times is a list or tuple, the elements of it should be integers or Tensors with shape [1]. If expand_times is an Variable, it should be an 1-D Tensor. Expand times number for each dimension of x .

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

A Tensor or LoDTensor. The data type is same as x. After expanding, size of each dimension of output is equal to the size of the corresponding dimension of x multiplying the corresponding value given by expand_times .

Return type

Variable

Raises
  • TypeError – The type of expand_times must be list, tuple or Variable.

  • ValueError – The elements of expand_times cannot be negative.

Examples

import paddle.fluid as fluid

# example 1:
data_1 = fluid.layers.fill_constant(shape=[2, 3, 1], dtype='int32', value=0)
expanded_1 = fluid.layers.expand(data_1, expand_times=[1, 2, 2])
# the shape of expanded_1 is [2, 6, 2].

# example 2:
data_2 = fluid.layers.fill_constant(shape=[12, 14], dtype="int32", value=3)
expand_times = fluid.layers.fill_constant(shape=[2], dtype="int32", value=4)
expanded_2 = fluid.layers.expand(data_2, expand_times=expand_times)
# the shape of expanded_2 is [48, 56].

fc

paddle.fluid.layers.fc(input, size, num_flatten_dims=1, param_attr=None, bias_attr=None, act=None, name=None)[source]

Fully Connected Layer

This operator creates a fully connected layer in the network. It can take a Tensor(or LoDTensor) or a list of Tensor(or LoDTensor) as its inputs(see Args in detail). It creates a variable called weight for each input Tensor, which represents a fully connected weight matrix from each input unit to each output unit. The fully connected layer multiplies each input Tensor with its corresponding weight to produce an output Tensor with shape \([M, size]\) , where M is batch size. If a list of Tensor is given, the results of multiple output Tensors with shape \([M, size]\) will be summed up. If bias_attr is not None, a bias variable will be created and added to the output. Finally, if act is not None, it will be applied to the output as well.

When the input is a single Tensor(or LoDTensor):

\[Out = Act({XW + b})\]

When the input is a list of Tensor(or LoDTensor):

\[Out = Act({\sum_{i=0}^{N-1}X_iW_i + b})\]

In the above equation:

  • \(N\): Number of the input. N equals to len(input) if input is list of Variable.

  • \(X_i\): The i-th input tensor.

  • \(W_i\): The i-th weights matrix corresponding i-th input tensor.

  • \(b\): The bias parameter created by this layer (if needed).

  • \(Act\): The activation function.

  • \(Out\): The output Tensor.

Case 1:
Given a single Tensor data_1, and num_flatten_dims = 2:
    data_1.data = [[[0.1, 0.2],
                    [0.3, 0.4]]]
    data_1.shape = (1, 2, 2) # 1 is batch_size

    out = fluid.layers.fc(input=data_1, size=1, num_flatten_dims=2)

Then output is:
    out.data = [[0.83234344], [0.34936576]]
    out.shape = (1, 2, 1)

Case 2:
Given a list of Tensor:
    data_1.data = [[[0.1, 0.2],
                   [0.3, 0.4]]]
    data_1.shape = (1, 2, 2) # 1 is batch_size

    data_2 = [[[0.1, 0.2, 0.3]]]
    data_2.shape = (1, 1, 3)

    out = fluid.layers.fc(input=[data_1, data_2], size=2)

Then:
    out.data = [[0.18669507, 0.1893476]]
    out.shape = (1, 2)
Parameters
  • input (Variable|list of Variable) – A Tensor(or LoDTensor) with shape \([N_1, N_2,..., N_k]\) or a list of Tensor(or LoDTensor). The dimensions of the input Tensor is at least 2 and the data type should be float32 or float64.

  • size (int) – The number of output units in this layer, which also means the feature size of ouput Tensor(or LoDTensor).

  • num_flatten_dims (int) – The fc layer can accept an input Tensor with more than two dimensions. If this happens, the multidimensional tensor will first be flattened into a 2-D matrix. The parameter num_flatten_dims determines how the input Tensor is flattened: the first num_flatten_dims (inclusive, index starts from 1) dimensions will be flatten to form the first dimension of the final matrix (height of the matrix), and the rest \(rank(X) - num\_flatten\_dims\) dimensions are flattened to form the second dimension of the final matrix (width of the matrix). For example, assuming that X is a 5-dimensional Tensor with a shape [2, 3, 4, 5, 6], and num_flatten_dims = 3. Then, the flattened matrix will have a shape [2 x 3 x 4, 5 x 6] = [24, 30]. Default: 1.

  • param_attr (ParamAttr) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • bias_attr (ParamAttr) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .

  • act (str) – Activation to be applied to the output of this layer, such as tanh, softmax, sigmoid, relu. For more information, please refer to Activation Function . Default: None.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

Tensor or LoDTensor calculated by fc layer. The data type is same with input.

Return type

Variable

Raises

ValueError – If dimensions of the input Tensor is less than 2.

Examples

import paddle.fluid as fluid
# when input is single tensor
data = fluid.data(name="data", shape=[-1, 32], dtype="float32")
fc = fluid.layers.fc(input=data, size=1000, act="tanh")

# when input are multiple tensors
data_1 = fluid.data(name="data_1", shape=[-1, 32], dtype="float32")
data_2 = fluid.data(name="data_2", shape=[-1, 36], dtype="float32")
fc = fluid.layers.fc(input=[data_1, data_2], size=1000, act="tanh")

flatten

paddle.fluid.layers.flatten(x, axis=1, name=None)[source]

Flatten op

Flatten the input tensor into a 2D matrix.

For Example:

Case 1:

  Given
    X.shape = (3, 100, 100, 4)

  and
    axis = 2

  We get:
    Out.shape = (3 * 100, 4 * 100)

Case 2:

  Given
    X.shape = (3, 100, 100, 4)

  and
    axis = 0

  We get:
    Out.shape = (1, 3 * 100 * 100 * 4)
Parameters
  • x (Variable) – A tensor of rank >= axis. A tensor with type float32, float64, int8, int32, int64.

  • axis (int) – Indicate up to which input dimensions (exclusive) should be flattened to the outer dimension of the output. The value for axis must be in the range [0, R], where R is the rank of the input tensor. Default: 1.

  • name (str, Optional) – For details, please refer to api_guide_Name. Generally, no setting is required. Default: None.

Returns

A 2D tensor with the contents of the input tensor, with input dimensions up to axis flattened to the outer dimension of the output and remaining input dimensions flattened into the inner dimension of the output. A Tensor with type same as input x.

Return type

Variable

Raises
  • ValueError – If x is not a variable.

  • ValueError – If axis is not in range [0, rank(x)].

Examples

import paddle.fluid as fluid
x = fluid.data(name="x", shape=[4, 4, 3], dtype="float32")
# x shape is [4, 4, 3]
out = fluid.layers.flatten(x=x, axis=2)
# out shape is [16, 3]

fsp_matrix

paddle.fluid.layers.fsp_matrix(x, y)[source]

FSP matrix op

This op is used to calculate the flow of solution procedure (FSP) matrix of two 4-D Tensor feature maps. Given feature map x with shape [x_channel, h, w] and feature map y with shape [y_channel, h, w], we can get the fsp matrix of x and y in two steps:

  1. reshape x into matrix with shape [x_channel, h * w] and reshape and transpose y into matrix with shape [h * w, y_channel].

  2. multiply x and y to get fsp matrix with shape [x_channel, y_channel].

The output is a batch of fsp matrices.

Parameters
  • x (Variable) – A 4-D Tensor feature map with shape [batch_size, x_channel, height, width]. A Tensor with type float32, float64.

  • y (Variable) – A 4-D Tensor feature map with shape [batch_size, y_channel, height, width]. The y_channel can be different with the x_channel of Input(X) while the other dimensions must be the same with Input(X)’s. A Tensor with type float32, float64.

Returns

The output of FSP op with shape [batch_size, x_channel, y_channel]. The x_channel is the channel of x and the y_channel is the channel of y. A Tensor with type float32, float64.

Return type

fsp matrix (Variable)

Examples

import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 32, 32])
feature_map_0 = fluid.layers.conv2d(data, num_filters=2,
                                    filter_size=3)
feature_map_1 = fluid.layers.conv2d(feature_map_0, num_filters=2,
                                    filter_size=1)
loss = fluid.layers.fsp_matrix(feature_map_0, feature_map_1)

gather

paddle.fluid.layers.gather(input, index, overwrite=True)[source]

Gather Layer

Output is obtained by gathering entries of the outer-most dimension of X indexed by index and concatenate them together.

\[Out = X[Index]\]
Given:

X = [[1, 2],
     [3, 4],
     [5, 6]]

Index = [1, 2]

Then:

Out = [[3, 4],
       [5, 6]]
Parameters
  • input (Variable) – The source input tensor with rank>=1. Supported data type is int32, int64, float32, float64 and uint8 (only for CPU), float16 (only for GPU).

  • index (Variable) – The index input tensor with rank=1. Data type is int32 or int64.

  • overwrite (bool, optional) – The mode that updating the grad when has same index. If True, use the overwrite mode to update the grad of the same index, if False, use the accumulate mode to update the grad of the same index. Default value is True.

Returns

The output is a tensor with the same rank as input.

Return type

output (Variable)

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[-1, 5], dtype='float32')
index = fluid.data(name='index', shape=[-1, 1], dtype='int32')
output = fluid.layers.gather(x, index)

gaussian_random

paddle.fluid.layers.gaussian_random(shape, mean=0.0, std=1.0, seed=0, dtype='float32')[source]

Generate a random tensor whose data is drawn from a Gaussian distribution.

Parameters
  • shape (Tuple[int] | List[int]) – Shape of the generated random tensor.

  • mean (float) – Mean of the random tensor, defaults to 0.0.

  • std (float) – Standard deviation of the random tensor, defaults to 1.0.

  • seed (int) – (int, default 0) Random seed of generator.0 means use system wide seed.Note that if seed is not 0, this operator will always generate the same random numbers every time

  • dtype (np.dtype | core.VarDesc.VarType | str) – Output data type, float32 or float64.

Returns

Random tensor whose data is drawn from a Gaussian distribution, dtype: flaot32 or float64 as specified.

Return type

Variable

Examples

# declarative mode
import numpy as np
from paddle import fluid

x = fluid.layers.gaussian_random((2, 3), std=2., seed=10)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
start = fluid.default_startup_program()
main = fluid.default_main_program()

exe.run(start)
x_np, = exe.run(main, feed={}, fetch_list=[x])

x_np
# array([[2.3060477, 2.676496 , 3.9911983],
#        [0.9990833, 2.8675377, 2.2279181]], dtype=float32)
# imperative mode
import numpy as np
from paddle import fluid
import paddle.fluid.dygraph as dg

place = fluid.CPUPlace()
with dg.guard(place) as g:
    x = fluid.layers.gaussian_random((2, 4), mean=2., dtype="float32", seed=10)
    x_np = x.numpy()
x_np
# array([[2.3060477 , 2.676496  , 3.9911983 , 0.9990833 ],
#        [2.8675377 , 2.2279181 , 0.79029655, 2.8447366 ]], dtype=float32)

gaussian_random_batch_size_like

paddle.fluid.layers.gaussian_random_batch_size_like(input, shape, input_dim_idx=0, output_dim_idx=0, mean=0.0, std=1.0, seed=0, dtype='float32')[source]

Used to initialize tensors with gaussian random generator. The default mean of the distribution is 0, and default standard deviation (std) of the distribution is 1.0. Uers can set mean and std via input arguments.

Parameters
  • input (Variable) – Tensor whose input_dim_idx’th dimension specifies the batch_size

  • shape (tuple|list) – The shape of the output

  • input_dim_idx (int) – default 0. The index of input’s batch size dimension

  • output_dim_idx (int) – default 0. The index of output’s batch size dimension

  • mean (float) – (float, default 0.0) The mean (or center) of the gaussian distribution

  • std (float) – (float, default 1.0) The standard deviation (std, or spread) of the gaussian distribution

  • seed (int) – (int, default 0) Random seed of generator.0 means don’t specify random seed.Note that if seed is not 0, this operator will always generate the same random numbers every time

  • dtype (np.dtype|core.VarDesc.VarType|str) – The type of output data, float32 or float_64.

Returns

Tensor of specified shape will be filled with the specified value

Return type

out (Variable)

Examples

import paddle.fluid as fluid
input = fluid.data(name="input", shape=[13, 11], dtype='float32')

out = fluid.layers.gaussian_random_batch_size_like(
    input, shape=[-1, 11], mean=1.0, std=2.0)

get_tensor_from_selected_rows

paddle.fluid.layers.get_tensor_from_selected_rows(x, name=None)[source]

This operator gets tensor data from input with SelectedRows type, and outputs a LoDTensor.

input x is SelectedRows:
   x.rows = [0, 5, 5, 4, 19]
   x.height = 20
   x.value = [[1, 1] [2, 2] [2, 2] [3, 3] [6, 6]]

Ouput is LoDTensor:
   out.shape = [5, 2]
   out.data = [[1, 1],
               [2, 2],
               [2, 2],
               [3, 3],
               [6, 6]]
Parameters
  • x (SelectedRows) – Input with SelectedRows type. The data type is float32, float64, int32 or int64.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

LoDTensor transformed from SelectedRows. The data type is same with input.

Return type

Variable

Examples

import paddle.fluid as fluid
b = fluid.default_main_program().global_block()
input = b.create_var(name="X", dtype="float32", persistable=True, type=fluid.core.VarDesc.VarType.SELECTED_ROWS)
out = fluid.layers.get_tensor_from_selected_rows(input)

grid_sampler

paddle.fluid.layers.grid_sampler(x, grid, name=None)[source]

This operation samples input X by using bilinear interpolation based on flow field grid, which is usually gennerated by affine_grid . The grid of shape [N, H, W, 2] is the concatenation of (x, y) coordinates with shape [N, H, W] each, where x is indexing the 4th dimension (in width dimension) of input data x and y is indexng the 3rd dimention (in height dimension), finally results is the bilinear interpolation value of 4 nearest corner points. The output tensor shape will be [N, C, H, W].

Step 1:
Get (x, y) grid coordinates and scale to [0, H-1/W-1].

.. code-block:: text

    grid_x = 0.5 * (grid[:, :, :, 0] + 1) * (W - 1)
    grid_y = 0.5 * (grid[:, :, :, 1] + 1) * (H - 1)

Step 2:
Indices input data X with grid (x, y) in each [H, W] area, and bilinear
interpolate point value by 4 nearest points.

  wn ------- y_n ------- en
  |           |           |
  |          d_n          |
  |           |           |
 x_w --d_w-- grid--d_e-- x_e
  |           |           |
  |          d_s          |
  |           |           |
  ws ------- y_s ------- wn

x_w = floor(x)              // west side x coord
x_e = x_w + 1               // east side x coord
y_n = floor(y)              // north side y coord
y_s = y_s + 1               // south side y coord

d_w = grid_x - x_w          // distance to west side
d_e = x_e - grid_x          // distance to east side
d_n = grid_y - y_n          // distance to north side
d_s = y_s - grid_y          // distance to south side

wn = X[:, :, y_n, x_w]      // north-west point value
en = X[:, :, y_n, x_e]      // north-east point value
ws = X[:, :, y_s, x_w]      // south-east point value
es = X[:, :, y_s, x_w]      // north-east point value

output = wn * d_e * d_s + en * d_w * d_s
       + ws * d_e * d_n + es * d_w * d_n
Parameters
  • x (Variable) – The input tensor, which is a 4-D tensor with shape [N, C, H, W], N is the batch size, C is the channel number, H and W is the feature height and width. The data type is float32 or float64.

  • grid (Variable) – Input grid tensor of shape [N, H, W, 2]. The data type is float32 or float64.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

Output of shape [N, C, H, W] data samples input X

using bilnear interpolation based on input grid. The data type is same as input tensor.

Return type

Variable

Examples

import paddle.fluid as fluid

# use with affine_grid
x = fluid.data(name='x', shape=[None, 10, 32, 32], dtype='float32')
theta = fluid.layers.data(name='theta', shape=[2, 3], dtype='float32')
grid = fluid.layers.affine_grid(theta=theta, out_shape=[3, 10, 32, 32])
out = fluid.layers.grid_sampler(x=x, grid=grid)

group_norm

paddle.fluid.layers.group_norm(input, groups, epsilon=1e-05, param_attr=None, bias_attr=None, act=None, data_layout='NCHW', name=None)[source]

Group Normalization Layer

Refer to Group Normalization .

Parameters
  • input (Variable) – 4-D Tensor, the data type is float32 or float64.

  • groups (int) – The number of groups that divided from channels, the data type is int32.

  • epsilon (float, optional) – The small value added to the variance to prevent division by zero, the data type is float32. Default: 1e-05.

  • param_attr (ParamAttr|bool, optional) – ParamAttr object that specifies weight parameter attribute. If a bool type, only False is supported, which means there is no weight parameter. Default: None, the default weight parameter attribute is used. For more information, please refer to api_guide_ParamAttr .

  • bias_attr (ParamAttr|bool, optional) – ParamAttr object that specifies bias parameter attribute. If a bool type, only False is supported, which means there is no bias parameter. Default: None, the default bias parameter attribute is used. For more information, please refer to api_guide_ParamAttr .

  • act (str, optional) – Activation to be applied to the output of group normalizaiton.

  • data_layout (str, optional) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. When it is “NCHW”, the data is stored in the order of: [batch_size, channels, height, width]. Default: “NCHW”.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

A 4-D Tensor has same data type and data format with input.

Return type

Variable

Raises

ValueError – If data_layout is neither ‘NCHW’ nor ‘NHWC’.

Examples

import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 8, 32, 32], dtype='float32')
x = fluid.layers.group_norm(input=data, groups=4)

gru_unit

paddle.fluid.layers.gru_unit(input, hidden, size, param_attr=None, bias_attr=None, activation='tanh', gate_activation='sigmoid', origin_mode=False)[source]

Gated Recurrent Unit (GRU) RNN cell. This operator performs GRU calculations for one time step and it supports these two modes:

If origin_mode is True, then the formula used is from paper Learning Phrase Representations using RNN Encoder Decoder for Statistical Machine Translation .

\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t-1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t-1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t-1}) + b_c)\\h_t & = u_t \odot h_{t-1} + (1-u_t) \odot \tilde{h_t}\end{aligned}\end{align} \]

if origin_mode is False, then the formula used is from paper Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t-1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t-1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t-1}) + b_c)\\h_t & = (1-u_t) \odot h_{t-1} + u_t \odot \tilde{h_t}\end{aligned}\end{align} \]

\(x_t\) is the input of current time step, but it is not input . This operator does not include the calculations \(W_{ux}x_{t}, W_{rx}x_{t}, W_{cx}x_{t}\) , Note thus a fully-connect layer whose size is 3 times of GRU hidden size should be used before this operator, and the output should be used as input here. \(h_{t-1}\) is the hidden state from previous time step. \(u_t\) , \(r_t\) , \(\tilde{h_t}\) and \(h_t\) stand for update gate, reset gate, candidate hidden and hidden output separately. \(W_{uh}, b_u\) , \(W_{rh}, b_r\) and \(W_{ch}, b_c\) stand for the weight matrix and bias used in update gate, reset gate, candidate hidden calculations. For implementation, the three weight matrix are merged into a tensor shaped \([D, D \times 3]\) , the three bias are concatenated as a tensor shaped \([1, D \times 3]\) , where \(D\) stands for the hidden size; The data layout of weight tensor is: \(W_{uh}\) and \(W_{rh}\) are concatenated with shape \([D, D \times 2]\) lying on the first part, and \(W_{ch}\) lying on the latter part with shape \([D, D]\) .

Parameters
  • input (Variable) – A 2D Tensor representing the input after linear projection after linear projection. Its shape should be \([N, D \times 3]\) , where \(N\) stands for batch size, \(D\) for the hidden size. The data type should be float32 or float64.

  • hidden (Variable) – A 2D Tensor representing the hidden state from previous step. Its shape should be \([N, D]\) , where \(N\) stands for batch size, \(D\) for the hidden size. The data type should be same as input .

  • size (int) – Indicate the hidden size.

  • param_attr (ParamAttr, optional) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • bias_attr (ParamAttr, optional) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .

  • activation (str, optional) – The activation fuction corresponding to \(act_c\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “tanh”.

  • gate_activation (str, optional) – The activation fuction corresponding to \(act_g\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “sigmoid”.

Returns

The tuple contains three Tensor variables with the same data type as input . They represent the hidden state for next time step ( \(h_t\) ), reseted previous hidden state ( \(r_t \odot h_{t-1}\) ), and the concatenation of \(h_t, r_t, \tilde{h_t}\) . And they have shape \([N, D]\) , \([N, D]\) , \([N, D imes 3]\) separately. Usually only the hidden state for next time step ( \(h_t\) ) is used as output and state, the other two are intermediate results of calculations.

Return type

tuple

Examples

import paddle.fluid as fluid

dict_dim, emb_dim = 128, 64
data = fluid.data(name='step_data', shape=[None], dtype='int64')
emb = fluid.embedding(input=data, size=[dict_dim, emb_dim])
hidden_dim = 512
x = fluid.layers.fc(input=emb, size=hidden_dim * 3)
pre_hidden = fluid.data(
    name='pre_hidden', shape=[None, hidden_dim], dtype='float32')
hidden = fluid.layers.gru_unit(
    input=x, hidden=pre_hidden, size=hidden_dim * 3)

hard_sigmoid

paddle.fluid.layers.hard_sigmoid(x, slope=0.2, offset=0.5, name=None)[source]

HardSigmoid Activation Operator.

A 3-part piecewise linear approximation of sigmoid(https://arxiv.org/abs/1603.00391), which is much faster than sigmoid.

\(out = \max(0, \min(1, slope * x + offset))\)

Parameters
  • x (Variable) – An N-D Tensor with data type float32, float64.

  • slope (float, optional) – The slope of the linear approximation of sigmoid. Its value MUST BE positive. Default is 0.2.

  • offset (float, optional) – The offset of the linear approximation of sigmoid. Default is 0.5.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

A Tensor with the same shape as input.

Return type

Variable

Examples

import paddle.fluid as fluid
data = fluid.layers.fill_constant(shape=[3, 2], value=0.5, dtype='float32') # [[0.5, 0.5], [0.5, 0.5], [0.5, 0.5]]
result = fluid.layers.hard_sigmoid(data) # [[0.6, 0.6], [0.6, 0.6], [0.6, 0.6]]

hash

paddle.fluid.layers.hash(input, hash_size, num_hash=1, name=None)[source]

This OP hash the input to an integer less than the hash_size. The hash algorithm we used was xxHash - Extremely fast hash algorithm (https://github.com/Cyan4973/xxHash/tree/v0.6.5)

Parameters
  • input (Variable) – A Two-Dimensional LoDTensor with type int32, int64. Only support LoDTensor.

  • num_hash (int, optional) – The times of hash, default is 1.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.

Returns

A LoDTensor with the same data type as input.

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

place = fluid.core.CPUPlace()

x = fluid.data(name="x", shape=[1], dtype="int32", lod_level=1)
res = fluid.layers.hash(name="res",input=x, hash_size=1000, num_hash=4)

exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
in1 = np.array([[1,2],[3,4]]).astype("int32")
print(in1)
x_i = fluid.core.LoDTensor()
x_i.set(in1,place)
x_i.set_recursive_sequence_lengths([[0,2]])
res = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res], return_numpy=False)
print(np.array(res[0]))
# [[[722]
#   [407]
#   [337]
#   [395]]
#  [[603]
#   [590]
#   [386]
#   [901]]]

hsigmoid

paddle.fluid.layers.hsigmoid(input, label, num_classes, param_attr=None, bias_attr=None, name=None, path_table=None, path_code=None, is_custom=False, is_sparse=False)[source]

The hierarchical sigmoid organizes the classes into a complete binary tree to reduce the computational complexity and speed up the model training, especially the training of language model. Each leaf node of the complete binary tree represents a class(word) and each non-leaf node acts as a binary classifier. For each class(word), there’s a unique path from root to itself, hsigmoid calculate the cost for each non-leaf node on the path, and sum them to get a total cost. Comparing to softmax, the OP can reduce the computational complexity from \(O(N)\) to \(O(logN)\), where \(N\) represents the number of classes or the size of word dict.

The OP supports default tree and custom tree. For the default tree, you can refer to Hierarchical Probabilistic Neural Network Language Model <http://www.iro.umontreal.ca/~lisa/pointeurs/hierarchical-nnlm-aistats05.pdf>. For the custom tree, you need to set is_custom to True, and do the following steps (take the language model as an example):

  1. Using a custom word dict to build a binary tree, each leaf node should be an word in the word dict.

  2. Creating a dict map word_id -> path that from the word to the root node, we call it path_table.

  3. Creating a dict map word_id -> code of path that from the word to the root node, we call it path_code. Code means the label of each binary classifier, 1 indicate true, 0 indicate false.

  4. Now, each word should has its path and code along the path, you can pass a batch of path and code related to the same batch of inputs.

Parameters
  • input (Variable) – A tensor with the shape [N, D], where N is the size of mini-batch, and D is the feature size. Its data type supports float32 and float64.

  • label (Variable) – A tensor contains the labels of training data. Its shape is [N, 1] and data type is int64.

  • num_classes (int) – The number of classes or the size of word dict, must be greater than 2. If the default tree is used (is_custom is set to False), num_classes should not be None. If the custom tree is used (is_custom is set to True), num_classes should be the number of non-leaf nodes, which indicates the num of classes using by the binary classifier.

  • param_attr (ParamAttr, optional) – The parameter attribute for the learnable parameters/weights of hsigmoid. If it is set to None or one attribute of ParamAttr, hsigmoid will create a ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.

  • bias_attr (ParamAttr|bool, optional) – The parameter attribute for the bias of hsigmoid. If it is set to False, no bias will be added. If it is set to None or one attribute of ParamAttr, hsigmoid will create a ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

  • name (str, optional) – Normally there is no need for user to set this property. For more information, please refer to api_guide_Name. Default: None.

  • path_table (Variable, optional) – A tensor that stores each batch of samples’ path from leaf to root node, its shape is [N, L] and data type is int64, where L is the length of path. For each sample i, path_table[i] is a np.array like structure and each element in this array is the indexes in parent nodes’ weight matrix. Default: None.

  • path_code (Variable, optional) – A tensor that stores each batch of samples’ code of path from leaf to root node, its shape is [N, L] and data type is int64, which is the same as path_table. Each code of path is consisted with the code of nodes from leaf to root node. Default: None.

  • is_custom (bool, optional) – Whether use custom binary tree. If it’s True, path_table, path_code and num_classes should be set, otherwise num_classes should be set. Default: False.

  • is_sparse (bool, optional) – Whether use sparse updating instead of dense updating, if it’s True, the gradient of W and input will be sparse. Default: False.

Returns

A tensor with the cost of hierarchical sigmoid, its shape is [N, 1] and data type is the same as input.

Return type

Variable

Examples

import paddle.fluid as fluid
x = fluid.layers.fill_constant(shape=[4, 3], value=0.9, dtype='float32')
# x = [[0.9, 0.9, 0.9], [0.9, 0.9, 0.9], [0.9, 0.9, 0.9], [0.9, 0.9, 0.9]]
y = fluid.layers.fill_constant(
    shape=[4, 1], value=1, dtype='int64')
# y = [[1], [1], [1], [1]]
out = fluid.layers.hsigmoid(input=x, label=y, num_classes=2, param_attr=fluid.initializer.Constant(
    value=0.05), bias_attr=fluid.initializer.Constant(value=.0))
# out = [[0.62792355], [0.62792355], [0.62792355], [0.62792355]]

huber_loss

paddle.fluid.layers.huber_loss(input, label, delta)[source]

This operator computes the Huber loss between input and label. Huber loss is commonly used in regression tasks. Compared to square_error_cost, Huber loss is more robust and less sensitivity to outliers.

When the absolute difference between input and label is greater than delta, the linear error is calculated:

\[huber\_loss = delta * (label - input) - 0.5 * delta * delta\]

When the absolute difference between input and label is greater than delta, the square error is calculated:

\[huber\_loss = 0.5 * (label - input) * (label - input)\]
Parameters
  • input (Variable) – Predicted data, 2D-Tensor with the shape of [batch_size, 1]. The data type should be float32 or float64.

  • label (Variable) – Ground truth label, 2D-Tensor with the shape of [batch_size, 1]. The data type should be float32 or float64.

  • delta (float) – The threshold for Huber loss, which is used to control the balance between the linear error and square error. The data type should be float32.

Returns

The huber loss, a tensor with the same shape and data type as input.

Return type

Variable

Examples:

import paddle.fluid as fluid
import numpy as np

DATATYPE='float32'
input_data = np.array([[1.],[2.],[3.],[4.]]).astype(DATATYPE)
label_data = np.array([[3.],[3.],[4.],[4.]]).astype(DATATYPE)

x = fluid.data(name='input', shape=[None, 1], dtype=DATATYPE)
y = fluid.data(name='label', shape=[None, 1], dtype=DATATYPE)
loss = fluid.layers.huber_loss(input=x, label=y, delta=1.0)

place = fluid.CPUPlace()
#place = fluid.CUDAPlace(0)
exe = fluid.Executor(place)
HuberLoss, = exe.run(feed={'input':input_data ,'label':label_data}, fetch_list=[loss.name])
print(HuberLoss)  #[[1.5], [0.5], [0.5], [0. ]], dtype=float32

im2sequence

paddle.fluid.layers.im2sequence(input, filter_size=1, stride=1, padding=0, input_image_size=None, out_stride=1, name=None)[source]

Extracts image patches from the input tensor to form a tensor of shape {input.batch_size * output_height * output_width, filter_size_height * filter_size_width * input.channels}. This op use filter to scan images and convert these images to sequences. After expanding, the number of time step are output_height * output_width for an image, in which output_height and output_width are calculated by below equation:

\[\begin{split}output\_height = 1 + (padding\_up + padding\_down + input\_height - filter\_size\_height + stride\_height - 1) / stride\_height \\ output\_width = 1 + (padding\_left + padding\_right + input\_width - filter\_size\_width + stride\_width - 1) / stride\_width\end{split}\]

And the dimension of each time step is filter_size_height * filter_size_width * input.channels.

Parameters
  • input (Variable) – The input should be a 4-D Tensor in \(NCHW\) format. The data type is float32.

  • filter_size (int32 | List[int32]) – The filter size. If filter_size is a List, it must contain two integers, \([filter\_size\_height, filter\_size\_width]\) . Otherwise, the filter size will be a square \([filter\_size, filter\_size]\) . Default is 1.

  • stride (int32 | List[int32]) – The stride size. If stride is a List, it must contain two integers, \([stride\_height, stride\_width]\) . Otherwise, the stride size will be a square \([stride\_size, stride\_size]\) . Default is 1.

  • padding (int32 | List[int32]) – The padding size. If padding is a List, it can contain four integers like \([padding\_up, padding\_left, padding\_down, padding\_right]\) to indicate paddings of four direction. Or it can contain two integers \([padding\_height, padding\_width]\) which means padding_up = padding_down = padding_height and padding_left = padding_right = padding_width. Otherwise, a scalar padding means padding_up = padding_down = padding_left = padding_right = padding. Default is 0.

  • input_image_size (Variable, optional) – the input contains image real size.It’s dim is \([batchsize, 2]\) . It is just for batch inference when not None. Default is None.

  • out_stride (int32 | List[int32]) – The scaling of image through CNN. It is valid only when input_image_size is not None. If out_stride is List, it must contain two intergers, \([out\_stride\_height, out\_stride\_W]\) . Otherwise, the out_stride_height = out_stride_width = out_stride. Default is 1.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

The output is a 2-D LoDTensor with shape {input.batch_size * output_height * output_width, filter_size_height * filter_size_width * input.channels}. The data type is float32.

Return Type: Variable

Examples

Given:

x = [[[[ 6.  2.  1.]
       [ 8.  3.  5.]
       [ 0.  2.  6.]]

      [[ 2.  4.  4.]
       [ 6.  3.  0.]
       [ 6.  4.  7.]]]

     [[[ 6.  7.  1.]
       [ 5.  7.  9.]
       [ 2.  4.  8.]]

      [[ 1.  2.  1.]
       [ 1.  3.  5.]
       [ 9.  0.  8.]]]]

x.dims = {2, 2, 3, 3}

And:

filter = [2, 2]
stride = [1, 1]
padding = [0, 0]

Then:

output.data = [[ 6.  2.  8.  3.  2.  4.  6.  3.]
               [ 2.  1.  3.  5.  4.  4.  3.  0.]
               [ 8.  3.  0.  2.  6.  3.  6.  4.]
               [ 3.  5.  2.  6.  3.  0.  4.  7.]
               [ 6.  7.  5.  7.  1.  2.  1.  3.]
               [ 7.  1.  7.  9.  2.  1.  3.  5.]
               [ 5.  7.  2.  4.  1.  3.  9.  0.]
               [ 7.  9.  4.  8.  3.  5.  0.  8.]]

output.dims = {8, 8}

output.lod = [[4, 4]]

Examples

import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 32, 32],
                         dtype='float32')
output = fluid.layers.im2sequence(
    input=data, stride=[1, 1], filter_size=[2, 2])

image_resize

paddle.fluid.layers.image_resize(input, out_shape=None, scale=None, name=None, resample='BILINEAR', actual_shape=None, align_corners=True, align_mode=1, data_format='NCHW')[source]

This op resizes a batch of images.

The input must be a 4-D Tensor of the shape (num_batches, channels, in_h, in_w) or (num_batches, in_h, in_w, channels), or a 5-D Tensor of the shape (num_batches, channels, in_d, in_h, in_w) or (num_batches, in_d, in_h, in_w, channels), and the resizing only applies on the three dimensions(depth, hight and width).

Warning: the parameter actual_shape will be deprecated in the future and only use out_shape instead.

Supporting resample methods:

‘BILINEAR’ : Bilinear interpolation

‘TRILINEAR’ : Trilinear interpolation

‘NEAREST’ : Nearest neighbor interpolation

Nearest neighbor interpolation is to perform nearest neighbor interpolation in both the 3rd dimention(in height direction) and the 4th dimention(in width direction) on input tensor.

Bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables (e.g. H-direction and W-direction in this op) on a rectilinear 2D grid. The key idea is to perform linear interpolation first in one direction, and then again in the other direction.

Trilinear interpolation is an extension of linear interpolation for interpolating functions of three variables (e.g. D-direction, H-direction and W-direction in this op) on a rectilinear 3D grid. The linear interpolation is performed on three directions.

Align_corners and align_mode are optinal parameters,the calculation method of interpolation can be selected by them.

Example:

For scale:

    if align_corners = True && out_size > 1 :

      scale_factor = (in_size-1.0)/(out_size-1.0)

    else:

      scale_factor = float(in_size/out_size)


Nearest neighbor interpolation:

  if:
      align_corners = False

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:

      H_out = floor (H_{in} * scale_{factor})
      W_out = floor (W_{in} * scale_{factor})

  else:
      align_corners = True

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:

      H_out = round(H_{in} * scale_{factor})
      W_out = round(W_{in} * scale_{factor})

Bilinear interpolation:

  if:
      align_corners = False , align_mode = 0

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:

      H_out = (H_{in}+0.5) * scale_{factor} - 0.5
      W_out = (W_{in}+0.5) * scale_{factor} - 0.5

  else:

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:

      H_out = H_{in} * scale_{factor}
      W_out = W_{in} * scale_{factor}

Trilinear interpolation:

  if:
      align_corners = False , align_mode = 0

      input : (N,C,D_in,H_in,W_in)
      output: (N,C,D_out,H_out,W_out) where:

      D_out = (D_{in}+0.5) * scale_{factor} - 0.5
      H_out = (H_{in}+0.5) * scale_{factor} - 0.5
      W_out = (W_{in}+0.5) * scale_{factor} - 0.5


  else:

      input : (N,C,D_in,H_in,W_in)
      output: (N,C,D_out,H_out,W_out) where:

      D_out = D_{in} * scale_{factor}
      H_out = H_{in} * scale_{factor}
      W_out = W_{in} * scale_{factor}

For details of nearest neighbor interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation.

For details of bilinear interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation.

For details of trilinear interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Trilinear_interpolation.

Parameters
  • input (Variable) – 4-D or 5-D Tensor, its data type is float32, float64, or uint8, its data format is specified by data_format.

  • out_shape (list|tuple|Variable|None) – Output shape of image resize layer, the shape is (out_h, out_w) when input is a 4-D Tensor and is (out_d, out_h, out_w) when input is a 5-D Tensor. Default: None. If a list, each element can be an integer or a Tensor Variable of shape: [1]. If a Tensor Variable, its dimensions size should be a 1.

  • scale (float|Variable|None) – The multiplier for the input height or width. At least one of out_shape or scale must be set. And out_shape has a higher priority than scale. Default: None.

  • name (str|None) – A name for this layer(optional). If set None, the layer will be named automatically.

  • resample (str) – The resample method. It supports ‘BILINEAR’, ‘TRILINEAR’ and ‘NEAREST’ currently. Default: ‘BILINEAR’

  • actual_shape (Variable) – An optional input to specify output shape dynamically. If provided, image resize according to this given shape rather than out_shape and scale specifying shape. That is to say actual_shape has the highest priority. It is recommended to use out_shape if you want to specify output shape dynamically, because actual_shape will be deprecated. When using actual_shape to specify output shape, one of out_shape and scale should also be set, otherwise errors would be occured in graph constructing stage. Default: None

  • align_corners (bool) – An optional bool, If True, the centers of the 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels. Default: True

  • align_mode (int) – An optional for bilinear interpolation. can be ‘0’ for src_idx = scale*(dst_indx+0.5)-0.5 , can be ‘1’ for src_idx = scale*dst_index.

  • data_format (str, optional) – NCHW(num_batches, channels, height, width) or NHWC(num_batches, height, width, channels) for 4-D Tensor, NCDHW(num_batches, channels, depth, height, width) or NDHWC(num_batches, depth, height, width, channels) for 5-D Tensor. Default: ‘NCHW’.

Returns

A 4-D Tensor of the shape (num_batches, channels, out_h, out_w) or (num_batches, out_h, out_w, channels), or 5-D Tensor of the shape (num_batches, channels, out_d, out_h, out_w) or (num_batches, out_d, out_h, out_w, channels).

Raises
  • TypeError – out_shape should be a list or tuple or Variable.

  • TypeError – actual_shape should either be Variable or None.

  • ValueError – The ‘resample’ of image_resize can only be ‘BILINEAR’, ‘TRILINEAR’ or ‘NEAREST’ currently.

  • ValueError – ‘BILINEAR’ and ‘NEAREST’ only support 4-D tensor.

  • ValueError – ‘TRILINEAR’ only support 5-D tensor.

  • ValueError – One of out_shape and scale must not be None.

  • ValueError – out_shape length should be 2 for input 4-D tensor.

  • ValueError – out_shape length should be 3 for input 5-D tensor.

  • ValueError – scale should be greater than zero.

  • TypeError – align_corners shoule be a bool value

  • ValueError – align_mode can only be ‘0’ or ‘1’

  • ValueError – data_format can only be ‘NCHW’, ‘NHWC’, ‘NCDHW’ or ‘NDHWC’.

Examples

#declarative mode
import paddle.fluid as fluid
import numpy as np
input = fluid.data(name="input", shape=[None,3,6,10])

#1
output = fluid.layers.image_resize(input=input,out_shape=[12,12])

#2
#x = np.array([2]).astype("int32")
#dim1 = fluid.data(name="dim1", shape=[1], dtype="int32")
#fluid.layers.assign(input=x, output=dim1)
#output = fluid.layers.image_resize(input=input,out_shape=[12,dim1])

#3
#x = np.array([3,12]).astype("int32")
#shape_tensor = fluid.data(name="shape_tensor", shape=[2], dtype="int32")
#fluid.layers.assign(input=x, output=shape_tensor)
#output = fluid.layers.image_resize(input=input,out_shape=shape_tensor)

#4
#x = np.array([0.5]).astype("float32")
#scale_tensor = fluid.data(name="scale", shape=[1], dtype="float32")
#fluid.layers.assign(x,scale_tensor)
#output = fluid.layers.image_resize(input=input,scale=scale_tensor)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

input_data = np.random.rand(2,3,6,10).astype("float32")

output_data = exe.run(fluid.default_main_program(),
    feed={"input":input_data},
    fetch_list=[output],
    return_numpy=True)

print(output_data[0].shape)

#1
# (2, 3, 12, 12)
#2
# (2, 3, 12, 2)
#3
# (2, 3, 3, 12)
#4
# (2, 3, 3, 5)

#imperative mode
import paddle.fluid.dygraph as dg

with dg.guard(place) as g:
    input = dg.to_variable(input_data)
    output = fluid.layers.image_resize(input=input, out_shape=[12,12])
    print(output.shape)

    # [2L, 3L, 12L, 12L]

image_resize_short

paddle.fluid.layers.image_resize_short(input, out_short_len, resample='BILINEAR')[source]

This op resizes a batch of images. The short edge of input images will be resized to the given ‘out_short_len’. The long edge of input images will be resized proportionately to make images’ length-width ratio constant.

Parameters
  • input (Variable) – 4-D tensor(NCHW), The input tensor of image resize layer.

  • out_short_len (int) – The length of output images’ short edge.

  • resample (str) – resample method, default: BILINEAR.

Returns

4-D tensor(NCHW).

Return type

Variable

Examples

import paddle.fluid as fluid
input = fluid.data(name="input", shape=[None,3,6,9], dtype="float32")
out = fluid.layers.image_resize_short(input, out_short_len=3)

kldiv_loss

paddle.fluid.layers.kldiv_loss(x, target, reduction='mean', name=None)[source]

This operator calculates the Kullback-Leibler divergence loss between Input(X) and Input(Target). Notes that Input(X) is the log-probability and Input(Target) is the probability.

KL divergence loss is calculated as follows:

$$l(x, y) = y * (log(y) - x)$$

While \(x\) is Input(X) and \(y\) is Input(Target).

While reduction is none, output loss is in the same shape as Input(X), loss in each point is calculated seperately and no reduction is applied.

While reduction is mean, output loss is in shape of [1] and loss value is the mean value of all losses.

While reduction is sum, output loss is in shape of [1] and loss value is the sum value of all losses.

While reduction is batchmean, output loss is in shape of [1] and loss value is the sum value of all losses divided by batch size.

Parameters
  • x (Variable) – The input tensor of KL divergence loss operator. This is a tensor with shape of [N, *], where N is the batch size, * means any number of additional dimensions. The data type is float32 or flaot64

  • target (Variable) – The tensor of KL divergence loss operator. This is a tensor with shape of Input(X). The data type is same as Input(X)

  • reduction (Variable) – The reduction type to apply to the output, available types are ‘none’ | ‘batchmean’ | ‘mean’ | ‘sum’, ‘none’ for no reduction, ‘batchmean’ for the sum of output divided by batch size, ‘mean’ for the average value of all output, ‘sum’ for the sum of the output

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

The KL divergence loss. The data type is same as input tensor

Return type

Variable(Tensor)

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[None,4,2,2], dtype='float32')
target = fluid.layers.data(name='target', shape=[4,2,2], dtype='float32')
loss = fluid.layers.kldiv_loss(x=x, target=target, reduction='batchmean')

l2_normalize

paddle.fluid.layers.l2_normalize(x, axis, epsilon=1e-12, name=None)[source]

This op normalizes x along dimension axis using an L2 norm. For a 1-D tensor (dim is fixed to 0), this layer computes

\[y = \frac{x}{ \sqrt{\sum {x^2} + epsion }}\]

For x with more dimensions, this layer independently normalizes each 1-D slice along dimension axis.

Parameters
  • x (Variable|list) – The input tensor could be N-D tensor, and the input data type could be float32 or float64.

  • axis (int) – The axis on which to apply normalization. If axis < 0, the dimension to normalization is rank(X) + axis. -1 is the last dimension.

  • epsilon (float) – The epsilon value is used to avoid division by zero, the default value is 1e-12.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

The output has the same shape and data type with x.

Return type

Variable

Examples

# declarative mode
import paddle.fluid as fluid
import numpy as np
input = fluid.data(name="input", shape=[2,3])
output = fluid.layers.l2_normalize(x=input,axis=0)
place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

input_data = np.random.rand(2,3).astype("float32")
print(input_data)

# [[0.5171216  0.12704141 0.56018186]
# [0.93251234 0.5382788  0.81709313]]

output_data = exe.run(fluid.default_main_program(),
    feed={"input":input_data},
    fetch_list=[output],
    return_numpy=True)

print(output_data)

# [array([[0.48496857, 0.22970329, 0.56545246],
# [0.8745316 , 0.9732607 , 0.82478094]], dtype=float32)]

# imperative mode
import paddle.fluid.dygraph as dg

with dg.guard(place) as g:
    input = dg.to_variable(input_data)
    output = fluid.layers.l2_normalize(x=input, axis=-1)
    print(output.numpy())

    # [[0.66907585 0.16437206 0.7247892 ]
    # [0.6899054  0.3982376  0.6045142 ]]

label_smooth

paddle.fluid.layers.label_smooth(label, prior_dist=None, epsilon=0.1, dtype='float32', name=None)[source]

Label smoothing is a mechanism to regularize the classifier layer and is called label-smoothing regularization (LSR).

Label smoothing is proposed to encourage the model to be less confident, since optimizing the log-likelihood of the correct label directly may cause overfitting and reduce the ability of the model to adapt. Label smoothing replaces the ground-truth label \(y\) with the weighted sum of itself and some fixed distribution \(\mu\). For class \(k\), i.e.

\[\tilde{y_k} = (1 - \epsilon) * y_k + \epsilon * \mu_k,\]

where \(1 - \epsilon\) and \(\epsilon\) are the weights respectively, and \(\tilde{y}_k\) is the smoothed label. Usually uniform distribution is used for \(\mu\).

See more details about label smoothing in https://arxiv.org/abs/1512.00567.

Parameters
  • label (Variable) – The input variable containing the label data. The label data should use one-hot representation. It’s a multidimensional tensor with a shape of \([N_1, ..., Depth]\), where Depth is class number.

  • prior_dist (Variable, optional) – The prior distribution to be used to smooth labels. If not provided, an uniform distribution is used. It’s a multidimensional tensor with a shape of \([1, class\_num]\) . The default value is None.

  • epsilon (float, optional) – The weight used to mix up the original ground-truth distribution and the fixed distribution. The default value is 0.1.

  • dtype (np.dtype|core.VarDesc.VarType|str, optional) – The data type can be set as ‘float32’, ‘float64’. The default value is ‘float32’.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.

Returns

The tensor variable containing the smoothed labels.

Return type

Variable

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers

label = layers.data(name="label", shape=[1], dtype="float32")
one_hot_label = layers.one_hot(input=label, depth=10)
smooth_label = layers.label_smooth(
    label=one_hot_label, epsilon=0.1, dtype="float32")

layer_norm

paddle.fluid.layers.layer_norm(input, scale=True, shift=True, begin_norm_axis=1, epsilon=1e-05, param_attr=None, bias_attr=None, act=None, name=None)[source]

Layer Normalization Layer

The API implements the function of the Layer Normalization Layer and can be applied to mini-batch input data. Refer to Layer Normalization

The formula is as follows:

\[ \begin{align}\begin{aligned}\mu & = \frac{1}{H}\sum_{i=1}^{H} x_i\\\sigma & = \sqrt{\frac{1}{H}\sum_{i=1}^{H}{(x_i - \mu)^2} + \epsilon}\\y & = f(\frac{g}{\sigma}(x - \mu) + b)\end{aligned}\end{align} \]
  • \(x\): the vector representation of the summed inputs to the neurons in that layer.

  • \(H\): the number of hidden units in a layers

  • \(\epsilon\): the small value added to the variance to prevent division by zero.

  • \(g\): the trainable scale parameter.

  • \(b\): the trainable bias parameter.

Parameters
  • input (Variable) – A multi-dimension Tensor , and the data type is float32 or float64.

  • scale (bool, optional) – Whether to learn the adaptive gain \(g\) after normalization. Default: True.

  • shift (bool, optional) – Whether to learn the adaptive bias \(b\) after normalization. Default: True.

  • begin_norm_axis (int, optional) – The normalization will be performed along dimensions from begin_norm_axis to rank(input). Default: 1.

  • epsilon (float, optional) – The small value added to the variance to prevent division by zero. Default: 1e-05.

  • param_attr (ParamAttr, optional) – The parameter attribute for the learnable gain \(g\). If scale is False, param_attr is omitted. If scale is True and param_attr is None, a default ParamAttr would be added as scale. The param_attr is initialized as 1 if it is added. Default: None.

  • bias_attr (ParamAttr, optional) – The parameter attribute for the learnable bias \(b\). If shift is False, bias_attr is omitted. If shift is True and param_attr is None, a default ParamAttr would be added as bias. The bias_attr is initialized as 0 if it is added. Default: None.

  • act (str, optional) – Activation to be applied to the output of layer normalizaiton. Default: None.

  • name (str) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

Tensor indicating the normalized result, the data type is the same as input , and the return dimension is the same as input .

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np
x = fluid.data(name='x', shape=[-1, 32, 32], dtype='float32')
hidden1 = fluid.layers.layer_norm(input=x, begin_norm_axis=1)
place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
np_x = np.random.random(size=(8, 3, 32, 32)).astype('float32')
output = exe.run(feed={"x": np_x}, fetch_list = [hidden1])
print(output)

leaky_relu

paddle.fluid.layers.leaky_relu(x, alpha=0.02, name=None)[source]

LeakyRelu Activation Operator.

$$out = max(x, alpha * x)$$

Parameters
  • x (Variable) – A LoDTensor or Tensor representing preactivation values. Must be one of the following types: float32, float64

  • alpha (FLOAT|0.02) – Slope of the activation function at x < 0

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

A LoDTensor or Tensor with the same type and size as that of x

Return type

output(Variable)

Examples

import paddle.fluid as fluid
import numpy as np

# Graph Organizing
x = fluid.layers.data(name="x", shape=[2], dtype="float32")
res = fluid.layers.leaky_relu(x, alpha=0.1)

# Create an executor using CPU as an example
exe = fluid.Executor(fluid.CPUPlace())

# Execute
x_i = np.array([[-1, 2], [3, -4]]).astype(np.float32)
res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res])
print(res_val) # [[-0.1, 2], [3, -0.4]]

linear_chain_crf

paddle.fluid.layers.linear_chain_crf(input, label, param_attr=None, length=None)[source]

Linear Chain CRF.

Conditional Random Field defines an undirected probabilistic graph with nodes denoting random variables and edges denoting dependencies between these variables. CRF learns the conditional probability \(P(Y|X)\), where \(X = (x_1, x_2, ... , x_n)\) are structured inputs and \(Y = (y_1, y_2, ... , y_n)\) are labels for the inputs.

Linear chain CRF is a special case of CRF that is useful for sequence labeling task. Sequence labeling tasks do not assume a lot of conditional independences among inputs. The only constraint they impose is that the input and output must be linear sequences. Thus, the graph of such a CRF is a simple chain or a line, which results in the linear chain CRF.

This operator implements the Forward-Backward algorithm for the linear chain CRF. Please refer to http://www.cs.columbia.edu/~mcollins/fb.pdf and http://cseweb.ucsd.edu/~elkan/250Bwinter2012/loglinearCRFs.pdf for details.

Equation:

  1. Denote Input(Emission) to this operator as \(x\) here. 2. The first D values of Input(Transition) to this operator are for starting weights, denoted as \(a\) here. 3. The next D values of Input(Transition) of this operator are for ending weights, denoted as \(b\) here. 4. The remaning values of Input(Transition) are for transition weights, denoted as \(w\) here. 5. Denote Input(Label) as \(s\) here.

The probability of a sequence \(s\) of length \(L\) is defined as: $$P(s) = (1/Z) exp(a_{s_1} + b_{s_L} + sum_{l=1}^L x_{s_l} + sum_{l=2}^L w_{s_{l-1},s_l})$$

where \(Z\) is a normalization value so that the sum of \(P(s)\) over all possible sequences is 1, and \(x\) is the emission feature weight to the linear chain CRF.

Finally, the linear chain CRF operator outputs the logarithm of the conditional likelihood of each training sample in a mini-batch.

NOTE:

  1. The feature function for a CRF is made up of the emission features and the transition features. The emission feature weights are NOT computed in this operator. They MUST be computed first before this operator is called.

  2. Because this operator performs global normalization over all possible sequences internally, it expects UNSCALED emission feature weights. Please do not call this op with the emission feature being output of any nonlinear activation.

  3. The 2nd dimension of Input(Emission) MUST be equal to the tag number.

Parameters
  • input (Variable) – (LoDTensor/Tensor<float>). When a LoDTensor input,A 2-D LoDTensor with shape [N x D], where N is the size of the mini-batch and D is the total tag number. The unscaled emission weight matrix for the linear chain CRF. When a Tensor input,A Tensor with shape [N x S x D], where N is batch number,S is max length of sequences, D is the total tag number.A LoDTensor or Tensor with type float32, float64

  • label (Variable) – (LoDTensor/Tensor<int64_t>), when a LoDTensor input, [N x 1], where N is the total element number in a mini-batch. when a Tensor input, [N x S], where N is batch number. S is max length of sequences. The ground truth.A LoDTensor or Tensor with int64

  • Length (Variable) – (Tensor, default Tensor<int64_t>) A Tensor with shape [M x 1], where M is the sequence number in a mini-batch.A Tensor with type int64

  • param_attr (ParamAttr) – The attribute of the learnable parameter for transition parameter.

Returns

(Tensor, default Tensor<float>), the same shape with Emission. The exponentials of Input(Emission). This is an intermediate computational result in forward computation, and will be reused in backward computation.A LoDTensor or Tensor with type float32, float64

output(Variable): (Tensor, default Tensor<float>) A 2-D Tensor with shape [(D + 2) x D]. The exponentials of Input(Transition). This is an intermediate computational result in forward computation, and will be reused in backward computation.A LoDTensor or Tensor with type float32, float64

output(Variable): (Tensor, default Tensor<float>) The logarithm of the conditional likelihood of each training sample in a mini-batch. This is a 2-D tensor with shape [S x 1], where S is the sequence number in a mini-batch. Note: S is equal to the sequence number in a mini-batch. A Tensor with type float32, float64

Return type

output(Variable)

Examples

import paddle.fluid as fluid
import numpy as np

#define net structure, using LodTensor
train_program = fluid.Program()
startup_program = fluid.Program()
with fluid.program_guard(train_program, startup_program):
    input_data = fluid.data(name='input_data', shape=[-1,10], dtype='float32')
    label = fluid.data(name='label', shape=[-1,1], dtype='int')
    emission= fluid.layers.fc(input=input_data, size=10, act="tanh")
    crf_cost = fluid.layers.linear_chain_crf(
        input=emission,
        label=label,
        param_attr=fluid.ParamAttr(
        name='crfw',
        learning_rate=0.01))
use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(startup_program)
#define data, using LoDTensor
a = fluid.create_lod_tensor(np.random.rand(12,10).astype('float32'), [[3,3,4,2]], place)
b = fluid.create_lod_tensor(np.array([[1],[1],[2],[3],[1],[1],[1],[3],[1],[1],[1],[1]]),[[3,3,4,2]] , place)
feed1 = {'input_data':a,'label':b}
loss= exe.run(train_program,feed=feed1, fetch_list=[crf_cost])
print(loss)

#define net structure, using padding
train_program = fluid.Program()
startup_program = fluid.Program()
with fluid.program_guard(train_program, startup_program):
    input_data2 = fluid.data(name='input_data2', shape=[-1,10,10], dtype='float32')
    label2 = fluid.data(name='label2', shape=[-1,10,1], dtype='int')
    label_length = fluid.data(name='length', shape=[-1,1], dtype='int')
    emission2= fluid.layers.fc(input=input_data2, size=10, act="tanh", num_flatten_dims=2)
    crf_cost2 = fluid.layers.linear_chain_crf(
        input=emission2,
        label=label2,
        length=label_length,
        param_attr=fluid.ParamAttr(
         name='crfw',
         learning_rate=0.01))

use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(startup_program)

#define data, using padding
cc=np.random.rand(4,10,10).astype('float32')
dd=np.random.rand(4,10,1).astype('int64')
ll=np.array([[3],[3],[4],[2]])
feed2 = {'input_data2':cc,'label2':dd,'length':ll}
loss2= exe.run(train_program,feed=feed2, fetch_list=[crf_cost2])
print(loss2)
#[array([[ 7.8902354],
#        [ 7.3602567],
#        [ 10.004011],
#        [ 5.86721  ]], dtype=float32)]

#you can use find_var to get transition parameter.
transition=np.array(fluid.global_scope().find_var('crfw').get_tensor())
print(transition)

lod_reset

paddle.fluid.layers.lod_reset(x, y=None, target_lod=None)[source]

Set LoD of x to a new one specified by y or target_lod. When y provided, y.lod would be considered as target LoD first, otherwise y.data would be considered as target LoD. If y is not provided, target LoD should be specified by target_lod. If target LoD is specified by y.data or target_lod, only one level LoD is supported.

* Example 1:

    Given a 1-level LoDTensor x:
        x.lod =  [[ 2,           3,                   1 ]]
        x.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]]
        x.dims = [6, 1]

    target_lod: [4, 2]

    then we get a 1-level LoDTensor:
        out.lod =  [[4,                          2]]
        out.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]]
        out.dims = [6, 1]

* Example 2:

    Given a 1-level LoDTensor x:
        x.lod =  [[2,            3,                   1]]
        x.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]]
        x.dims = [6, 1]

    y is a Tensor:
        y.data = [[2, 4]]
        y.dims = [1, 3]

    target_lod:
        This parameter does not work when y is not none.

    then we get a 1-level LoDTensor:
        out.lod =  [[2,            4]]
        out.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]]
        out.dims = [6, 1]

* Example 3:

    Given a 1-level LoDTensor x:
        x.lod =  [[2,            3,                   1]]
        x.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]]
        x.dims = [6, 1]

    y is a 2-level LoDTensor:
        y.lod =  [[2, 2], [2, 2, 1, 1]]
        y.data = [[1.1], [2.1], [3.1], [4.1], [5.1], [6.1]]
        y.dims = [6, 1]

    target_lod:
        This parameter does not work when y is not none.

    then we get a 2-level LoDTensor:
        out.lod =  [[2, 2], [2, 2, 1, 1]]
        out.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]]
        out.dims = [6, 1]
Parameters
  • x (Variable) – Input variable which could be a Tensor or LoDTensor.

  • y (Variable|optional) – If provided, output’s LoD would be derived from y.

  • target_lod (list|tuple|optional) – One level LoD which should be considered as target LoD when y not provided.

Returns

Output variable with LoD specified by this layer.

Return type

Variable

Raises

ValueError – If y and target_lod are both None.

Examples

import paddle.fluid as fluid
import numpy

# Graph Organizing
x = fluid.data(name='x', shape=[6])
y = fluid.data(name='y', shape=[6], lod_level=1)
output = fluid.layers.lod_reset(x=x, y=y)

# Create an executor using CPU as an example
place = fluid.CPUPlace()
exe = fluid.Executor(place)

# Execute
x_tensor = fluid.core.LoDTensor()
x_tensor.set(numpy.ones([6]).astype(numpy.float32), place)
y_ndarray = numpy.ones([6]).astype(numpy.float32)
y_lod = [[2, 2], [2, 2, 1, 1]]
y_tensor = fluid.create_lod_tensor(y_ndarray, y_lod, place)

res, = exe.run(fluid.default_main_program(),
               feed={'x':x_tensor, 'y':y_tensor},
               fetch_list=[output],
               return_numpy=False)
print(res)
# Output Value:
# lod: [[0, 2, 4], [0, 2, 4, 5, 6]]
# dim: 6
# layout: NCHW
# dtype: float
# data: [1 1 1 1 1 1]

log

paddle.fluid.layers.log(x, name=None)[source]

Calculates the natural log of the given input tensor, element-wise.

\[Out = \ln(x)\]
Parameters
  • x (Variable) – Input LoDTensor or Tensor. Must be one of the following types: float32, float64.

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

The natural log of the input LoDTensor or Tensor computed element-wise.

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

# Graph Organizing
x = fluid.layers.data(name="x", shape=[1], dtype="float32")
res = fluid.layers.log(x)

# Create an executor using CPU as an example
exe = fluid.Executor(fluid.CPUPlace())

# Execute
x_i = np.array([[1], [2]]).astype(np.float32)
res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res])
print(res_val) # [[0.], [0.6931472]]

log_loss

paddle.fluid.layers.log_loss(input, label, epsilon=0.0001, name=None)[source]

Negative Log Loss Layer

This layer accepts input predictions and target label and returns the negative log loss.

\[Out = -label * \log{(input + \epsilon)} - (1 - label) * \log{(1 - input + \epsilon)}\]
Parameters
  • input (Variable|list) – A 2-D tensor with shape [N x 1], where N is the batch size. This input is a probability computed by the previous operator. Data type float32.

  • label (Variable|list) – The ground truth which is a 2-D tensor with shape [N x 1], where N is the batch size. Data type float32.

  • epsilon (float, optional) – A small number for numerical stability. Default 1e-4.

  • name (str|None) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.

Returns

A 2-D tensor with shape [N x 1], the negative log loss.

Return type

Variable

Examples

import paddle.fluid as fluid
label = fluid.data(name='label', shape=[-1, 1], dtype='int64')
prob = fluid.data(name='prob', shape=[-1, 10], dtype='float32')
cost = fluid.layers.log_loss(input=prob, label=label)

logical_and

paddle.fluid.layers.logical_and(x, y, out=None, name=None)[source]

logical_and Operator

It operates element-wise on X and Y, and returns the Out. X, Y and Out are N-dim boolean LoDTensor or Tensor. Each element of Out is calculated by

\[Out = X \land Y\]
Parameters
  • x (Variable) – Left hand operand of logical_and operator. Must be a LoDTensor or Tensor of type bool

  • y (Variable) – Right hand operand of logical_and operator. Must be a LoDTensor or Tensor of type bool

  • out (LoDTensor or Tensor) – The LoDTensor or Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

n-dim bool LoDTensor or Tensor

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

# Graph organizing
x = fluid.layers.data(name='x', shape=[2], dtype='bool')
y = fluid.layers.data(name='y', shape=[2], dtype='bool')
res = fluid.layers.logical_and(x=x, y=y)
# The comment lists another available method.
# res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0)
# fluid.layers.logical_and(x=x, y=y, out=res)

# Create an executor using CPU as an example
exe = fluid.Executor(fluid.CPUPlace())

# Execute
x_i = np.array([[1, 0], [0, 1]]).astype(np.bool)
y_i = np.array([[1, 1], [0, 0]]).astype(np.bool)
res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i, 'y':y_i}, fetch_list=[res])
print(res_val) # [[True, False], [False, False]]

logical_not

paddle.fluid.layers.logical_not(x, out=None, name=None)[source]

logical_not Operator

It operates element-wise on X, and returns the Out. X and Out are N-dim boolean LoDTensor or Tensor. Each element of Out is calculated by

\[Out = \lnot X\]
Parameters
  • x (Variable) – Operand of logical_not operator. Must be a LoDTensor or Tensor of type bool

  • out (LoDTensor/Tensor) – The LoDTensor/Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

n-dim bool LoDTensor or Tensor

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

# Graph organizing
x = fluid.layers.data(name='x', shape=[2], dtype='bool')
res = fluid.layers.logical_not(x)
# The comment lists another availble method.
# res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0)
# fluid.layers.logical_not(x, out=res)

# Create an executor using CPU as an example
exe = fluid.Executor(fluid.CPUPlace())

# Execute
x_i = np.array([[1, 0]]).astype(np.bool)
res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res])
print(res_val) # [[False, True]]

logical_or

paddle.fluid.layers.logical_or(x, y, out=None, name=None)[source]

logical_or Operator

It operates element-wise on X and Y, and returns the Out. X, Y and Out are N-dim boolean LoDTensor or Tensor. Each element of Out is calculated by

\[Out = X \lor Y\]
Parameters
  • x (Variable) – Left hand operand of logical_or operator. Must be a LoDTensor or Tensor of type bool

  • y (Variable) – Right hand operand of logical_or operator. Must be a LoDTensor or Tensor of type bool

  • out (LoDTensor or Tensor) – The LoDTensor or Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

n-dim bool LoDTensor or Tensor

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

# Graph organizing
x = fluid.layers.data(name='x', shape=[2], dtype='bool')
y = fluid.layers.data(name='y', shape=[2], dtype='bool')
res = fluid.layers.logical_or(x=x, y=y)
# The comment lists another available method.
# res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0)
# fluid.layers.logical_or(x=x, y=y, out=res)

# Create an executor using CPU as an example
exe = fluid.Executor(fluid.CPUPlace())

# Execute
x_i = np.array([[1, 0], [0, 1]]).astype(np.bool)
y_i = np.array([[1, 1], [0, 0]]).astype(np.bool)
res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i, 'y':y_i}, fetch_list=[res])
print(res_val) # [[True, True], [False, True]]

logical_xor

paddle.fluid.layers.logical_xor(x, y, out=None, name=None)[source]

logical_xor Operator

It operates element-wise on X and Y, and returns the Out. X, Y and Out are N-dim boolean LoDTensor or Tensor. Each element of Out is calculated by

\[Out = (X \lor Y) \land \lnot (X \land Y)\]
Parameters
  • x (Variable) – Left hand operand of logical_xor operator. Must be a LoDTensor or Tensor of type bool

  • y (Variable) – Right hand operand of logical_xor operator. Must be a LoDTensor or Tensor of type bool

  • out (LoDTensor or Tensor) – The LoDTensor or Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

n-dim bool LoDTensor or Tensor

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

# Graph organizing
x = fluid.layers.data(name='x', shape=[2], dtype='bool')
y = fluid.layers.data(name='y', shape=[2], dtype='bool')
res = fluid.layers.logical_xor(x=x, y=y)
# The comment lists another available method.
# res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0)
# fluid.layers.logical_xor(x=x, y=y, out=res)

# Create an executor using CPU as an example
exe = fluid.Executor(fluid.CPUPlace())

# Execute
x_i = np.array([[1, 0], [0, 1]]).astype(np.bool)
y_i = np.array([[1, 1], [0, 0]]).astype(np.bool)
res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i, 'y':y_i}, fetch_list=[res])
print(res_val) # [[False, True], [False, True]]

lrn

paddle.fluid.layers.lrn(input, n=5, k=1.0, alpha=0.0001, beta=0.75, name=None)[source]

This operator implements the Local Response Normalization Layer. This layer performs a type of “lateral inhibition” by normalizing over local input regions. For more information, please refer to ImageNet Classification with Deep Convolutional Neural Networks

The formula is as follows:

\[Output(i, x, y) = Input(i, x, y) / \left(k + \alpha \sum\limits^{\min(C-1, i + n/2)}_{j = \max(0, i - n/2)}(Input(j, x, y))^2\right)^{\beta}\]

In the above equation:

  • \(n\) : The number of channels to sum over.

  • \(k\) : The offset (avoid being divided by 0).

  • \(\alpha\) : The scaling parameter.

  • \(\beta\) : The exponent parameter.

Parameters
  • input (Variable) – Input feature, 4D-Tensor with the shape of [N,C,H,W], where N is the batch size, C is the input channel, H is Height, W is weight. The data type is float32. The rank of this tensor must be 4, otherwise it will raise ValueError.

  • n (int, optional) – The number of channels to sum over. Default: 5

  • k (float, optional) – An offset, positive. Default: 1.0

  • alpha (float, optional) – The scaling parameter, positive. Default:1e-4

  • beta (float, optional) – The exponent, positive. Default:0.75

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

A tensor variable storing the transformation result with the same shape and data type as input.

Return type

Variable

Examples:

import paddle.fluid as fluid
data = fluid.data(
    name="data", shape=[None, 3, 112, 112], dtype="float32")
lrn = fluid.layers.lrn(input=data)
print(lrn.shape)  # [-1, 3, 112, 112]
print(lrn.dtype)  # float32

lstm

paddle.fluid.layers.lstm(input, init_h, init_c, max_len, hidden_size, num_layers, dropout_prob=0.0, is_bidirec=False, is_test=False, name=None, default_initializer=None, seed=-1)[source]
Note:

This OP only supports running on GPU devices.

This OP implements LSTM operation - Hochreiter, S., & Schmidhuber, J. (1997) .

The implementation of this OP does not include diagonal/peephole connections. Please refer to Gers, F. A., & Schmidhuber, J. (2000) . If you need peephole connections, please use dynamic_lstm .

This OP computes each timestep as follows:

\[i_t = \sigma(W_{ix}x_{t} + W_{ih}h_{t-1} + b_{x_i} + b_{h_i})\]
\[f_t = \sigma(W_{fx}x_{t} + W_{fh}h_{t-1} + b_{x_f} + b_{h_f})\]
\[o_t = \sigma(W_{ox}x_{t} + W_{oh}h_{t-1} + b_{x_o} + b_{h_o})\]
\[\widetilde{c_t} = tanh(W_{cx}x_t + W_{ch}h_{t-1} + b{x_c} + b_{h_c})\]
\[c_t = f_t \odot c_{t-1} + i_t \odot \widetilde{c_t}\]
\[h_t = o_t \odot tanh(c_t)\]

The symbolic meanings in the formula are as follows:

  • \(x_{t}\) represents the input at timestep \(t\)

  • \(h_{t}\) represents the hidden state at timestep \(t\)

  • \(h_{t-1}, c_{t-1}\) represent the hidden state and cell state at timestep \(t-1\) , respectively

  • \(\widetilde{c_t}\) represents the candidate cell state

  • \(i_t\) , \(f_t\) and \(o_t\) represent input gate, forget gate, output gate, respectively

  • \(W\) represents weight (e.g., \(W_{ix}\) is the weight of a linear transformation of input \(x_{t}\) when calculating input gate \(i_t\) )

  • \(b\) represents bias (e.g., \(b_{i}\) is the bias of input gate)

  • \(\sigma\) represents nonlinear activation function for gate, default sigmoid

  • \(\odot\) represents the Hadamard product of a matrix, i.e. multiplying the elements of the same position for two matrices with the same dimension to get another matrix with the same dimension

Parameters
  • input (api_guide_Variable_en) – LSTM input tensor, 3-D Tensor of shape \([batch\_size, seq\_len, input\_dim]\) . Data type is float32 or float64

  • init_h (api_guide_Variable_en) – The initial hidden state of the LSTM, 3-D Tensor of shape \([num\_layers, batch\_size, hidden\_size]\) . If is_bidirec = True, shape should be \([num\_layers*2, batch\_size, hidden\_size]\) . Data type is float32 or float64.

  • init_c (api_guide_Variable_en) – The initial cell state of the LSTM, 3-D Tensor of shape \([num\_layers, batch\_size, hidden\_size]\) . If is_bidirec = True, shape should be \([num\_layers*2, batch\_size, hidden\_size]\) . Data type is float32 or float64.

  • max_len (int) – max length of LSTM. the first dim of input tensor CAN NOT greater than max_len.

  • hidden_size (int) – hidden size of the LSTM.

  • num_layers (int) – total layers number of the LSTM.

  • dropout_prob (float, optional) – dropout prob, dropout ONLY work between rnn layers, NOT between time steps There is NO dropout work on rnn output of the last RNN layers. Default: 0.0.

  • is_bidirec (bool, optional) – If it is bidirectional. Default: False.

  • is_test (bool, optional) – If it is in test phrase. Default: False.

  • name (str, optional) – A name for this layer. If set None, the layer will be named automatically. Default: None.

  • default_initializer (Initializer, optional) – Where use initializer to initialize the Weight If set None, defaule initializer will be used. Default: None.

  • seed (int, optional) – Seed for dropout in LSTM, If it’s -1, dropout will use random seed. Default: 1.

Returns

Three tensors, rnn_out, last_h, last_c:

  • rnn_out is result of LSTM hidden, shape is \([seq\_len, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([seq\_len, batch\_size, hidden\_size*2]\)

  • last_h is the hidden state of the last step of LSTM shape is \([num\_layers, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([num\_layers*2, batch\_size, hidden\_size]\)

  • last_c(Tensor): the cell state of the last step of LSTM shape is \([num\_layers, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([num\_layers*2, batch\_size, hidden\_size]\)

Return type

tuple ( api_guide_Variable_en , api_guide_Variable_en , api_guide_Variable_en )

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers

emb_dim = 256
vocab_size = 10000
data = fluid.data(name='x', shape=[None, 100], dtype='int64')
emb = fluid.embedding(input=data, size=[vocab_size, emb_dim], is_sparse=True)
batch_size = 20
max_len = 100
dropout_prob = 0.2
input_size = 100
hidden_size = 150
num_layers = 1
init_h = layers.fill_constant( [num_layers, batch_size, hidden_size], 'float32', 0.0 )
init_c = layers.fill_constant( [num_layers, batch_size, hidden_size], 'float32', 0.0 )
rnn_out, last_h, last_c = layers.lstm( emb, init_h, init_c,                     max_len, hidden_size, num_layers,                     dropout_prob=dropout_prob)
rnn_out.shape  # (-1, 100, 150)
last_h.shape  # (1, 20, 150)
last_c.shape  # (1, 20, 150)

lstm_unit

paddle.fluid.layers.lstm_unit(x_t, hidden_t_prev, cell_t_prev, forget_bias=0.0, param_attr=None, bias_attr=None, name=None)[source]

Long-Short Term Memory (LSTM) RNN cell. This operator performs LSTM calculations for one time step, whose implementation is based on calculations described in RECURRENT NEURAL NETWORK REGULARIZATION .

We add forget_bias to the biases of the forget gate in order to reduce the scale of forgetting. The formula is as follows:

\[ \begin{align}\begin{aligned}i_{t} & = \sigma(W_{x_{i}}x_{t} + W_{h_{i}}h_{t-1} + b_{i})\\f_{t} & = \sigma(W_{x_{f}}x_{t} + W_{h_{f}}h_{t-1} + b_{f} + forget\_bias)\\c_{t} & = f_{t}c_{t-1} + i_{t} tanh (W_{x_{c}}x_{t} + W_{h_{c}}h_{t-1} + b_{c})\\o_{t} & = \sigma(W_{x_{o}}x_{t} + W_{h_{o}}h_{t-1} + b_{o})\\h_{t} & = o_{t} tanh (c_{t})\end{aligned}\end{align} \]

\(x_{t}\) stands for x_t , corresponding to the input of current time step; \(h_{t-1}\) and \(c_{t-1}\) correspond to hidden_t_prev and cell_t_prev , representing the output of from previous time step. \(i_{t}, f_{t}, c_{t}, o_{t}, h_{t}\) are input gate, forget gate, cell, output gate and hidden calculation.

Parameters
  • x_t (Variable) – A 2D Tensor representing the input of current time step. Its shape should be \([N, M]\) , where \(N\) stands for batch size, \(M\) for the feature size of input. The data type should be float32 or float64.

  • hidden_t_prev (Variable) – A 2D Tensor representing the hidden value from previous step. Its shape should be \([N, D]\) , where \(N\) stands for batch size, \(D\) for the hidden size. The data type should be same as x_t .

  • cell_t_prev (Variable) – A 2D Tensor representing the cell value from previous step. It has the same shape and data type with hidden_t_prev .

  • forget_bias (float, optional) – \(forget\_bias\) added to the biases of the forget gate. Default 0.

  • param_attr (ParamAttr, optional) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • bias_attr (ParamAttr, optional) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

The tuple contains two Tensor variables with the same shape and data type with hidden_t_prev , representing the hidden value and cell value which correspond to \(h_{t}\) and \(c_{t}\) in the formula.

Return type

tuple

Raises
  • ValueError – Rank of x_t must be 2.

  • ValueError – Rank of hidden_t_prev must be 2.

  • ValueError – Rank of cell_t_prev must be 2.

  • ValueError – The 1st dimensions of x_t, hidden_t_prev and cell_t_prev must be the same.

  • ValueError – The 2nd dimensions of hidden_t_prev and cell_t_prev must be the same.

Examples

import paddle.fluid as fluid

dict_dim, emb_dim, hidden_dim = 128, 64, 512
data = fluid.data(name='step_data', shape=[None], dtype='int64')
x = fluid.embedding(input=data, size=[dict_dim, emb_dim])
pre_hidden = fluid.data(
    name='pre_hidden', shape=[None, hidden_dim], dtype='float32')
pre_cell = fluid.data(
    name='pre_cell', shape=[None, hidden_dim], dtype='float32')
hidden = fluid.layers.lstm_unit(
    x_t=x,
    hidden_t_prev=pre_hidden,
    cell_t_prev=pre_cell)

margin_rank_loss

paddle.fluid.layers.margin_rank_loss(label, left, right, margin=0.1, name=None)[source]

Margin Ranking Loss Layer for ranking problem, which compares left score and right score passed in. The ranking loss can be defined as following equation:

\[rank\_loss = max(0, -label * (left - right) + margin)\]
Parameters
  • label (Variable) – Indicates whether the left is ranked higher than the right or not. Data type is float32.

  • left (Variable) – Ranking score for left. Data type float32.

  • right (Variable) – Ranking score for right. Data type float32.

  • margin (float) – Indicates the given margin.

  • name (str|None) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.

Returns

The ranking loss.

Return type

Variable

Raises

ValueError – Any of label, left, and right is not a Variable.

Examples

import paddle.fluid as fluid
label = fluid.data(name="label", shape=[-1, 1], dtype="float32")
left = fluid.data(name="left", shape=[-1, 1], dtype="float32")
right = fluid.data(name="right", shape=[-1, 1], dtype="float32")
out = fluid.layers.margin_rank_loss(label, left, right)

matmul

paddle.fluid.layers.matmul(x, y, transpose_x=False, transpose_y=False, alpha=1.0, name=None)[source]

Applies matrix multiplication to two tensors.

Currently, the input tensors’ rank can be any, but when the rank of any inputs is bigger than 3, this two inputs’ rank should be equal.

The actual behavior depends on the shapes of \(x\), \(y\) and the flag values of transpose_x, transpose_y. Specifically:

  • If a transpose flag is specified, the last two dimensions of the tensor are transposed. If the tensor is rank-1 of shape \([D]\), then for \(x\) it is treated as \([1, D]\) in nontransposed form and as \([D, 1]\) in transposed form, whereas for \(y\) it is the opposite: It is treated as \([D, 1]\) in nontransposed form and as \([1, D]\) in transposed form.

  • After transpose, the two tensors are 2-D or n-D and matrix multiplication performs in the following way.

    • If both are 2-D, they are multiplied like conventional matrices.

    • If either is n-D, it is treated as a stack of matrices residing in the last two dimensions and a batched matrix multiply supporting broadcast applies on the two tensors.

Also note that if the raw tensor \(x\) or \(y\) is rank-1 and nontransposed, the prepended or appended dimension \(1\) will be removed after matrix multiplication.

Parameters
  • x (Variable) – The input variable which is a Tensor or LoDTensor.

  • y (Variable) – The input variable which is a Tensor or LoDTensor.

  • transpose_x (bool) – Whether to transpose \(x\) before multiplication.

  • transpose_y (bool) – Whether to transpose \(y\) before multiplication.

  • alpha (float) – The scale of output. Default 1.0.

  • name (str|optional) – A name for this layer(optional). If set None, the layer will be named automatically.

Returns

The product Tensor (or LoDTensor) variable.

Return type

Variable

Examples

# Examples to clarify shapes of the inputs and output
# x: [B, ..., M, K], y: [B, ..., K, N]
# fluid.layers.matmul(x, y)
# out: [B, ..., M, N]

# x: [B, M, K], y: [B, K, N]
# fluid.layers.matmul(x, y)
# out: [B, M, N]

# x: [B, M, K], y: [K, N]
# fluid.layers.matmul(x, y)
# out: [B, M, N]

# x: [M, K], y: [K, N]
# fluid.layers.matmul(x, y)
# out: [M, N]

# x: [B, M, K], y: [K]
# fluid.layers.matmul(x, y)
# out: [B, M]

# x: [K], y: [K]
# fluid.layers.matmul(x, y)
# out: [1]

# x: [M], y: [N]
# fluid.layers.matmul(x, y, True, True)
# out: [M, N]

import paddle.fluid as fluid
import numpy

# Graph Organizing
x = fluid.data(name='x', shape=[2, 3], dtype='float32')
y = fluid.data(name='y', shape=[3, 2], dtype='float32')
output = fluid.layers.matmul(x, y, True, True)

# Create an executor using CPU as an example
exe = fluid.Executor(fluid.CPUPlace())

# Execute
input_x = numpy.ones([2, 3]).astype(numpy.float32)
input_y = numpy.ones([3, 2]).astype(numpy.float32)
res, = exe.run(fluid.default_main_program(),
               feed={'x':input_x, 'y':input_y},
               fetch_list=[output])
print(res)
'''
Output Value:
[[2. 2. 2.]
 [2. 2. 2.]
 [2. 2. 2.]]
'''

maxout

paddle.fluid.layers.maxout(x, groups, name=None)[source]

MaxOut Operator.

Assumed the input shape is (N, Ci, H, W). The output shape is (N, Co, H, W). Then \(Co = Ci / groups\) and the operator formula is as follows:

$$ y_{si+j} = max_{k} x_{gsi + sk + j} $$ $$ g = groups $$ $$ s = \frac{input.size}{num\_channels} $$ $$ 0 \le i < \frac{num\_channels}{groups} $$ $$ 0 \le j < s $$ $$ 0 \le k < groups $$

Please refer to Paper: - Maxout Networks: http://www.jmlr.org/proceedings/papers/v28/goodfellow13.pdf - Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks: https://arxiv.org/pdf/1312.6082v4.pdf

Parameters
  • x (Variable) – (Tensor) The input tensor of maxout operator with data type of float32. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of feature

  • groups (INT) – (int),Specifies how many groups the input tensor will be splitin the channel dimension. And the number of output channel is the number of channels divided by groups

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

out(Variable): (Tensor) The output tensor of maxout operator.The data type is float32.The format of output tensor is also NCHW.Where N is batch size, C is the number of channels, H and W is the height and width of feature

Return type

Variable

Examples

import paddle.fluid as fluid
input = fluid.data(
    name='data',
    shape=[None, 256, 32, 32],
    dtype='float32')
out = fluid.layers.maxout(input, groups=2)

mean

paddle.fluid.layers.mean(x, name=None)[source]

Mean Operator calculates the mean of all elements in X.

Parameters
  • x (Variable) – (Tensor) The input of mean op

  • name (basestring|None) – Name of the output.

Returns

(Tensor) The output of mean op

Return type

out(Variable)

Examples

import paddle.fluid as fluid
import numpy

# Graph Organizing
input = fluid.data(
    name='data', shape=[2, 3], dtype='float32')
output = fluid.layers.mean(input)

# Create an executor using CPU as an example
place = fluid.CPUPlace()
exe = fluid.Executor(place)

# Execute
x_ndarray = numpy.ones([2, 3]).astype(numpy.float32)
res, = exe.run(fluid.default_main_program(),
               feed={'data':x_ndarray},
               fetch_list=[output])
print(res)
'''
Output Value:
[1.]
'''

mean_iou

paddle.fluid.layers.mean_iou(input, label, num_classes)[source]

Mean Intersection-Over-Union is a common evaluation metric for semantic image segmentation, which first computes the IOU for each semantic class and then computes the average over classes. IOU is defined as follows:

\[IOU = \frac{true\_positive}{(true\_positive + false\_positive + false\_negative)}.\]

The predictions are accumulated in a confusion matrix and mean-IOU is then calculated from it.

Parameters
  • input (Variable) – A n-D Tensor of prediction results for semantic labels with type int32 or int64.

  • label (Variable) – A Tensor of ground truth labels with type int32 or int64. Its shape should be the same as input.

  • num_classes (int32) – The possible number of labels.

Returns

Three Variables.

  • mean_iou(Variable) : A 1-D Tensor representing the mean intersection-over-union with shape [1]. Data type is float32.

  • out_wrong(Variable) : A 1-D Tensor with shape [num_classes]. Data type is int32. The wrong numbers of each class.

  • out_correct(Variable): A 1-D Tensor with shape [num_classes]. Data type is int32. The correct numbers of each class.

Examples

import paddle.fluid as fluid
iou_shape = [None, 32, 32]
num_classes = 5
predict = fluid.data(name='predict', shape=iou_shape, dtype='int64')
label = fluid.data(name='label', shape=iou_shape, dtype='int64')
mean_iou, out_wrong, out_correct = fluid.layers.mean_iou(predict, label,
                                              num_classes)

merge_selected_rows

paddle.fluid.layers.merge_selected_rows(x, name=None)[source]

MergeSelectedRows Operator.

MergeSelectedRows is used to merge the duplicated rows of the input. The output’s row has no duplicated, and it’s order is incremental.

Example: Input: X.rows is [0, 5, 5, 4, 19] X.height is 20 X.value is: [[1, 1] [2, 2] [3, 3] [4, 4] [6, 6]]

Output: Out.row is [0, 4, 5, 19] Out.height is 20 Out.value is: [[1, 1] [4, 4] [5, 5] [6, 6]]

Parameters
  • x (Variable) – The input type is SelectedRows, and the selected rows may be duplicated

  • name (basestring|None) – Name of the output.

Returns

The output type is SelectedRows, and the selected rows are not duplicated

Return type

out(Variable)

Examples

import paddle.fluid as fluid
import numpy

place = fluid.CPUPlace()
block = fluid.default_main_program().global_block()

var = block.create_var(name="X2",
                       dtype="float32",
                       persistable=True,
                       type=fluid.core.VarDesc.VarType.SELECTED_ROWS)
y = fluid.layers.merge_selected_rows(var)
z = fluid.layers.get_tensor_from_selected_rows(y)

x_rows = [0, 2, 2, 4, 19]
row_numel = 2
np_array = numpy.ones((len(x_rows), row_numel)).astype("float32")

x = fluid.global_scope().var("X2").get_selected_rows()
x.set_rows(x_rows)
x.set_height(20)
x_tensor = x.get_tensor()
x_tensor.set(np_array, place)

exe = fluid.Executor(place=place)
result = exe.run(fluid.default_main_program(), fetch_list=[z])

print("x_rows: ", x_rows)
print("np_array: ", np_array)
print("result: ", result)
'''
Output Values:
('x_rows: ', [0, 2, 2, 4, 19])
('np_array: ', array([[1., 1.],
       [1., 1.],
       [1., 1.],
       [1., 1.],
       [1., 1.]], dtype=float32))
('result: ', [array([[1., 1.],
       [2., 2.],
       [1., 1.],
       [1., 1.]], dtype=float32)])
'''

mul

paddle.fluid.layers.mul(x, y, x_num_col_dims=1, y_num_col_dims=1, name=None)[source]

Mul Operator. This operator is used to perform matrix multiplication for input $x$ and $y$. The equation is:

\[Out = x * y\]

Both the input $x$ and $y$ can carry the LoD (Level of Details) information, or not. But the output only shares the LoD information with input $x$.

Parameters
  • x (Variable) – The first input Tensor/LoDTensor of mul_op.

  • y (Variable) – The second input Tensor/LoDTensor of mul_op.

  • x_num_col_dims (int, optional) – The mul_op can take tensors with more than two dimensions as its inputs. If the input $x$ is a tensor with more than two dimensions, $x$ will be flattened into a two-dimensional matrix first. The flattening rule is: the first num_col_dims will be flattened to form the first dimension of the final matrix (the height of the matrix), and the rest rank(x) - num_col_dims dimensions are flattened to form the second dimension of the final matrix (the width of the matrix). As a result, height of the flattened matrix is equal to the product of $x$’s first x_num_col_dims dimensions’ sizes, and width of the flattened matrix is equal to the product of $x$’s last rank(x) - num_col_dims dimensions’ size. For example, suppose $x$ is a 6-dimensional tensor with the shape [2, 3, 4, 5, 6], and x_num_col_dims = 3. Thus, the flattened matrix will have a shape [2 x 3 x 4, 5 x 6] = [24, 30]. Default is 1.

  • y_num_col_dims (int, optional) – The mul_op can take tensors with more than two dimensions as its inputs. If the input $y$ is a tensor with more than two dimensions, $y$ will be flattened into a two-dimensional matrix first. The attribute y_num_col_dims determines how $y$ is flattened. See comments of x_num_col_dims for more details. Default is 1.

  • name (str, optional) – Name of the output. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name. Default is None.

Returns

The output Tensor/LoDTensor of mul op.

Return type

Variable(Tensor/LoDTensor)

Examples

import paddle.fluid as fluid
dataX = fluid.layers.data(name="dataX", append_batch_size = False, shape=[2, 5], dtype="float32")
dataY = fluid.layers.data(name="dataY", append_batch_size = False, shape=[5, 3], dtype="float32")
output = fluid.layers.mul(dataX, dataY,
                          x_num_col_dims = 1,
                          y_num_col_dims = 1)

multiplex

paddle.fluid.layers.multiplex(inputs, index)[source]

Based on the given index parameter, the OP selects a specific row from each input Tensor to construct the output Tensor.

If the input of this OP contains \(m\) Tensors, where \(I_{i}\) means the i-th input Tensor, \(i\) between \([0,m)\) .

And \(O\) means the output, where \(O[i]\) means the i-th row of the output, then the output satisfies that \(O[i] = I_{index[i]}[i]\) .

For Example:

Given:

inputs = [[[0,0,3,4], [0,1,3,4], [0,2,4,4], [0,3,3,4]],
          [[1,0,3,4], [1,1,7,8], [1,2,4,2], [1,3,3,4]],
          [[2,0,3,4], [2,1,7,8], [2,2,4,2], [2,3,3,4]],
          [[3,0,3,4], [3,1,7,8], [3,2,4,2], [3,3,3,4]]]

index = [[3],[0],[1],[2]]

out = [[3,0,3,4],    # out[0] = inputs[index[0]][0] = inputs[3][0] = [3,0,3,4]
       [0,1,3,4],    # out[1] = inputs[index[1]][1] = inputs[0][1] = [0,1,3,4]
       [1,2,4,2],    # out[2] = inputs[index[2]][2] = inputs[1][2] = [1,2,4,2]
       [2,3,3,4]]    # out[3] = inputs[index[3]][3] = inputs[2][3] = [2,3,3,4]
Parameters
  • inputs (list) – The input Tensor list. The list elements are N-D Tensors of data types float32, float64, int32, int64. All input Tensor shapes should be the same and rank must be at least 2.

  • index (Variable) – Used to select some rows in the input Tensor to construct an index of the output Tensor. It is a 2-D Tensor with data type int32 or int64 and shape [M, 1], where M is the number of input Tensors.

Returns

Output of multiplex OP, with data type being float32, float64, int32, int64.

Return type

Variable(Tensor)

Examples

import paddle.fluid as fluid
import numpy as np

x1 = fluid.data(name='x1', shape=[None, 2], dtype='float32')
x2 = fluid.data(name='x2', shape=[None, 2], dtype='float32')
index = fluid.data(name='index', shape=[None, 1], dtype='int32')
out = fluid.layers.multiplex(inputs=[x1, x2], index=index)

exe = fluid.Executor(fluid.CPUPlace())
exe.run(fluid.default_startup_program())

img1 = np.array([[1, 2], [3, 4]]).astype(np.float32)
img2 = np.array([[5, 6], [7, 8]]).astype(np.float32)
index = np.array([[1], [0]]).astype(np.int32)

res = exe.run(fluid.default_main_program(), feed={'x1':img1, 'x2':img2, 'index':index}, fetch_list=[out])
print(res) # [array([[5., 6.], [3., 4.]], dtype=float32)]

nce

paddle.fluid.layers.nce(input, label, num_total_classes, sample_weight=None, param_attr=None, bias_attr=None, num_neg_samples=None, name=None, sampler='uniform', custom_dist=None, seed=0, is_sparse=False)[source]

Compute and return the noise-contrastive estimation training loss. See Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. By default this operator uses a uniform distribution for sampling.

Parameters
  • input (Variable) – Input variable, 2-D tensor with shape [batch_size, dim], and data type is float32 or float64.

  • label (Variable) – Input label, 2-D tensor with shape [batch_size, num_true_class], and data type is int64.

  • num_total_classes (int) – Total number of classes in all samples.

  • sample_weight (Variable|None) – A Variable of shape [batch_size, 1] storing a weight for each sample. The default weight for each sample is 1.0.

  • param_attr (ParamAttr|None) – To specify the weight parameter attribute. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • bias_attr (ParamAttr|None) – To specify the bias parameter attribute. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .

  • num_neg_samples (int) – The number of negative classes. The default value is 10.

  • name (str|None) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.

  • sampler (str, optional) – The sampler used to sample class from negtive classes. It can be ‘uniform’, ‘log_uniform’ or ‘custom_dist’. default: ‘uniform’.

  • custom_dist (nd.array|None) – A numpy ndarray with size=num_total_classes. It is used when sampler is set to ‘custom_dist’. custom_dist[i] is the probsbility of i-th class to be sampled. default: None.

  • seed (int, optional) – The seed used in sampler. Default 0, means no random seed.

  • is_sparse (bool, optional) – The flag indicating whether to use sparse update, the weight@GRAD and bias@GRAD will be changed to SelectedRows. Default False.

Returns

The output nce loss.

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np

window_size = 5
words = []
for i in xrange(window_size):
    words.append(fluid.data(
        name='word_{0}'.format(i), shape=[-1, 1], dtype='int64'))

dict_size = 10000
label_word = int(window_size / 2) + 1

embs = []
for i in xrange(window_size):
    if i == label_word:
        continue

    emb = fluid.layers.embedding(input=words[i], size=[dict_size, 32],
                       param_attr='embed', is_sparse=True)
    embs.append(emb)

embs = fluid.layers.concat(input=embs, axis=1)
loss = fluid.layers.nce(input=embs, label=words[label_word],
          num_total_classes=dict_size, param_attr='nce.w_0',
          bias_attr='nce.b_0')

 #or use custom distribution
 dist = np.array([0.05,0.5,0.1,0.3,0.05])
 loss = fluid.layers.nce(input=embs, label=words[label_word],
           num_total_classes=5, param_attr='nce.w_1',
           bias_attr='nce.b_1',
           num_neg_samples=3,
           sampler="custom_dist",
           custom_dist=dist)

npair_loss

paddle.fluid.layers.npair_loss(anchor, positive, labels, l2_reg=0.002)[source]

Npair Loss Layer

Read Improved Deep Metric Learning with Multi class N pair Loss Objective .

Npair loss requires paired data. Npair loss has two parts: the first part is L2 regularizer on the embedding vector; the second part is cross entropy loss which takes the similarity matrix of anchor and positive as logits.

Parameters
  • anchor (Variable) – embedding vector for the anchor image. shape=[batch_size, embedding_dims], the data type is float32 or float64.

  • positive (Variable) – embedding vector for the positive image. shape=[batch_size, embedding_dims], the data type is float32 or float64.

  • labels (Variable) – 1-D tensor. shape=[batch_size], the data type is float32 or float64 or int64.

  • l2_reg (float32) – L2 regularization term on embedding vector, default: 0.002.

Returns

A Variable holding Tensor representing the npair loss, the data type is the same as anchor, the shape is [1].

Examples

import paddle.fluid as fluid
anchor = fluid.data(
              name = 'anchor', shape = [18, 6], dtype = 'float32')
positive = fluid.data(
              name = 'positive', shape = [18, 6], dtype = 'float32')
labels = fluid.data(
              name = 'labels', shape = [18], dtype = 'float32')

npair_loss = fluid.layers.npair_loss(anchor, positive, labels, l2_reg = 0.002)

one_hot

paddle.fluid.layers.one_hot(input, depth, allow_out_of_range=False)[source]

WARING: This OP requires the last dimension of Tensor shape must be equal to 1. This OP will be deprecated in a future release. It is recommended to use fluid. api_fluid_one_hot .

The operator converts each id in the input to an one-hot vector with a depth length. The value in the vector dimension corresponding to the id is 1, and the value in the remaining dimension is 0.

The shape of output Tensor or LoDTensor is generated by adding depth dimension behind the last dimension of the input shape.

Example 1 (allow_out_of_range=False):

input:
    X.shape = [4, 1]
    X.data = [[1], [1], [3], [0]]
    depth = 4

output:
    Out.shape = [4, 4]
    Out.data = [[0., 1., 0., 0.],
                [0., 1., 0., 0.],
                [0., 0., 0., 1.],
                [1., 0., 0., 0.]]

Example 2 (allow_out_of_range=True):

input:
    X.shape = [4, 1]
    X.data = [[1], [1], [5], [0]]
    depth = 4
    allow_out_of_range = True

output:
    Out.shape = [4, 4]
    Out.data = [[0., 1., 0., 0.],
                [0., 1., 0., 0.],
                [0., 0., 0., 0.], # This id is 5, which goes beyond depth, so set it all-zeros data.
                [1., 0., 0., 0.]]

Example 3 (allow_out_of_range=False):

input:
    X.shape = [4, 1]
    X.data = [[1], [1], [5], [0]]
    depth = 4
    allow_out_of_range = False

output: Throw an exception for Illegal value
    The second dimension in X is 5, which is greater than depth.
    Allow_out_of_range =False means that does not allow the word id to exceed depth,
    so it throws an exception.
Parameters
  • input (Variable) – Tensor or LoDTensor with shape \([N_1, N_2, ..., N_k, 1]\) , which contains at least one dimension and the last dimension must be 1. The data type is int32 or int64.

  • depth (scalar) – An integer defining the depth of the one hot dimension. If input is word id, depth is generally the dictionary size.

  • allow_out_of_range (bool) – A bool value indicating whether the input indices could be out of range \([0, depth)\) . When input indices are out of range, exceptions Illegal value is raised if allow_out_of_range is False, or zero-filling representations is created if it is set True. Default: False.

Returns

The one-hot representations of input. A Tensor or LoDTensor with type float32.

Return type

Variable

Examples

import paddle.fluid as fluid
# Correspond to the first example above, where label.shape is [4, 1] and one_hot_label.shape is [4, 4].
label = fluid.data(name="label", shape=[4, 1], dtype="int64")
one_hot_label = fluid.layers.one_hot(input=label, depth=4)

pad

paddle.fluid.layers.pad(x, paddings, pad_value=0.0, name=None)[source]

This op will pad a tensor with a constant value given by pad_value, and the padded shape is specified by paddings.

Specifically, the number of values padded before the elements of x in dimension i is indicated by paddings[2*i], and the number of values padded after the elements of x in dimension i is indicated by paddings[2*i+1].

See below for an example.

Given:
    x = [[1, 2], [3, 4]]

    paddings = [0, 1, 1, 2]

    pad_value = 0

Return:

    out = [[0, 1, 2, 0, 0]
           [0, 3, 4, 0, 0]
           [0, 0, 0, 0, 0]]
Parameters
  • x (Variable) – Tensor, data type is float32.

  • paddings (list) – A list of integers. Its elements specify the padded width before and after each dimension in turn. The length of paddings must be equal to \(rank(x) \times 2\).

  • pad_value (float) – The constant value used to pad.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

The padded tensor, with the same data type and rank as x

Return Type:

Variable

Examples

# x is a rank 2 tensor variable with shape [100, 224].
# out will be a tensor of shape [101, 227]
import paddle.fluid as fluid
x = fluid.data(name='data', shape=[100, 224], dtype='float32')
out = fluid.layers.pad(
    x=x, paddings=[0, 1, 1, 2], pad_value=0.)

pad2d

paddle.fluid.layers.pad2d(input, paddings=[0, 0, 0, 0], mode='constant', pad_value=0.0, data_format='NCHW', name=None)[source]

Pad 2-d images accordding to ‘paddings’ and ‘mode’. If mode is ‘reflect’, paddings[0] and paddings[1] must be no greater than height-1. And the width dimension has the same condition.

Parameters
  • input (Variable) – The input image with [N, C, H, W] format or [N, H, W, C] format, which is a 4-D Tensor with data type float32.

  • paddings (Variable | List[int32]) – The padding size. If padding is a List, it must contain four integers, (padding_top, padding_bottom, padding_left, padding_right). Otherwise, it is a 1-D Tensor with shape [4]. Data type is int32. Default is [0, 0, 0, 0].

  • mode (str) – Three modes: ‘constant’ (default), ‘reflect’, ‘edge’ . When in ‘constant’ mode, this op uses a constant value to pad the input tensor. When in ‘reflect’ mode, uses reflection of the input boundaries to pad the input tensor. When in ‘edge’ mode, uses input boundaries to pad the input tensor. Default is ‘constant’

  • pad_value (float32) – The value to fill the padded areas in ‘constant’ mode . Default is 0.0

  • data_format (str) – An string from: “NHWC”, “NCHW”. Specify the data format of the input data. Default is “NCHW”

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns: a 4-D Tensor padded accordding to paddings and mode and data type is same as input.

Return Type: Variable

Examples

Given that X is a channel of image from input:

X = [[1, 2, 3],
     [4, 5, 6]]

Case 0:

  paddings = [0, 1, 2, 3],
  mode = 'constant'
  pad_value = 0

  Out = [[0, 0, 1, 2, 3, 0, 0, 0]
         [0, 0, 4, 5, 6, 0, 0, 0]
         [0, 0, 0, 0, 0, 0, 0, 0]]

Case 1:

  paddings = [0, 1, 2, 1],
  mode = 'reflect'

  Out = [[3, 2, 1, 2, 3, 2]
         [6, 5, 4, 5, 6, 5]
         [3, 2, 1, 2, 3, 2]]

Case 2:

  paddings = [0, 1, 2, 1],
  mode = 'edge'

  Out = [[1, 1, 1, 2, 3, 3]
         [4, 4, 4, 5, 6, 6]
         [4, 4, 4, 5, 6, 6]]
Code Examples:
import paddle.fluid as fluid
data = fluid.data(name='data', shape=[None, 3, 32, 32],
                         dtype='float32')
result = fluid.layers.pad2d(input=data, paddings=[1, 2, 3, 4],
                            mode='reflect')

pad_constant_like

paddle.fluid.layers.pad_constant_like(x, y, pad_value=0.0, name=None)[source]

Pad y with pad_value, the number of values padded to the edges of each axis is specified by the difference of the shape of x and y . ((0, shape_x_0 - shape_y_0), … (0, shape_x_n - shape_y_n)) specify padding widths for each axis. The input should be a k-D tensor(k > 0 and k < 7).

See below for an example.

Given:
    X = [[[[ 0,  1,  2],
           [ 3,  4,  5]],
          [[ 6,  7,  8],
           [ 9, 10, 11]],
          [[12, 13, 14],
           [15, 16, 17]]],
         [[[18, 19, 20],
           [21, 22, 23]],
          [[24, 25, 26],
           [27, 28, 29]],
          [[30, 31, 32],
           [33, 34, 35]]]]
    X.shape = (2, 3, 2, 3)

    Y = [[[[35, 36, 37]],
          [[38, 39, 40]],
          [[41, 42, 43]]]]
    Y.shape = (1, 3, 1, 3)
        And
    pad_value = -1,

Return:
    Out = [[[[35, 36, 37],
             [-1, -1, -1]],
            [[38, 39, 40],
             [-1, -1, -1]],
            [[41, 42, 43],
             [-1, -1, -1]]],
          [[[-1, -1, -1],
            [-1, -1, -1]],
           [[-1, -1, -1],
            [-1, -1, -1]],
           [[-1, -1, -1],
            [-1, -1, -1]]]]
    Out.shape = (2, 3, 2, 3)
Parameters
  • x (Variable) – Tensor, its shape spicifies the shape of output.

  • y (Variable) – Tensor, its rank is the same with x, and for each dimension \(i\) , \(y\_shape[i] <= x\_shape[i]\) . The data type can be float32 or float64.

  • pad_value (float) – The constant value used to pad.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

The padded tensor, with the same shape as x and the same data type as y

Return Type:

Variable

Examples

# x is a rank 4 tensor variable, x.shape = (2, 3, 2, 3)
# y is a rank 4 tensor variable, y.shape = (1, 3, 1, 3)
import paddle.fluid as fluid
x = fluid.data(name='x', shape=[2,3,2,3], dtype='float32')
y = fluid.data(name='y', shape=[1,3,1,3], dtype='float32')
out = fluid.layers.pad_constant_like(x=x, y=y, pad_value=0.)
# out is a rank 4 tensor variable, and out.shape = [2, 3 ,2 , 3]

pixel_shuffle

paddle.fluid.layers.pixel_shuffle(x, upscale_factor)[source]

This op rearranges elements in a tensor of shape [N, C, H, W] to a tensor of shape [N, C/r**2, H*r, W*r]. This is useful for implementing efficient sub-pixel convolution with a stride of 1/r. Please refer to the paper: Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network . by Shi et. al (2016) for more details.

Parameters
  • x (Variable) – 4-D tensor, the data type should be float32 or float64.

  • upscale_factor (int) – factor to increase spatial resolution.

Returns

Reshaped tensor according to the new dimension.

Return type

Out(Variable)

Raises

ValueError – If the square of upscale_factor cannot divide the channels of input.

Examples

# declarative mode
import paddle.fluid as fluid
import numpy as np
input = fluid.data(name="input", shape=[2,9,4,4])
output = fluid.layers.pixel_shuffle(x=input, upscale_factor=3)
place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

input_data = np.random.rand(2,9,4,4).astype("float32")
output_data = exe.run(fluid.default_main_program(),
    feed={"input":input_data},
    fetch_list=[output],
    return_numpy=True)

# print(output.shape)
# (2L, 1L, 12L, 12L)

pool2d

paddle.fluid.layers.pool2d(input, pool_size=-1, pool_type='max', pool_stride=1, pool_padding=0, global_pooling=False, use_cudnn=True, ceil_mode=False, name=None, exclusive=True, data_format='NCHW')[source]

This operation calculates the pooling output based on the input, pooling_type and pool_size, pool_stride, pool_padding parameters. Input(X) and Output(Out) are in NCHW or NHWC format, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Parameters(pool_size, pool_stride, pool_padding) hold two integer elements. These two elements represent height and width, respectively. The input(X) size and output(Out) size may be different.

Example:

Input:

X shape: \((N, C, H_{in}, W_{in})\)

Output:

Out shape: \((N, C, H_{out}, W_{out})\)

For pool_padding = “SAME”: $$ H_{out} = \frac{(H_{in} + strides[0] - 1)}{strides[0]} $$ $$ W_{out} = \frac{(W_{in} + strides[1] - 1)}{strides[1]} $$

For pool_padding = “VALID”: $$ H_{out} = \frac{(H_{in} - ksize[0] + strides[0])}{strides[0]} $$ $$ W_{out} = \frac{(W_{in} - ksize[1] + strides[1])}{strides[1]} $$

For ceil_mode = false: $$ H_{out} = \frac{(H_{in} - ksize[0] + pad_height_top + pad_height_bottom}{strides[0]} + 1 $$ $$ W_{out} = \frac{(W_{in} - ksize[1] + pad_width_left + pad_width_right}{strides[1]} + 1 $$

For ceil_mode = true: $$ H_{out} = \frac{(H_{in} - ksize[0] + pad_height_top + pad_height_bottom + strides[0] - 1)}{strides[0]} + 1 $$ $$ W_{out} = \frac{(W_{in} - ksize[1] + pad_width_left + pad_width_right + strides[1] - 1)}{strides[1]} + 1 $$

For exclusive = false: $$ hstart = i * strides[0] - pad_height_top $$ $$ hend = hstart + ksize[0] $$ $$ wstart = j * strides[1] - pad_width_left $$ $$ wend = wstart + ksize[1] $$ $$ Output(i ,j) = \frac{sum(Input[hstart:hend, wstart:wend])}{ksize[0] * ksize[1]} $$

For exclusive = true: $$ hstart = max(0, i * strides[0] - pad_height_top) $$ $$ hend = min(H, hstart + ksize[0]) $$ $$ wstart = max(0, j * strides[1] - pad_width_left) $$ $$ wend = min(W, wstart + ksize[1]) $$ $$ Output(i ,j) = \frac{sum(Input[hstart:hend, wstart:wend])}{(hend - hstart) * (wend - wstart)} $$

Parameters
  • input (Variable) – The input tensor of pooling operator which is a 4-D tensor with shape [N, C, H, W]. The format of input tensor is “NCHW” or “NHWC”, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. The data type if float32 or float64.

  • pool_size (int|list|tuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain two integers, (pool_size_Height, pool_size_Width). Otherwise, the pool kernel size will be a square of an int.

  • pool_type – (string), pooling type, can be “max” for max-pooling and “avg” for average-pooling

  • pool_stride (int|list|tuple) – The pool stride size. If pool stride size is a tuple or list, it must contain two integers, (pool_stride_Height, pool_stride_Width). Otherwise, the pool stride size will be a square of an int.

  • pool_padding (string|int|list|tuple) – The pool padding. If pool_padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If pool padding size is a tuple or list, it could be in three forms: [pad_height, pad_width] or [pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCHW”, pool_padding can be in the form [[0,0], [0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NHWC”, pool_padding can be in the form [[0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Otherwise, the pool padding size will be a square of an int.

  • global_pooling (bool) – (bool) Whether to use the global pooling. If global_pooling = true, kernel size and paddings will be ignored. Default False

  • use_cudnn (bool) – (bool) Only used in cudnn kernel, need install cudnn. Default False

  • ceil_mode (bool) – (bool) Whether to use the ceil function to calculate output height and width. False is the default. If it is set to False, the floor function will be used. Default False

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

  • exclusive (bool) – Whether to exclude padding points in average pooling mode, default is true.

  • data_format (string) – The data format of the input and output data. An optional string from: “NCHW”, “NDHW”. The default is “NCHW”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width].

Returns

The output tensor of pooling result. The data type is same as input tensor.

Return type

Variable

Raises
  • ValueError – If pool_type is not “max” nor “avg”

  • ValueError – If global_pooling is False and pool_size is -1

  • ValueError – If use_cudnn is not a bool value.

Examples

import paddle.fluid as fluid

data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32')

# max pool2d
pool2d = fluid.layers.pool2d(
  input = data,
  pool_size = 2,
  pool_type = "max",
  pool_stride = 1,
  global_pooling=False)

# average pool2d
pool2d = fluid.layers.pool2d(
  input = data,
  pool_size = 2,
  pool_type = "avg",
  pool_stride = 1,
  global_pooling=False)

# global average pool2d
pool2d = fluid.layers.pool2d(
  input = data,
  pool_size = 2,
  pool_type = "avg",
  pool_stride = 1,
  global_pooling=True)

# Attr(pool_padding) is a list with 4 elements, Attr(data_format) is "NCHW".
out_1 = fluid.layers.pool2d(
  input = data,
  pool_size = 3,
  pool_type = "avg",
  pool_stride = 1,
  pool_padding = [1, 2, 1, 0],
  data_format = "NCHW")

# Attr(pool_padding) is a string, Attr(data_format) is "NCHW".
out_2 = fluid.layers.pool2d(
  input = data,
  pool_size = 3,
  pool_type = "avg",
  pool_stride = 1,
  pool_padding = "VALID",
  data_format = "NCHW")

pool3d

paddle.fluid.layers.pool3d(input, pool_size=-1, pool_type='max', pool_stride=1, pool_padding=0, global_pooling=False, use_cudnn=True, ceil_mode=False, name=None, exclusive=True, data_format='NCDHW')[source]

This operation calculates the output based on the input, pooling_type, pool_size, pool_stride, and pool_padding parameters. Input(X) and output(Out) are in NCDHW or NDHWC format, where N is batch size, C is the number of channels, and D, H and W are the depth, height and width of the feature, respectively. Parameters(pool_size, pool_stride, pool_padding) hold three integer elements. These three elements represent depth, height and width, respectively. The input(X) size and output(Out) size may be different.

Example: Input: X shape: \((N, C, D_{in}, H_{in}, W_{in})\) Output: Out shape: \((N, C, D_{out}, H_{out}, W_{out})\)

For pool_padding = “SAME”: $$ D_{out} = \frac{(D_{in} + strides[0] - 1)}{strides[0]} $$ $$ H_{out} = \frac{(H_{in} + strides[1] - 1)}{strides[1]} $$ $$ W_{out} = \frac{(W_{in} + strides[2] - 1)}{strides[2]} $$

For pool_padding = “VALID”: $$ D_{out} = \frac{(D_{in} - ksize[0] + strides[0])}{strides[0]} $$ $$ H_{out} = \frac{(H_{in} - ksize[1] + strides[1])}{strides[1]} $$ $$ W_{out} = \frac{(W_{in} - ksize[2] + strides[2])}{strides[2]} $$

For ceil_mode = false: $$ D_{out} = \frac{(D_{in} - ksize[0] + pad_depth_front + pad_depth_back)}{strides[0]} + 1 $$ $$ H_{out} = \frac{(H_{in} - ksize[1] + pad_height_top + pad_height_bottom)}{strides[1]} + 1 $$ $$ W_{out} = \frac{(W_{in} - ksize[2] + pad_width_left + pad_width_right)}{strides[2]} + 1 $$ For ceil_mode = true: $$ D_{out} = \frac{(D_{in} - ksize[0] + pad_depth_front + pad_depth_back + strides[0] -1)}{strides[0]} + 1 $$ $$ H_{out} = \frac{(H_{in} - ksize[1] + pad_height_top + pad_height_bottom + strides[1] -1)}{strides[1]} + 1 $$ $$ W_{out} = \frac{(W_{in} - ksize[2] + pad_width_left + pad_width_right + strides[2] -1)}{strides[2]} + 1 $$

For exclusive = false: $$ dstart = i * strides[0] - pad_depth_front $$ $$ dend = dstart + ksize[0] $$ $$ hstart = j * strides[1] - pad_height_top $$ $$ hend = hstart + ksize[1] $$ $$ wstart = k * strides[2] - pad_width_left $$ $$ wend = wstart + ksize[2] $$ $$ Output(i ,j, k) = \frac{sum(Input[dstart:dend, hstart:hend, wstart:wend])}{ksize[0] * ksize[1] * ksize[2]} $$

For exclusive = true: $$ dstart = max(0, i * strides[0] - pad_depth_front) $$ $$ dend = min(D, dstart + ksize[0]) $$ $$ hstart = max(0, j * strides[1] - pad_height_top) $$ $$ hend = min(H, hstart + ksize[1]) $$ $$ wstart = max(0, k * strides[2] - pad_width_left) $$ $$ wend = min(W, wstart + ksize[2]) $$ $$ Output(i ,j, k) = \frac{sum(Input[dstart:dend, hstart:hend, wstart:wend])}{(dend - dstart) * (hend - hstart) * (wend - wstart)} $$

Parameters
  • input (Variable) – The input tensor of pooling operator, which is a 5-D tensor with shape [N, C, D, H, W]. The format of input tensor is “NCDHW” or “NDHWC”, where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.

  • pool_size (int|list|tuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain three integers, (pool_size_Depth, pool_size_Height, pool_size_Width). Otherwise, the pool kernel size will be the cube of an int.

  • pool_type (string) – (string) Pooling type, can be “max” for max-pooling and “avg” for average-pooling

  • pool_stride (string|int|list|tuple)) – The pool padding. If pool_padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If pool stride size is a tuple or list, it must contain three integers, [stride_Depth, stride_Height, stride_Width]. Otherwise, the pool stride size will be a cube of an int.

  • pool_padding (int|list|tuple) – The pool padding size. If pool padding size is a tuple or list, it could be in three forms: [pad_depth, pad_height, pad_width] or [pad_depth_front, pad_depth_back, pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCDHW”, pool_padding can be in the form [[0,0], [0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NDHWC”, pool_padding can be in the form [[0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]].

  • global_pooling (bool) – (bool) Whether to use the global pooling. If global_pooling = true, kernel size and paddings will be ignored. Default False

  • use_cudnn (bool) – (bool) Only used in cudnn kernel, need install cudnn. Default False

  • ceil_mode (bool) – (bool) Whether to use the ceil function to calculate output height and width. False is the default. If it is set to False, the floor function will be used. Default False

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

  • exclusive (bool) – Whether to exclude padding points in average pooling mode, default is true.

  • data_format (string) – The data format of the input and output data. An optional string from: “NCDHW”, “NDHWC”. The default is “NCDHW”. When it is “NCDHW”, the data is stored in the order of: [batch_size, input_channels, input_depth, input_height, input_width].

Returns

The output tensor of pooling result. The data type is same as input tensor.

Return type

Variable

Examples

import paddle.fluid as fluid

data = fluid.data(name='data', shape=[None, 3, 32, 32, 32], dtype='float32')

# max pool3d
pool3d = fluid.layers.pool3d(
  input = data,
  pool_size = 2,
  pool_type = "max",
  pool_stride = 1,
  global_pooling=False)

# average pool3d
pool3d = fluid.layers.pool3d(
  input = data,
  pool_size = 2,
  pool_type = "avg",
  pool_stride = 1,
  global_pooling=False)

# global average pool3d
pool3d = fluid.layers.pool3d(
  input = data,
  pool_size = 2,
  pool_type = "avg",
  pool_stride = 1,
  global_pooling=True)

# example 1:
# Attr(pool_padding) is a list with 6 elements, Attr(data_format) is "NCDHW".
out_1 = fluid.layers.pool3d(
  input = data,
  pool_size = 2,
  pool_type = "avg",
  pool_stride = 1,
  pool_padding = [1, 2, 1, 0, 1, 2],
  global_pooling = False,
  data_format = "NCDHW")

# example 2:
# Attr(pool_padding) is a string, Attr(data_format) is "NCDHW".
out_2 = fluid.layers.pool3d(
  input = data,
  pool_size = 3,
  pool_type = "avg",
  pool_stride = 1,
  pool_padding = "VALID",
  global_pooling = False,
  data_format = "NCDHW")

pow

paddle.fluid.layers.pow(x, factor=1.0, name=None)[source]

This is Pow Activation Operator.

\(out = x^{factor}\)

Parameters
  • x (Variable) – A Tensor or LoDTensor . The data type is float32 or float64.

  • factor (float32|Variable, optional) – A scalar with type float32 or a Tensor with shape [1] and type float32. The exponential factor of Pow. Default 1.0.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

A Tensor or LoDTensor. The data type is same as x.

Return type

Variable

Examples

import paddle.fluid as fluid

x = fluid.data(name="x", shape=[32,32], dtype="float32")

# example 1: argument factor is float
y_1 = fluid.layers.pow(x, factor=2.0)
# y_1 is x^{2.0}

# example 2: argument factor is Variable
factor_tensor = fluid.layers.fill_constant([1], "float32", 3.0)
y_2 = fluid.layers.pow(x, factor=factor_tensor)
# y_2 is x^{3.0}

prelu

paddle.fluid.layers.prelu(x, mode, param_attr=None, name=None)[source]

Equation:

\[y = \max(0, x) + \alpha * \min(0, x)\]

There are three modes for the activation:

all: All elements share same alpha.
channel: Elements in same channel share same alpha.
element: All elements do not share alpha. Each element has its own alpha.
Parameters
  • x (Variable) – The input Tensor or LoDTensor with data type float32.

  • mode (str) – The mode for weight sharing.

  • param_attr (ParamAttr|None) – The parameter attribute for the learnable weight (alpha), it can be create by ParamAttr. None by default. For detailed information, please refer to ParamAttr.

  • name (str|None) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

output(Variable): The tensor or LoDTensor with the same shape as input. The data type is float32.

Return type

Variable

Examples

import paddle.fluid as fluid
from paddle.fluid.param_attr import ParamAttr
x = fluid.data(name="x", shape=[None,5,10,10], dtype="float32")
mode = 'channel'
output = fluid.layers.prelu(
         x,mode,param_attr=ParamAttr(name='alpha'))

psroi_pool

paddle.fluid.layers.psroi_pool(input, rois, output_channels, spatial_scale, pooled_height, pooled_width, name=None)[source]

PSROIPool Operator, rois of this op should be a LoDTensor

Position sensitive region of interest pooling (also known as PSROIPooling) is to perform position-sensitive average pooling on regions of interest specified by input, takes as input N position-sensitive score maps and a list of num_rois regions of interest.

PSROIPooling for R-FCN. Please refer to https://arxiv.org/abs/1605.06409 for more details.

Parameters
  • input (Variable) – Tensor, the input of PSROIPoolOp. The format of input tensor is NCHW. Where N is the batch size, C is the number of input channels, H is the height of the input feature map, and W is the width. The data type can be float32 or float64

  • rois (Variable) – LoDTensor, ROIs (Regions of Interest) to pool over.It should be a 2-D LoDTensor of shape (num_rois, 4), the lod level is 1. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates. The data type is the same as input

  • output_channels (int) – (int), the number of channels of the output feature map. For a task of C classes of objects, output_channels should be (C + 1) for classification only

  • spatial_scale (float) – (float, default 1.0), Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling Default: 1.0

  • pooled_height (int) – (int, default 1), the pooled output height Default: 1

  • pooled_width (int) – (int, default 1), the pooled output width Default: 1

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Tensor, the output of PSROIPoolOp is a 4-D Tensor with shape (num_rois, output_channels, pooled_h, pooled_w). The data type is the same as x .

Return Type:

Variable

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[100, 490, 28, 28], dtype='float32')
rois = fluid.data(name='rois', shape=[None, 4], lod_level=1, dtype='float32')
pool_out = fluid.layers.psroi_pool(x, rois, 10, 1.0, 7, 7)

py_func

paddle.fluid.layers.py_func(func, x, out, backward_func=None, skip_vars_in_backward_input=None)[source]

This API is used to register customized OP to Fluid. The forward function of the registered OP is func and the backward function of that is backward_func. Paddle will call func at forward runtime and call backward_func at backward runtime(if backward_func is not None). x is the input of func, whose type must be LoDTensor; out is the output of func, whose type can be either LoDTensor or NumPy array.

The input of the backward function backward_func is x, out and the gradient of out. If some variables of out have no gradient, the relevant input variable of backward_func is None. If some variables of x do not have a gradient, the user should return None in backward_func.

The data type and shape of out should also be set correctly before this API is called, and the data type and shape of the gradient of out and x will be inferred automatically.

This API can also be used to debug the neural network by setting the func as a function that only print variables.

Parameters
  • func (callable) – The forward function of the registered OP. When the network is running, the forward output out will be calculated according to this function and the forward input x.

  • x (Variable) – The input of the forward function func, its type can be Variable | tuple[Variable] | list[Variale], in which Variable is LoDTensor.

  • out (Variable) – The output of the forward function func, its type can be Variable | tuple[Variable] | list[Variale], in which Variable can be either LoDTensor or NumPy array. Since Paddle cannot automatically infer the shape and data type of out, out must be created in advance.

  • backward_func (callable, optional) – The backward function of the registered OP. Its default value is None, which means there is no reverse calculation. If it is not None, backward_func is called to calculate the gradient of x when the network is at backward runtime.

  • skip_vars_in_backward_input (Variable, optional) – It’s used to limit the input variable list of backward_func, and it can be single Variable, tuple[Variable] or list[Variable]. It must belong to either x or out. The default value is None, which means that no variables need to be removed from x and out. If it is not None, these variables will not be the input of backward_func. This parameter is only useful when backward_func is not None.

Returns

The output out of the forward function func.

Return type

Variable

Examples

import paddle.fluid as fluid
import six

def create_tmp_var(name, dtype, shape):
return fluid.default_main_program().current_block().create_var(
name=name, dtype=dtype, shape=shape)

# Tanh activation function provided by Paddle C++ op
# Here, tanh is used as an example to show how to use py_func
def tanh(x):
    return np.tanh(x)

# Skip forward input x
def tanh_grad(y, dy):
    return np.array(dy) * (1 - np.square(np.array(y)))

def debug_func(x):
    print(x)

def simple_net(img, label):
    hidden = img
    for idx in six.moves.range(4):
        hidden = fluid.layers.fc(hidden, size=200)
        new_hidden = create_tmp_var(name='hidden_{}'.format(idx),
            dtype=hidden.dtype, shape=hidden.shape)

        # User-defined forward and backward
        hidden = fluid.layers.py_func(func=tanh, x=hidden,
            out=new_hidden, backward_func=tanh_grad,
            skip_vars_in_backward_input=hidden)

        # User-defined debugging layer, which can print out variable details
        fluid.layers.py_func(func=debug_func, x=hidden, out=None)

    prediction = fluid.layers.fc(hidden, size=10, act='softmax')
    loss = fluid.layers.cross_entropy(input=prediction, label=label)
    return fluid.layers.mean(loss)

random_crop

paddle.fluid.layers.random_crop(x, shape, seed=None)[source]

This operator takes a batch of instance, and do random cropping on each instance. It means that cropping positions differs on each instance, which is determined by an uniform random generator. All cropped instances have the same shape, which is determined by the operator’s attribute ‘shape’.

Parameters
  • x (Variable) – A batch of instances to random crop

  • shape (INTS) – The shape of a cropped instance

  • seed (int|Variable|None) – The random seed By default, the seed will get from random.randint(-65536, 65535).

Returns

The cropped instance batch

Examples

import paddle.fluid as fluid
img = fluid.data("img", [None, 3, 256, 256])
# cropped_img is [-1, 3, 224, 224]
cropped_img = fluid.layers.random_crop(img, shape=[3, 224, 224])

# cropped_img2 shape: [-1, 2, 224, 224]
# cropped_img2 = fluid.layers.random_crop(img, shape=[2, 224, 224])

# cropped_img3 shape: [-1, 3, 128, 224]
# cropped_img3 = fluid.layers.random_crop(img, shape=[128, 224])

rank

paddle.fluid.layers.rank(input)[source]

The OP returns the number of dimensions for a tensor, which is a 0-D int32 Tensor.

Parameters

input (Variable) – The input N-D tensor with shape of \([N_1, N_2, ..., N_k]\), the data type is arbitrary.

Returns

The 0-D tensor with the dimensions of the input variable.

Return type

Variable, the output data type is int32.

Examples

import paddle.fluid as fluid

input = fluid.data(name="input", shape=[3, 100, 100], dtype="float32")
rank = fluid.layers.rank(input) # rank=(3,)

rank_loss

paddle.fluid.layers.rank_loss(label, left, right, name=None)[source]

This operator implements the sort loss layer in the RankNet model. RankNet is a pairwise ranking model with a training sample consisting of a pair of documents (A and B), The label (P) indicates whether A is ranked higher than B or not. Please refer to more details: RankNet

Rank loss layer takes three inputs: left ( \(o_i\) ), right ( \(o_j\) ) and label ( \(P_{i,j}\) ). The inputs respectively represent RankNet’s output scores for documents A and B and the value of label P. Rank loss layer takes batch inputs with size batch_size (batch_size >= 1), P = {0, 1} or {0, 0.5, 1}, where 0.5 means that there is no information about the rank of the input pair. The following equation computes rank loss C_{i,j} from the inputs:

\[\begin{split}C_{i,j} &= -\tilde{P_{ij}} * o_{i,j} + \log(1 + e^{o_{i,j}}) \\\end{split}\]
\[\begin{split}o_{i,j} &= o_i - o_j \\\end{split}\]
\[\tilde{P_{i,j}} &= \left \{0, 0.5, 1 \right \} \ or \ \left \{0, 1 \right \}\]
Parameters
  • label (Variable) – 2-D Tensor with the shape of \([batch,1]\), the data type is float32, batch indicates the size of the data. Indicats whether A ranked higher than B or not.

  • left (Variable) – 2-D Tensor with the shape of \([batch,1]\), the data type is float32. RankNet’s output score for doc A.

  • right (Variable) – 2-D Tensor with the shape of \([batch,1]\), the data type is float32. RankNet’s output score for doc B.

  • name (str|None) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

Tensor indicating the output value of the sort loss layer, the data type is float32, and the return value’s shape is \([batch,1]\) .

Return type

Variable

Raises

ValueError – Any of label, left, and right is not a Variable .

Examples

import paddle.fluid as fluid
label = fluid.data(name="label", shape=[-1, 1], dtype="float32")
left = fluid.data(name="left", shape=[-1, 1], dtype="float32")
right = fluid.data(name="right", shape=[-1, 1], dtype="float32")
out = fluid.layers.rank_loss(label, left, right)

reduce_all

paddle.fluid.layers.reduce_all(input, dim=None, keep_dim=False, name=None)[source]

This OP computes the logical and of tensor elements over the given dimension, and output the result.

Parameters
  • input (Variable) – The input variable which is a Tensor or LoDTensor, the input data type should be bool.

  • dim (list|int|optional) – The dimension along which the logical and is computed. If None, compute the logical and over all elements of input and return a Tensor variable with a single element, otherwise must be in the range \([-rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\). The default value is None.

  • keep_dim (bool) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the input unless keep_dim is true. The default value is False.

  • name (str|None) – A name for this layer(optional). If set None, the layer will be named automatically. The default value is None.

Returns

The reduced tensor variable with logical and in given dims.

Return type

Variable, the output data type is bool.

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers
import numpy as np

# x is a bool Tensor variable with following elements:
#    [[True, False]
#     [True, True]]
x = layers.assign(np.array([[1, 0], [1, 1]], dtype='int32'))
x = layers.cast(x, 'bool')

out = layers.reduce_all(x)  # False
out = layers.reduce_all(x, dim=0)  # [True, False]
out = layers.reduce_all(x, dim=-1)  # [False, True]
# keep_dim=False, x.shape=(2,2), out.shape=(2,)

out = layers.reduce_all(x, dim=1, keep_dim=True)  # [[False], [True]]
# keep_dim=True, x.shape=(2,2), out.shape=(2,1)

reduce_any

paddle.fluid.layers.reduce_any(input, dim=None, keep_dim=False, name=None)[source]

This OP computes the logical or of tensor elements over the given dimension, and output the result.

Parameters
  • input (Variable) – The input variable which is a Tensor or LoDTensor, the input data type should be bool.

  • dim (list|int|optional) – The dimension along which the logical and is computed. If None, compute the logical and over all elements of input and return a Tensor variable with a single element, otherwise must be in the range \([-rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\). The default value is None.

  • keep_dim (bool) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the input unless keep_dim is true. The default value is False.

  • name (str|None) – A name for this layer(optional). If set None, the layer

Returns

The reduced tensor variable with logical or in given dims.

Return type

Variable, the output data type is bool.

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers
import numpy as np

# x is a bool Tensor variable with following elements:
#    [[True, False]
#     [False, False]]
x = layers.assign(np.array([[1, 0], [0, 0]], dtype='int32'))
x = layers.cast(x, 'bool')

out = layers.reduce_any(x)  # True
out = layers.reduce_any(x, dim=0)  # [True, False]
out = layers.reduce_any(x, dim=-1)  # [True, False]
# keep_dim=False, x.shape=(2,2), out.shape=(2,)

out = layers.reduce_any(x, dim=1,
                         keep_dim=True)  # [[True], [False]]
# keep_dim=True, x.shape=(2,2), out.shape=(2,1)

reduce_max

paddle.fluid.layers.reduce_max(input, dim=None, keep_dim=False, name=None)[source]

Computes the maximum of tensor elements over the given dimension.

Parameters
  • input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.

  • dim (list|int, optional) – The dimension along which the maximum is computed. If None, compute the maximum over all elements of input and return a Tensor variable with a single element, otherwise must be in the range \([-rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).

  • keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the input unless keep_dim is true, default value is False.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Tensor, results of maximum on the specified dim of input tensor, it’s data type is the same as input’s Tensor.

Return type

Variable

Examples

import paddle.fluid as fluid
# x is a Tensor variable with following elements:
#    [[0.2, 0.3, 0.5, 0.9]
#     [0.1, 0.2, 0.6, 0.7]]
# Each example is followed by the correspending output tensor.
x = fluid.data(name='x', shape=[2, 4], dtype='float32')
fluid.layers.reduce_max(x)  # [0.9]
fluid.layers.reduce_max(x, dim=0)  # [0.2, 0.3, 0.6, 0.9]
fluid.layers.reduce_max(x, dim=-1)  # [0.9, 0.7]
fluid.layers.reduce_max(x, dim=1, keep_dim=True)  # [[0.9], [0.7]]

# y is a Tensor variable with shape [2, 2, 2] and elements as below:
#      [[[1.0, 2.0], [3.0, 4.0]],
#      [[5.0, 6.0], [7.0, 8.0]]]
# Each example is followed by the correspending output tensor.
y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32')
fluid.layers.reduce_max(y, dim=[1, 2]) # [4.0, 8.0]
fluid.layers.reduce_max(y, dim=[0, 1]) # [7.0, 8.0]

reduce_mean

paddle.fluid.layers.reduce_mean(input, dim=None, keep_dim=False, name=None)[source]

Computes the mean of the input tensor’s elements along the given dimension.

Parameters
  • input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.

  • dim (list|int, optional) – The dimension along which the mean is computed. If None, compute the mean over all elements of input and return a variable with a single element, otherwise it must be in the range \([-rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank(input) + dim[i]\).

  • keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the input unless keep_dim is true, default value is False.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Tensor, results of average on the specified dim of input tensor, it’s data type is the same as input’s Tensor.

Return type

Variable

Raises

TypeError, if out data type is different with the input data type.

Examples

import paddle.fluid as fluid
# x is a Tensor variable with following elements:
#    [[0.2, 0.3, 0.5, 0.9]
#     [0.1, 0.2, 0.6, 0.7]]
# Each example is followed by the correspending output tensor.
x = fluid.data(name='x', shape=[2, 4], dtype='float32')
fluid.layers.reduce_mean(x)  # [0.4375]
fluid.layers.reduce_mean(x, dim=0)  # [0.15, 0.25, 0.55, 0.8]
fluid.layers.reduce_mean(x, dim=-1)  # [0.475, 0.4]
fluid.layers.reduce_mean(x, dim=1, keep_dim=True)  # [[0.475], [0.4]]

# y is a Tensor variable with shape [2, 2, 2] and elements as below:
#      [[[1.0, 2.0], [3.0, 4.0]],
#      [[5.0, 6.0], [7.0, 8.0]]]
# Each example is followed by the correspending output tensor.
y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32')
fluid.layers.reduce_mean(y, dim=[1, 2]) # [2.5, 6.5]
fluid.layers.reduce_mean(y, dim=[0, 1]) # [4.0, 5.0]

reduce_min

paddle.fluid.layers.reduce_min(input, dim=None, keep_dim=False, name=None)[source]

Computes the minimum of tensor elements over the given dimension.

Parameters
  • input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.

  • dim (list|int, optional) – The dimensions along which the minimum is computed. If None, compute the minimum over all elements of input and return a Tensor variable with a single element, otherwise must be in the range \([-rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).

  • keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the input unless keep_dim is true, default value is False.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Tensor, result of minimum on the specified dim of input tensor, it’s data type is the same as input’s Tensor.

Return type

Variable

Examples

import paddle.fluid as fluid
# x is a Tensor variable with following elements:
#    [[0.2, 0.3, 0.5, 0.9]
#     [0.1, 0.2, 0.6, 0.7]]
# Each example is followed by the correspending output tensor.
x = fluid.data(name='x', shape=[2, 4], dtype='float32')
fluid.layers.reduce_min(x)  # [0.1]
fluid.layers.reduce_min(x, dim=0)  # [0.1, 0.2, 0.5, 0.7]
fluid.layers.reduce_min(x, dim=-1)  # [0.2, 0.1]
fluid.layers.reduce_min(x, dim=1, keep_dim=True)  # [[0.2], [0.1]]

# y is a Tensor variable with shape [2, 2, 2] and elements as below:
#      [[[1.0, 2.0], [3.0, 4.0]],
#      [[5.0, 6.0], [7.0, 8.0]]]
# Each example is followed by the correspending output tensor.
y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32')
fluid.layers.reduce_min(y, dim=[1, 2]) # [1.0, 5.0]
fluid.layers.reduce_min(y, dim=[0, 1]) # [1.0, 2.0]

reduce_prod

paddle.fluid.layers.reduce_prod(input, dim=None, keep_dim=False, name=None)[source]

Computes the product of tensor elements over the given dimension.

Parameters
  • input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.

  • dim (list|int, optional) – The dimensions along which the product is performed. If None, multipy all elements of input and return a Tensor variable with a single element, otherwise must be in the range \([-rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).

  • keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the input unless keep_dim is true, default value is False.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Tensor, result of product on the specified dim of input tensor, it’s data type is the same as input’s Tensor.

Return type

Variable

Examples

import paddle.fluid as fluid
# x is a Tensor variable with following elements:
#    [[0.2, 0.3, 0.5, 0.9]
#     [0.1, 0.2, 0.6, 0.7]]
# Each example is followed by the correspending output tensor.
x = fluid.data(name='x', shape=[2, 4], dtype='float32')
fluid.layers.reduce_prod(x)  # [0.0002268]
fluid.layers.reduce_prod(x, dim=0)  # [0.02, 0.06, 0.3, 0.63]
fluid.layers.reduce_prod(x, dim=-1)  # [0.027, 0.0084]
fluid.layers.reduce_prod(x, dim=1,
                         keep_dim=True)  # [[0.027], [0.0084]]

# y is a Tensor variable with shape [2, 2, 2] and elements as below:
#      [[[1.0, 2.0], [3.0, 4.0]],
#      [[5.0, 6.0], [7.0, 8.0]]]
# Each example is followed by the correspending output tensor.
y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32')
fluid.layers.reduce_prod(y, dim=[1, 2]) # [24.0, 1680.0]
fluid.layers.reduce_prod(y, dim=[0, 1]) # [105.0, 384.0]

reduce_sum

paddle.fluid.layers.reduce_sum(input, dim=None, keep_dim=False, name=None)[source]

Computes the sum of tensor elements over the given dimension.

Parameters
  • input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.

  • dim (list|int, optional) – The dimensions along which the sum is performed. If None, sum all elements of input and return a Tensor variable with a single element, otherwise must be in the range \([-rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).

  • keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the input unless keep_dim is true, default value is False.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Tensor, results of summation operation on the specified dim of input tensor, it’s data type is the same as input’s Tensor.

Return type

Variable

Raises

TypeError, if out data type is different with the input data type.

Examples

import paddle.fluid as fluid
# x is a Tensor variable with following elements:
#    [[0.2, 0.3, 0.5, 0.9]
#     [0.1, 0.2, 0.6, 0.7]]
# Each example is followed by the corresponding output tensor.
x = fluid.data(name='x', shape=[2, 4], dtype='float32')
fluid.layers.reduce_sum(x)  # [3.5]
fluid.layers.reduce_sum(x, dim=0)  # [0.3, 0.5, 1.1, 1.6]
fluid.layers.reduce_sum(x, dim=-1)  # [1.9, 1.6]
fluid.layers.reduce_sum(x, dim=1, keep_dim=True)  # [[1.9], [1.6]]

# y is a Tensor variable with shape [2, 2, 2] and elements as below:
#      [[[1, 2], [3, 4]],
#      [[5, 6], [7, 8]]]
# Each example is followed by the corresponding output tensor.
y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32')
fluid.layers.reduce_sum(y, dim=[1, 2]) # [10, 26]
fluid.layers.reduce_sum(y, dim=[0, 1]) # [16, 20]

relu

paddle.fluid.layers.relu(x, name=None)[source]

Relu Activation Operator.

\(out = \max(x, 0)\)

Parameters
  • x (Variable) – Input of Relu operator, an N-D Tensor, with data type float32, float64 or float16

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.

Returns

Output of Relu operator, a Tensor with shape same as input

Return type

Variable

Examples

import paddle.fluid as fluid
import numpy as np
in1 = np.array([[-1,0],[1,2.6]])
with fluid.dygraph.guard():
    x1 = fluid.dygraph.to_variable(in1)
    out1 = fluid.layers.relu(x1)
    print(out1.numpy())
    # [[0.  0. ]
    #  [1.  2.6]]

relu6

paddle.fluid.layers.relu6(x, threshold=6.0, name=None)[source]

Relu6 Activation Operator.

\(out = \min(\max(0, x), threshold)\)

Parameters
  • x (Variable) – Input of relu6 operator, an N-D Tensor, with data type float32, float64

  • threshold (float, optional) – The threshold value of Relu6. Default is 6.0.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.

Returns

Output of relu6 operator, a Tensor with the same shape as input

Return type

output(Variable)

Examples

import paddle.fluid as fluid
import numpy as np
in1 = np.array([[-1,0],[2.5,7.8]])
with fluid.dygraph.guard():
    x1 = fluid.dygraph.to_variable(in1)
    out1 = fluid.layers.relu6(x=x1, threshold=6.0)
    print(out1.numpy())
    # [[0.  0. ]
    #  [2.5 6. ]]

reshape

paddle.fluid.layers.reshape(x, shape, actual_shape=None, act=None, inplace=False, name=None)[source]

This operator changes the shape of x without changing its data.

The target shape can be given by shape or actual_shape. When shape and actual_shape are set at the same time, actual_shape has a higher priority than shape but at this time shape can only be an integer list or tuple, and shape still should be set correctly to gurantee shape inference in compile-time.

Some tricks exist when specifying the target shape.

1. -1 means the value of this dimension is inferred from the total element number of x and remaining dimensions. Thus one and only one dimension can be set -1.

2. 0 means the actual dimension value is going to be copied from the corresponding dimension of x. The indice of 0s in shape can not exceed the dimension of x.

Here are some examples to explain it.

1. Given a 3-D tensor x with a shape [2, 4, 6], and the target shape is [6, 8], the reshape operator will transform x into a 2-D tensor with shape [6, 8] and leaving x’s data unchanged.

2. Given a 3-D tensor x with a shape [2, 4, 6], and the target shape specified is [2, 3, -1, 2], the reshape operator will transform x into a 4-D tensor with shape [2, 3, 4, 2] and leaving x’s data unchanged. In this case, one dimension of the target shape is set to -1, the value of this dimension is inferred from the total element number of x and remaining dimensions.

3. Given a 3-D tensor x with a shape [2, 4, 6], and the target shape is [-1, 0, 3, 2], the reshape operator will transform x into a 4-D tensor with shape [2, 4, 3, 2] and leaving x’s data unchanged. In this case, besides -1, 0 means the actual dimension value is going to be copied from the corresponding dimension of x.

Note:

The parameter actual_shape will be deprecated in the future and only use shape instead to represent the target shape.

Parameters
  • x (Variable) – A Tensor or LoDTensor . The data type is float32, float64, int32 or int64.

  • shape (list|tuple|Variable) – Define the target shape. At most one dimension of the target shape can be -1. The data type is int32 . If shape is a list or tuple, the elements of it should be integers or Tensors with shape [1]. If shape is an Variable, it should be an 1-D Tensor .

  • actual_shape (variable, optional) – An 1-D Tensor or LoDTensor . The data type is int32 . If provided, reshape according to this given shape rather than shape specifying shape. That is to say actual_shape has a higher priority than shape(list|tuple) but not shape(Variable). This argument actual_shape will be removed in a future version. Instructions for updating: actual_shape will be removed in future versions and replaced by shape.

  • act (str, optional) – The non-linear activation to be applied to the reshaped input. Default None.

  • inplace (bool, optional) – If inplace is True, the input and output of layers.reshape are the same variable. Otherwise, the input and output of layers.reshape are different variable. Default False. Note that if x is more than one OPs’ input, inplace must be False.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

A Tensor or LoDTensor. The data type is same as x. It is a new tensor variable if inplace is False, otherwise it is x. If act is None, return the reshaped tensor variable, otherwise return the activated tensor variable.

Return type

Variable

Raises
  • TypeError – If actual_shape is neither Variable nor None.

  • ValueError – If more than one elements of shape is -1.

  • ValueError – If the element of shape is 0, the corresponding dimension should be less than or equal to the dimension of x.

  • ValueError – If the elements in shape is negative except -1.

Examples

import paddle.fluid as fluid

# example 1:
# attr shape is a list which doesn't contain tensor Variable.
data_1 = fluid.data(
  name='data_1', shape=[2, 4, 6], dtype='float32')
reshaped_1 = fluid.layers.reshape(
  x=data_1, shape=[-1, 0, 3, 2], inplace=True)
# the shape of reshaped_1 is [2,4,3,2].

# example 2:
# attr shape is a list which contains tensor Variable.
data_2 = fluid.layers.fill_constant([2,25], "int32", 3)
dim = fluid.layers.fill_constant([1], "int32", 5)
reshaped_2 = fluid.layers.reshape(data_2, shape=[dim, 10])
# the shape of reshaped_2 is [5,10].

resize_bilinear

paddle.fluid.layers.resize_bilinear(input, out_shape=None, scale=None, name=None, actual_shape=None, align_corners=True, align_mode=1, data_format='NCHW')[source]

This op resizes the input by performing bilinear interpolation based on given output shape which specified by actual_shape, out_shape and scale in priority order.

Warning: the parameter actual_shape will be deprecated in the future and only use out_shape instead.

Bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables (e.g. H-direction and W-direction in this op) on a rectilinear 2D grid. The key idea is to perform linear interpolation first in one direction, and then again in the other direction.

For details of bilinear interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation

Align_corners and align_mode are optinal parameters,the calculation method of interpolation can be selected by them.

Example:

For scale:

    if align_corners = True && out_size > 1 :

      scale_factor = (in_size-1.0)/(out_size-1.0)

    else:

      scale_factor = float(in_size/out_size)

Bilinear interpolation:

  if:
      align_corners = False , align_mode = 0

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:

      H_out = (H_{in}+0.5) * scale_{factor} - 0.5
      W_out = (W_{in}+0.5) * scale_{factor} - 0.5

  else:

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:
      H_out = H_{in} * scale_{factor}
      W_out = W_{in} * scale_{factor}
Parameters
  • input (Variable) – 4-D Tensor(NCHW), its data type is float32, float64, or uint8, its data format is specified by data_format.

  • out_shape (list|tuple|Variable|None) – Output shape of resize bilinear layer, the shape is (out_h, out_w).Default: None. If a list, each element can be an integer or a Tensor Variable with shape: [1]. If a Tensor Variable, its dimension size should be 1.

  • scale (float|Variable|None) – The multiplier for the input height or width. At least one of out_shape or scale must be set. And out_shape has a higher priority than scale. Default: None.

  • actual_shape (Variable) – An optional input to specify output shape dynamically. If provided, image resize according to this given shape rather than out_shape and scale specifying shape. That is to say actual_shape has the highest priority. It is recommended to use out_shape if you want to specify output shape dynamically, because actual_shape will be deprecated. When using actual_shape to specify output shape, one of out_shape and scale should also be set, otherwise errors would be occured in graph constructing stage. Default: None

  • align_corners (bool) – an optional bool. Defaults to True. If True, the centers of 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels, If False, are not aligned

  • align_mode (bool) – (int, default ‘1’), optional for bilinear interpolation, can be ‘0’ for src_idx = scale*(dst_indx+0.5)-0.5 , can be ‘1’ for src_idx = scale*dst_index

  • data_format (str, optional) – NCHW(num_batches, channels, height, width) or NHWC(num_batches, height, width, channels). Default: ‘NCHW’.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

4-D tensor(NCHW or NHWC).

Return type

Variable

Examples

#declarative mode
import paddle.fluid as fluid
import numpy as np
input = fluid.data(name="input", shape=[None,3,6,10])

#1
output = fluid.layers.resize_bilinear(input=input,out_shape=[12,12])

#2
#x = np.array([2]).astype("int32")
#dim1 = fluid.data(name="dim1", shape=[1], dtype="int32")
#fluid.layers.assign(input=x, output=dim1)
#output = fluid.layers.resize_bilinear(input=input,out_shape=[12,dim1])

#3
#x = np.array([3,12]).astype("int32")
#shape_tensor = fluid.data(name="shape_tensor", shape=[2], dtype="int32")
#fluid.layers.assign(input=x, output=shape_tensor)
#output = fluid.layers.resize_bilinear(input=input,out_shape=shape_tensor)

#4
#x = np.array([0.5]).astype("float32")
#scale_tensor = fluid.data(name="scale", shape=[1], dtype="float32")
#fluid.layers.assign(x,scale_tensor)
#output = fluid.layers.resize_bilinear(input=input,scale=scale_tensor)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

input_data = np.random.rand(2,3,6,10).astype("float32")

output_data = exe.run(fluid.default_main_program(),
    feed={"input":input_data},
    fetch_list=[output],
    return_numpy=True)

print(output_data[0].shape)

#1
# (2, 3, 12, 12)
#2
# (2, 3, 12, 2)
#3
# (2, 3, 3, 12)
#4
# (2, 3, 3, 5)

#imperative mode
import paddle.fluid.dygraph as dg

with dg.guard(place) as g:
    input = dg.to_variable(input_data)
    output = fluid.layers.resize_bilinear(input=input, out_shape=[12,12])
    print(output.shape)

    # [2L, 3L, 12L, 12L]

resize_nearest

paddle.fluid.layers.resize_nearest(input, out_shape=None, scale=None, name=None, actual_shape=None, align_corners=True, data_format='NCHW')[source]

This op resizes the input by performing nearest neighbor interpolation in both the height direction and the width direction based on given output shape which is specified by actual_shape, out_shape and scale in priority order.

Warning: the parameter actual_shape will be deprecated in the future and only use out_shape instead.

Example:

For scale:

    if align_corners = True && out_size > 1 :
      scale_factor = (in_size-1.0)/(out_size-1.0)

    else:

      scale_factor = float(in_size/out_size)

Nearest neighbor interpolation:

  if:
      align_corners = False

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:

      H_out = floor(H_{in} * scale_{factor})
      W_out = floor(W_{in} * scale_{factor})

  else:
      align_corners = True

      input : (N,C,H_in,W_in)
      output: (N,C,H_out,W_out) where:

      H_out = round(H_{in} * scale_{factor})
      W_out = round(W_{in} * scale_{factor})

For details of nearest neighbor interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation

Parameters
  • input (Variable) – 4-D Tensor, its data type is float32, float64, or uint8, its data format is specified by data_format.

  • out_shape (list|tuple|Variable|None) – The output shape of resized tensor, the shape is (out_h, out_w). Default: None. Every element should be an integer or a tensor Variable with shape: [1] if it is a list. If it is a tensor Variable, its dimension size should be 1.

  • scale (float|Variable|None) – The multiplier for the input height or width. At least one of out_shape or scale must be set. And out_shape has a higher priority than scale. Default: None.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

  • actual_shape (Variable) – An optional input to specify output shape dynamically. If provided, image resize according to this given shape rather than out_shape and scale specifying shape. That is to say actual_shape has the highest priority. It is recommended to use out_shape if you want to specify output shape dynamically, because actual_shape will be deprecated. When using actual_shape to specify output shape, one of out_shape and scale should also be set, otherwise errors would be occured in graph constructing stage. Default: None

  • align_corners (bool) – an optional bool. Defaults to True. If True, the centers of 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels, If False, are not aligned

  • data_format (str, optional) – NCHW(num_batches, channels, height, width) or NHWC(num_batches, height, width, channels). Default: ‘NCHW’.

Returns

4-D tensor(NCHW or NHWC).

Return type

Variable

Examples

#declarative mode
import paddle.fluid as fluid
import numpy as np
input = fluid.data(name="input", shape=[None,3,6,10])

#1
output = fluid.layers.resize_nearest(input=input,out_shape=[12,12])

#2
#x = np.array([2]).astype("int32")
#dim1 = fluid.data(name="dim1", shape=[1], dtype="int32")
#fluid.layers.assign(input=x, output=dim1)
#output = fluid.layers.resize_nearest(input=input,out_shape=[12,dim1])

#3
#x = np.array([3,12]).astype("int32")
#shape_tensor = fluid.data(name="shape_tensor", shape=[2], dtype="int32")
#fluid.layers.assign(input=x, output=shape_tensor)
#output = fluid.layers.resize_nearest(input=input,out_shape=shape_tensor)

#4
#x = np.array([0.5]).astype("float32")
#scale_tensor = fluid.data(name="scale", shape=[1], dtype="float32")
#fluid.layers.assign(x,scale_tensor)
#output = fluid.layers.resize_nearest(input=input,scale=scale_tensor)

place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

input_data = np.random.rand(2,3,6,10).astype("float32")

output_data = exe.run(fluid.default_main_program(),
    feed={"input":input_data},
    fetch_list=[output],
    return_numpy=True)

print(output_data[0].shape)

#1
# (2, 3, 12, 12)
#2
# (2, 3, 12, 2)
#3
# (2, 3, 3, 12)
#4
# (2, 3, 3, 5)

#imperative mode
import paddle.fluid.dygraph as dg

with dg.guard(place) as g:
    input = dg.to_variable(input_data)
    output = fluid.layers.resize_nearest(input=input, out_shape=[12,12])
    print(output.shape)

    # [2L, 3L, 12L, 12L]

roi_align

paddle.fluid.layers.roi_align(input, rois, pooled_height=1, pooled_width=1, spatial_scale=1.0, sampling_ratio=-1, name=None)[source]

RoIAlign Operator

Region of interest align (also known as RoI align) is to perform bilinear interpolation on inputs of nonuniform sizes to obtain fixed-size feature maps (e.g. 7*7)

Dividing each region proposal into equal-sized sections with the pooled_width and pooled_height. Location remains the origin result.

In each ROI bin, the value of the four regularly sampled locations are computed directly through bilinear interpolation. The output is the mean of four locations. Thus avoid the misaligned problem.

Parameters
  • input (Variable) – (Tensor), The input of ROIAlignOp. The data type is float32 or float64.The format of input tensor is NCHW. Where N is batch size, C is the number of input channels, H is the height of the feature, and W is the width of the feature

  • rois (Variable) – ROIs (Regions of Interest) to pool over.It should be a 2-D LoDTensor of shape (num_rois, 4), the lod level is 1. The data type is float32 or float64. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates.

  • pooled_height (int32, optional) – (int, default 1), The pooled output height Default: 1

  • pooled_width (int32, optional) – (int, default 1), The pooled output width Default: 1

  • spatial_scale (float32, optional) – (float, default 1.0), Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling Default: 1.0

  • sampling_ratio (int32, optional) – (int,default -1),number of sampling points in the interpolation gridIf <=0, then grid points are adaptive to roi_width and pooled_w, likewise for height Default: -1

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns

Output: (Tensor), The output of ROIAlignOp is a 4-D tensor with shape (num_rois, channels, pooled_h, pooled_w). The data type is float32 or float64.

Return type

Variable

Examples

import paddle.fluid as fluid
x = fluid.data(
    name='data', shape=[None, 256, 32, 32], dtype='float32')
rois = fluid.data(
    name='rois', shape=[None, 4], dtype='float32')
align_out = fluid.layers.roi_align(input=x,
                                   rois=rois,
                                   pooled_height=7,
                                   pooled_width=7,
                                   spatial_scale=0.5,
                                   sampling_ratio=-1)

roi_pool

paddle.fluid.layers.roi_pool(input, rois, pooled_height=1, pooled_width=1, spatial_scale=1.0)[source]

This operator implements the roi_pooling layer. Region of interest pooling (also known as RoI pooling) is to perform max pooling on inputs of nonuniform sizes to obtain fixed-size feature maps (e.g. 7*7).

The operator has three steps:

  1. Dividing each region proposal into equal-sized sections with the pooled_width and pooled_height;

  2. Finding the largest value in each section;

  3. Copying these max values to the output buffer.

For more information, please refer to https://stackoverflow.com/questions/43430056/what-is-roi-layer-in-fast-rcnn

Parameters
  • input (Variable) – Input feature, 4D-Tensor with the shape of [N,C,H,W], where N is the batch size, C is the input channel, H is Height, W is weight. The data type is float32 or float64.

  • rois (Variable) – ROIs (Regions of Interest) to pool over. 2D-LoDTensor with the shape of [num_rois,4], the lod level is 1. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates.

  • pooled_height (int, optional) – The pooled output height, data type is int32. Default: 1

  • pooled_width (int, optional) – The pooled output height, data type is int32. Default: 1

  • spatial_scale (float, optional) – Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling. Default: 1.0

Returns

The pooled feature, 4D-Tensor with the shape of [num_rois, C, pooled_height, pooled_width].

Return type

Variable

Examples:

import paddle.fluid as fluid
import numpy as np

DATATYPE='float32'

place = fluid.CPUPlace()
#place = fluid.CUDAPlace(0)

input_data = np.array([i for i in range(1,17)]).reshape(1,1,4,4).astype(DATATYPE)
roi_data =fluid.create_lod_tensor(np.array([[1., 1., 2., 2.], [1.5, 1.5, 3., 3.]]).astype(DATATYPE),[[2]], place)

x = fluid.data(name='input', shape=[None,1,4,4], dtype=DATATYPE)
rois = fluid.data(name='roi', shape=[None,4], dtype=DATATYPE)

pool_out = fluid.layers.roi_pool(
        input=x,
        rois=rois,
        pooled_height=1,
        pooled_width=1,
        spatial_scale=1.0)

exe = fluid.Executor(place)
out, = exe.run(feed={'input':input_data ,'roi':roi_data}, fetch_list=[pool_out.name])
print(out)   #array([[[[11.]]], [[[16.]]]], dtype=float32)
print(np.array(out).shape)  # (2, 1, 1, 1)

row_conv

paddle.fluid.layers.row_conv(input, future_context_size, param_attr=None, act=None)[source]

Row-convolution operator

The row convolution is called lookahead convolution. This operator was introduced in the following paper for DeepSpeech2: http://www.cs.cmu.edu/~dyogatam/papers/wang+etal.iclrworkshop2016.pdf

The main motivation is that a bidirectional RNN, useful in DeepSpeech like speech models, learns representation for a sequence by performing a forward and a backward pass through the entire sequence. However, unlike unidirectional RNNs, bidirectional RNNs are challenging to deploy in an online and low-latency setting. The lookahead convolution incorporates information from future subsequences in a computationally efficient manner to improve unidirectional recurrent neural networks. The row convolution operator is different from the 1D sequence convolution, and is computed as follows:

Given an input sequence \(X\) of length \(t\) and input dimension \(D\), and a filter (\(W\)) of size \(context \times D\), the output sequence is convolved as:

$$ out_{i} = \sum_{j=i}^{i + context - 1} X_{j} \cdot W_{j-i} $$

In the above equation:

  • \(Out_{i}\): The i-th row of output variable with shape [1, D].

  • \(context\): Future context size.

  • \(X_{j}\): The j-th row of input variable with shape [1, D].

  • \(W_{j-i}\): The (j-i)-th row of parameters with shape [1, D].

More details about row_conv please refer to the design document https://github.com/PaddlePaddle/Paddle/issues/2228#issuecomment-303903645 .

Parameters
  • input (Variable) – the input(X) is a LodTensor or tensor, LodTensor(X) supports variable time-length input sequences. The underlying tensor in this LoDTensor is a matrix with shape (T x N), where T is the total time steps in this mini-batch and N is the input data dimension. the shape of Tensor input(X) has shape (B x T x N), B is batch size;.

  • future_context_size (int) – Future context size. Please note, the shape of convolution kernel is [future_context_size + 1, D].

  • param_attr (ParamAttr) – Attributes of parameters, including name, initializer etc.

  • act (str) – Non-linear activation to be applied to output variable.

Returns

the output(Out) is a LodTensor or Tensor, which has same type and same shape as X.

Examples

>>>  # for LodTensor inputs
>>> import paddle.fluid as fluid
>>> x = fluid.data(name='x', shape=[9, 16],
>>>                        dtype='float32', lod_level=1)
>>> out = fluid.layers.row_conv(input=x, future_context_size=2)
>>> # for Tensor inputs
>>> x = fluid.data(name='x', shape=[9, 4, 16], dtype='float32')
>>> out = fluid.layers.row_conv(input=x, future_context_size=2)

sampled_softmax_with_cross_entropy

paddle.fluid.layers.sampled_softmax_with_cross_entropy(logits, label, num_samples, num_true=1, remove_accidental_hits=True, use_customized_samples=False, customized_samples=None, customized_probabilities=None, seed=0)[source]

Sampled Softmax With Cross Entropy Operator.

Cross entropy loss with sampled softmax is used as the output layer for larger output classes extensively. This operator samples a number of samples for all examples, and computes the softmax normalized values for each row of the sampled tensor, after which cross-entropy loss is computed.

Because this operator performs a softmax on logits internally, it expects unscaled logits. This operator should not be used with the output of softmax operator since that would produce incorrect results.

For examples with T true labels (T >= 1), we assume that each true label has a probability of 1/T. For each sample, S samples are generated using a log uniform distribution. True labels are concatenated with these samples to form T + S samples for each example. So, assume the shape of logits is [N x K], the shape for samples is [N x (T+S)]. For each sampled label, a probability is calculated, which corresponds to the Q(y|x) in [Jean et al., 2014](http://arxiv.org/abs/1412.2007).

Logits are sampled according to the sampled labels. Then if remove_accidental_hits is True, if a sample[i, j] accidentally hits true labels, then the corresponding sampled_logits[i, j] is minus by 1e20 to make its softmax result close to zero. Then sampled logits are subtracted by logQ(y|x), these sampled logits and re-indexed labels are used to compute a softmax with cross entropy.

Parameters
  • logits (Variable) – The unscaled log probabilities, which is a 2-D tensor with shape [N x K]. N is the batch_size, and K is the class number.

  • label (Variable) – The ground truth which is a 2-D tensor. Label is a Tensor<int64> with shape [N x T], where T is the number of true labels per example.

  • num_samples (int) – The number for each example, num_samples should be less than the number of class.

  • num_true (int) – The number of target classes per training example.

  • remove_accidental_hits (bool) – A flag indicating whether to remove accidental hits when sampling. If True and if a sample[i, j] accidentally hits true labels, then the corresponding sampled_logits[i, j] is minus by 1e20 to make its softmax result close to zero. Default is True.

  • use_customized_samples (bool) – Whether to use custom samples and probabities to sample logits.

  • customized_samples (Variable) – User defined samples, which is a 2-D tensor with shape [N, T + S]. S is the num_samples, and T is the number of true labels per example.

  • customized_probabilities (Variable) – User defined probabilities of samples, a 2-D tensor which has the same shape with customized_samples.

  • seed (int) – The random seed for generating random number, which is used in the process of sampling. Default is 0.

Returns

Return the cross entropy loss which is a 2-D tensor with shape

[N x 1].

Return type

Variable

Examples

import paddle.fluid as fluid

input = fluid.layers.data(name='data', shape=[256], dtype='float32')
label = fluid.layers.data(name='label', shape=[1], dtype='int64')
fc = fluid.layers.fc(input=input, size=100)
out = fluid.layers.sampled_softmax_with_cross_entropy(
          logits=fc, label=label, num_samples=25)

sampling_id

paddle.fluid.layers.sampling_id(x, min=0.0, max=1.0, seed=0, dtype='float32')[source]

This op is used for sampling id from multinomial distribution from the input, sampling one id for one sample.

Parameters
  • x (Variable) – 2-D tensor, [batch_size, input_feature_dimensions]

  • min (Float) – minimum , default 0.0.

  • max (Float) – maximum, default 1.0.

  • seed (Float) – Random seed, default 0. if seed is not 0, will generate same number every time.

  • dtype (np.dtype|core.VarDesc.VarType|str) – The type of output data : float32, float_16, int etc

Returns

sampling tensor.

Return type

Variable

Examples

import paddle.fluid as fluid
x = fluid.data(
    name="X",
    shape=[13, 11],
    dtype='float32')

out = fluid.layers.sampling_id(x)

scale

paddle.fluid.layers.scale(x, scale=1.0, bias=0.0, bias_after_scale=True, act=None, name=None)[source]

Scale operator.

Putting scale and bias to the input Tensor as following:

bias_after_scale is True:

\[Out=scale*X+bias\]

bias_after_scale is False:

\[Out=scale*(X+bias)\]
Parameters
  • x (Variable) – Input N-D Tensor of scale operator. Data type can be float32, float64, int8, int16, int32, int64, uint8.

  • scale (float) – The scale factor of the input.

  • bias (float) – The bias to be put on the input.

  • bias_after_scale (bool) – Apply bias addition after or before scaling. It is useful for numeric stability in some circumstances.

  • act (str, optional) – Activation applied to the output such as tanh, softmax, sigmoid, relu.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name

Returns

Output tensor of scale operator, with shape and data type same as input.

Return type

Variable(Tensor|LoDTensor)

Examples

import paddle.fluid as fluid
import numpy as np

inputs = fluid.layers.data(name="x", shape=[2, 3], dtype='float32')
output = fluid.layers.scale(inputs, scale = 2.0, bias = 1.0)

exe = fluid.Executor(fluid.CPUPlace())
exe.run(fluid.default_startup_program())

img = np.array([[1, 2, 3], [4, 5, 6]]).astype(np.float32)

res = exe.run(fluid.default_main_program(), feed={'x':img}, fetch_list=[output])
print(res) # [array([[ 3.,  5.,  7.], [ 9., 11., 13.]], dtype=float32)]

scatter

paddle.fluid.layers.scatter(input, index, updates, name=None, overwrite=True)[source]

Scatter Layer

Output is obtained by updating the input on selected indices based on updates.

Parameters
  • input (Variable) – The input N-D Tensor with rank>=1. Data type can be float32.

  • index (Variable) – The index 1-D Tensor. Data type can be int32, int64. The length of index cannot exceed updates’s length, and the value in index cannot exceed input’s length.

  • updates (Variable) – update input with updates parameter based on index. shape should be the same as input, and dim value with dim > 1 shoule be the same as input.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

  • overwrite (bool) – The mode that updating the output when there are same indices. If True, use the overwrite mode to update the output of the same index, if False, use the accumulate mode to update the output of the same index. Default value is True.

Returns

The output is a Tensor with the same shape as input.

Return type

Variable(Tensor|LoDTensor)

Examples

import numpy as np
import paddle.fluid as fluid

input = fluid.layers.data(name='data', shape=[3, 2], dtype='float32', append_batch_size=False)
index = fluid.layers.data(name='index', shape=[4], dtype='int64', append_batch_size=False)
updates = fluid.layers.data(name='update', shape=[4, 2], dtype='float32', append_batch_size=False)

output = fluid.layers.scatter(input, index, updates, overwrite=False)

exe = fluid.Executor(fluid.CPUPlace())
exe.run(fluid.default_startup_program())

in_data = np.array([[1, 1], [2, 2], [3, 3]]).astype(np.float32)
index_data = np.array([2, 1, 0, 1]).astype(np.int64)
update_data = np.array([[1, 1], [2, 2], [3, 3], [4, 4]]).astype(np.float32)

res = exe.run(fluid.default_main_program(), feed={'data':in_data, "index":index_data, "update":update_data}, fetch_list=[output])
print(res)
# [array([[3., 3.],
#   [6., 6.],
#   [1., 1.]], dtype=float32)]

selu

paddle.fluid.layers.selu(x, scale=None, alpha=None, name=None)[source]

Selu Operator.

The equation is:

\[ \begin{align}\begin{aligned}selu= \lambda* \begin{cases} x &\quad \text{ if } x>0\\ \alpha * e^x - \alpha &\quad \text{ if } x<=0 \end{cases}\end{aligned}\end{align} \]

The input X can carry the LoD (Level of Details) information, or not. And the output shares the LoD information with input X.

Parameters
  • x (Variable) – The input N-D Tensor.

  • scale (float, optional) – lambda in selu activation function, the default value is 1.0507009873554804934193349852946. For more information about this value, please refer to: https://arxiv.org/abs/1706.02515.

  • alpha (float, optional) – alpha in selu activation function, the default value is 1.6732632423543772848170429916717. For more information about this value, please refer to: https://arxiv.org/abs/1706.02515.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

The output Tensor or LoDTensor with the same shape and LoD information as input.

Return type

Variable(Tensor|LoDTensor)

Examples

import paddle.fluid as fluid
import numpy as np

inputs = fluid.layers.data(name="x", shape=[2, 2], dtype="float32")
output = fluid.layers.selu(inputs)

exe = fluid.Executor(fluid.CPUPlace())
exe.run(fluid.default_startup_program())

img = np.array([[0, 1],[2, 3]]).astype(np.float32)

res = exe.run(fluid.default_main_program(), feed={'x':img}, fetch_list=[output])
print(res) # [array([[0.      , 1.050701],[2.101402, 3.152103]], dtype=float32)]

sequence_concat

paddle.fluid.layers.sequence_concat(input, name=None)[source]

Notes: The Op only receives LoDTensor as input. If your input is Tensor, please use concat Op.(fluid.layers. concat ).

This operator only supports LoDTensor as input. It concatenates the multiple LoDTensor from input by the LoD information, and outputs the concatenated LoDTensor.

input is a list of LoDTensor:
    input = [x1, x2]
where:
    x1.lod = [[0, 3, 5]]
    x1.data = [[1], [2], [3], [4], [5]]
    x1.shape = [5, 1]

    x2.lod = [[0, 2, 4]]
    x2.data = [[6], [7], [8], [9]]
    x2.shape = [4, 1]
and should satisfy: len(x1.lod[0]) == len(x2.lod[0])

output is LoDTensor:
    out.lod = [[0, 3+2, 5+4]]
    out.data = [[1], [2], [3], [6], [7], [4], [5], [8], [9]]
    out.shape = [9, 1]
Parameters
  • input (list of Variable) – List of LoDTensor to be concatenated. The length of each LoDTensor should be same. The data type can be float32, float64 or int64.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

Output the concatenated LoDTensor. The data type is same as input.

Return type

Variable

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[-1, 10], dtype='float32', lod_level=1)
y = fluid.data(name='y', shape=[-1, 10], dtype='float32', lod_level=1)
out = fluid.layers.sequence_concat(input=[x, y])

sequence_conv

paddle.fluid.layers.sequence_conv(input, num_filters, filter_size=3, filter_stride=1, padding=True, padding_start=None, bias_attr=None, param_attr=None, act=None, name=None)[source]

Notes: The Op only receives LoDTensor as input. If your input is Tensor, please use conv2d Op.(fluid.layers. conv2d ).

This operator receives input sequences with variable length and other convolutional configuration parameters(num_filters, filter_size) to apply the convolution operation. It fills all-zero padding data on both sides of the sequence by default to ensure that the output is the same length as the input. You can customize the padding behavior by configuring the parameter padding_start .

Warning: the parameter padding take no effect and will be deprecated in the future.

Here we will illustrate the details of the padding operation:
For a mini-batch of 2 variable lengths sentences, containing 3, and 1 time-steps:
Assumed input (X) is a [4, N] float LoDTensor, and for the sake of simplicity, we assume N=2.
input.data = [[1, 1],
              [2, 2],
              [3, 3],
              [4, 4]]

This is to say that input (X) has 4 words and the dimension of each word
representation is 2.

* Case1:

    If padding_start is -1 and filter_size is 3.
    The length of padding data is calculated as follows:
    up_pad_len = max(0, -padding_start) = 1
    down_pad_len = max(0, filter_size + padding_start - 1) = 1

    The output of the input sequence after padding is:
    data_aftet_padding = [[0, 0, 1, 1, 2, 2],
                          [1, 1, 2, 2, 3, 3],
                          [2, 2, 3, 3, 0, 0],
                          [0, 0, 4, 4, 0, 0]]

    It will be multiplied by the filter weight to get the final output.
    Assume num_filters = 3
    output.data = [[ 0.3234, -0.2334,  0.7433],
                   [ 0.5646,  0.9464, -0.1223],
                   [-0.1343,  0.5653,  0.4555],
                   [ 0.9954, -0.1234, -0.1234]]
    output.shape = [4, 3]     # 3 = num_filters
    output.lod = [[0, 3, 4]]  # Remain the same
Parameters
  • input (Variable) – LoDTensor with shape \((M, K)\), where M is the total time-step of mini-batch and K is hidden_size of input. Only lod_level of 1 is supported. The data type should be float32 or float64.

  • num_filters (int) – the number of filters.

  • filter_size (int) – the height of filter. Specified filter width is not supported, the width is hidden_size by default. Default: 3.

  • filter_stride (int) – stride of the filter. Currently only supports stride = 1.

  • padding (bool) – the parameter padding take no effect and will be discarded in the future. Currently, it will always pad input to make sure the length of the output is the same as input whether padding is set true or false. Because the length of input sequence may be shorter than filter_size, which will cause the convolution result to not be computed correctly. These padding data will not be trainable or updated while trainnig. Default: True.

  • padding_start (int) – It is used to indicate the start index for padding the input sequence, which can be negative. The negative number means to pad |padding_start| time-steps of all-zero data at the beginning of each instance. The positive number means to skip padding_start time-steps of each instance, and it will pad \(filter\_size + padding\_start - 1\) time-steps of all-zero data at the end of the sequence to ensure that the output is the same length as the input. If set None, the same length \(\frac{filter\_size}{2}\) of data will be filled on both sides of the sequence. If set 0, the length of \(filter\_size - 1\) data is padded at the end of each input sequence. Default: None.

  • bias_attr (ParamAttr) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .

  • param_attr (ParamAttr) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .

  • act (str) – Activation to be applied to the output of this layer, such as tanh, softmax, sigmoid, relu. For more information, please refer to Activation Function . Default: None.

  • name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .

Returns

LoDTensor with the same length as input. The data type is float32 or float64, which is same as input.

Return type

Variable

Examples

import paddle.fluid as fluid

x = fluid.data(name='x', shape=[-1, 10], dtype='float32', lod_level=1)
x_conved = fluid.layers.sequence_conv(input=x, num_filters=2, filter_size=3, padding_start=-1)

sequence_enumerate

paddle.fluid.layers.sequence_enumerate(input, win_size, pad_value=0, name=None)[source]

Generate a new sequence for the input index sequence with shape [d_1, win_size], which enumerates all the sub-sequences with length win_size of the input with shape [d_1, 1], and padded by pad_value if necessary in generation.

Please note that the input must be LodTensor.

Input x:
    x.lod = [[0, 3, 5]]
    x.data = [[1], [2], [3], [4], [5]]
    x.dims = [5, 1]

Attrs:
    win_size = 2
    pad_value = 0

Output:
    out.lod = [[0, 3, 5]]
    out.data = [[1, 2], [2, 3], [3, 0], [4, 5], [5, 0]]
    out.dims = [5, 2]
Parameters
  • input (Variable) – The input variable which is a index sequence, which should be a LodTensor with shape [d_1, 1] and 1-level lod info. The data type should be float32, float64, int8, int32 or int64.

  • win_size (int) – The window size for enumerating all sub-sequences.

  • pad_value (int, optional) – The padding value, default 0.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns: The enumerate sequence variable which is a LoDTensor with shape [d_1, win_size] and 1-level lod info. The data type is same as input.

Return Type: Variable

Examples

import paddle.fluid as fluid

x = fluid.data(name='x', shape=[-1, 1], dtype='int32', lod_level=1)
out = fluid.layers.sequence_enumerate(input=x, win_size=3, pad_value=0)

sequence_expand

paddle.fluid.layers.sequence_expand(x, y, ref_level=-1, name=None)[source]

Sequence Expand Layer. This layer will expand the input variable x according to specified level ref_level lod of y. Please note that the lod level of x is at most 1. If the lod level of x is 1, than the size of lod of x must be equal to the length of ref_level lod of y. If the lod level of x is 0, then the first dim of x should be equal to the size of ref_level of y. The rank of x is at least 2. When rank of x is greater than 2, then it would be viewed as a 2-D tensor.

Please note that the input x should be LodTensor or Tensor, and input y must be LodTensor.

Following examples will explain how sequence_expand works:

Case 1

Consider 2 sequences [a][b] and [c][d], now we want to expand them to [a][b], [a][b], [c][d] and [c][d].
Sequence [a][b] expand twice and [c][d] expands twice, so the lod which according to is [2, 2].

Input x is a 1-level LoDTensor:
    x.lod  = [[2,        2]]    #lod based on length may be easier to understand
    x.data = [[a], [b], [c], [d]]
    x.dims = [4, 1]

input y is a LoDTensor:
    y.lod = [[2,    2],    #the 0th level lod, according to this level
             [3, 3, 1, 1]] #the 1st level lod, it has nothing to do with this level

ref_level: 0

then output is a 1-level LoDTensor out:
    out.lod =  [[2,        2,        2,        2]]    #lod based on offfset
    out.data = [[a], [b], [a], [b], [c], [d], [c], [d]]
    out.dims = [8, 1]


Case 2

Consider 3 sequences [a], [b], [c], now we want to expand them to [a][a], [c][c][c].
It's obvious that the lod info of expanded sequences is [2, 0, 3].

x is a Tensor:
    x.data = [[a], [b], [c]]
    x.dims = [3, 1]

y is a LoDTensor:
    y.lod = [[2, 0, 3]]

ref_level: -1

then output is a 1-level LodTensor:
    out.data = [[a], [a], [c], [c], [c]]
    out.dims = [5, 1]
Parameters
  • x (Variable) – The input variable which is a Tensor or LoDTensor, with the dims [M, K]. The lod level is at most 1. The data type should be float32, float64, int8, int32 or int64.

  • y (Variable) – The input variable which is a LoDTensor, the lod level is at least 1.

  • ref_level (int) – Lod level of y to be referred by x. If set to -1, refer the last level of lod.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns: The expanded variable which is a LoDTensor, with dims [N, K]. N depends on the lod info of x and y. The data type is same as input.

Return Type: Variable

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers
import numpy as np

x = fluid.data(name='x', shape=[4, 1], dtype='float32')
y = fluid.data(name='y', shape=[8, 1],
            dtype='float32', lod_level=1)
out = layers.sequence_expand(x=x, y=y, ref_level=0)

exe = fluid.Executor(fluid.CPUPlace())
place = fluid.CPUPlace()

np_data = np.array([[1], [2], [3], [4]]).astype('float32')
x_lod_tensor = fluid.create_lod_tensor(np_data, [[2, 2]], place)
print(x_lod_tensor)
#lod: [[0, 2, 4]]
#    dim: 4, 1
#    layout: NCHW
#    dtype: float
#    data: [1 2 3 4]

np_data = np.array([[1], [2], [3], [4], [5], [6], [7], [8]]).astype('float32')
y_lod_tensor = fluid.create_lod_tensor(np_data, [[2, 2], [3,3,1,1]], place)
print(y_lod_tensor)
#lod: [[0, 2, 4][0, 3, 6, 7, 8]]
#    dim: 8, 1
#    layout: NCHW
#    dtype: int64_t
#    data: [0 0 1 1 1 1 1 0]

out_main = exe.run(fluid.default_main_program(),
                feed={'x': x_lod_tensor, 'y': y_lod_tensor},
                fetch_list=[out], return_numpy=False)
print(out_main[0])
#lod: [[0, 2, 4, 6, 8]]
#    dim: 8, 1
#    layout: NCHW
#    dtype: float
#    data: [1 2 1 2 3 4 3 4]

sequence_expand_as

paddle.fluid.layers.sequence_expand_as(x, y, name=None)[source]

Sequence Expand As Layer. This OP will expand the input variable x according to the zeroth level lod of y. Current implementation requires the level number of y’s lod must be 1, and the first dimension of x should be equal to the size of y’s zeroth level lod, thus the expanded LodTensor has the same lod info as y. The expanded result has nothing to do with x’s lod, so the lod of Input(X) is not considered.

Please note that the input x should be LodTensor or Tensor, and input y must be LodTensor.

Following examples will explain how sequence_expand_as works:

Case 1:

Consider 4 sequences [a], [b], [c], [d], now we want to expand them to [a][a][a], [b][b][b], [c] and [d].
It's obvious that the lod info of expanded sequences is [0, 3, 6, 7, 8].
Given a 1-level LodTensor ``x``:
    x.data = [[a], [b], [c], [d]]
    x.dims = [4, 1]
and input ``y``
    y.lod = [[3, 3, 1, 1]] #lod based on length may be easier to understand

then we get 1-level LoDTensor out:
    Out.lod =  [[0,            3,              6,  7,  8]] #based on offset
    Out.data = [[a], [a], [a], [b], [b], [b], [c], [d]]
    Out.dims = [8, 1]


Case 2:

Given a common Tensor ``x``:
    x.data = [[a, b], [c, d], [e, f]]
    x.dims = [3, 2]
and input ``y``:
    y.lod = [[0, 2, 3, 6]]

then we get a 1-level LoDTensor:
    out.lod =  [[0,             2,     3,                    6]]
    out.data = [[a, b], [a, b] [c, d], [e, f], [e, f], [e, f]]
    out.dims = [6, 2]
Parameters
  • x (Variable) – The input variable which is a Tensor or LoDTensor, with the dims [M, K]. The data type should be float32, float64, int8, int32 or int64.

  • y (Variable) – The input variable which is a LoDTensor with 1-level lod.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns: The expanded variable which is a LoDTensor with the dims [N, K]. N depends on the lod of y, and the lod level must be 1. The data type is same as input.

Return Type: Variable

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers
import numpy as np

x = fluid.data(name='x', shape=[4, 1], dtype='float32')
y = fluid.data(name='y', shape=[8, 1], dtype='float32', lod_level=1)
out = layers.sequence_expand_as(x=x, y=y)

exe = fluid.Executor(fluid.CPUPlace())
place = fluid.CPUPlace()

np_data = np.array([[1], [2], [3], [4]]).astype('float32')
x_lod_tensor = fluid.create_lod_tensor(np_data, [[2, 2]], place)
print(x_lod_tensor)
#lod: [[0, 2, 4]]
#    dim: 4, 1
#    layout: NCHW
#    dtype: float
#    data: [1 2 3 4]

np_data = np.array([[1], [2], [3], [4], [5], [6], [7], [8]]).astype('float32')
y_lod_tensor = fluid.create_lod_tensor(np_data, [[3,3,1,1]], place)
print(y_lod_tensor)
#lod: [[0, 3, 6, 7, 8]]
#    dim: 8, 1
#    layout: NCHW
#    dtype: int64_t
#    data: [0 0 1 0 1 1 1 0]

out_main = exe.run(fluid.default_main_program(),
                feed={'x': x_lod_tensor, 'y': y_lod_tensor},
                fetch_list=[out], return_numpy=False)
print(out_main[0])
#lod: [[0, 3, 6, 7, 8]]
#    dim: 8, 1
#    layout: NCHW
#    dtype: float
#    data: [1 1 1 2 2 2 3 4]

sequence_first_step

paddle.fluid.layers.sequence_first_step(input)[source]

This operator only supports LoDTensor as input. Given the input LoDTensor, it will select first time-step feature of each sequence as output.

Case 1:
 input is 1-level LoDTensor:
     input.lod = [[0, 2, 5, 7]]
     input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]]
     input.shape = [7, 1]

 output is a LoDTensor:
     out.shape = [3, 1]
     out.shape[0] == len(x.lod[-1]) == 3
     out.data = [[1.], [2.], [5.]], where 1.=first(1., 3.), 2.=first(2., 4., 6.), 5.=first(5., 1.)

 Case 2:
 input is a 2-level LoDTensor containing 3 sequences with length info [2, 0, 3],
 where 0 means empty sequence.
 The first sequence contains 2 subsequence with length info [1, 2];
 The last sequence contains 3 subsequence with length info [1, 0, 3].
     input.lod = [[0, 2, 2, 5], [0, 1, 3, 4, 4, 7]]
     input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]]
     input.shape = [7, 1]

 It will apply pooling on last lod_level [0, 1, 3, 4, 4, 7]. pad_value = 0.0
 output is a LoDTensor:
     out.shape= [5, 1]
     out.lod = [[0, 2, 2, 5]]
     out.shape[0] == len(x.lod[-1]) == 5
     out.data = [[1.], [3.], [4.], [0.0], [6.]]
     where 1.=first(1.), 3.=first(3., 2.), 4.=first(4.), 0.0 = pad_value, 6.=first(6., 5., 1.)
Parameters

input (Variable) – LoDTensor with lod_level no more than 2. The data type should be float32.

Returns

LoDTensor consist of the sequence’s first step vector. The data type is float32.

Return type

Variable

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[None, 10], dtype='float32', lod_level=1)
x_first_step = fluid.layers.sequence_first_step(input=x)

sequence_last_step

paddle.fluid.layers.sequence_last_step(input)[source]

This operator only supports LoDTensor as input. Given the input LoDTensor, it will select last time-step feature of each sequence as output.

Case 1:
input is 1-level LoDTensor:
    input.lod = [[0, 2, 5, 7]]
    input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]]
    input.shape = [7, 1]

output is a LoDTensor:
    out.shape = [3, 1]
    out.shape[0] == len(x.lod[-1]) == 3
    out.data = [[3.], [6.], [1.]], where 3.=last(1., 3.), 6.=last(2., 4., 6.), 1.=last(5., 1.)

Case 2:
input is a 2-level LoDTensor containing 3 sequences with length info [2, 0, 3],
where 0 means empty sequence.
The first sequence contains 2 subsequence with length info [1, 2];
The last sequence contains 3 subsequence with length info [1, 0, 3].
    input.lod = [[0, 2, 2, 5], [0, 1, 3, 4, 4, 7]]
    input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]]
    input.shape = [7, 1]

It will apply pooling on last lod_level [0, 1, 3, 4, 4, 7]. pad_value = 0.0
output is a LoDTensor:
    out.shape= [5, 1]
    out.lod = [[0, 2, 2, 5]]
    out.shape[0] == len(x.lod[-1]) == 5
    out.data = [[1.], [2.], [4.], [0.0], [1.]]
    where 1.=last(1.), 2.=last(3., 2.), 4.=last(4.), 0.0 = pad_value, 1=last(6., 5., 1.)
Parameters

input (Variable) – LoDTensor with lod_level no more than 2. The data type should be float32.

Returns

LoDTensor consist of the sequence’s last step vector. The data type is float32.

Return type

Variable

Examples

import paddle.fluid as fluid
x = fluid.data(name='x', shape=[None, 10], dtype='float32', lod_level=1)
x_last_step = fluid.layers.sequence_last_step(input=x)

sequence_mask

paddle.fluid.layers.sequence_mask(x, maxlen=None, dtype='int64', name=None)[source]

SequenceMask Layer

This layer outputs a mask according to the input x and maxlen with data type of dtype.

Supposing x is a Tensor with shape [d_1, d_2, …, d_n], the y is a mask with shape [d_1, d_2, …, d_n, maxlen], where:

\[y(i_1, i_2,..., i_n, j) = (j < x(i_1, i_2,..., i_n))\]
Case:

Consider input:
    x = [3, 1, 1, 0]    max_len = 4

then we get out:
    mask = [[1, 1, 1, 0],
            [1, 0, 0, 0],
            [1, 0, 0, 0],
            [0, 0, 0, 0]]
Parameters
  • x (Variable) – Input tensor of sequence_mask layer, whose elements are integers less than maxlen. Tensor or LodTensor with shape [d_1, d_2, …, d_n].

  • maxlen (int, optional) – Maximum length of the sequence. If maxlen is None, it would be replace with \(max(x)\).

  • dtype (np.dtype|core.VarDesc.VarType|str, optional) – Data type of the output, int64 by default.

  • name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.

Returns: The output sequence mask. Tensor or LodTensor with shape [d_1, d_2, …, d_n, maxlen] and data type of dtype. The data type should be float32, float64, int8, int32 or int64.

Return Type: Variable

Examples