nn¶
adaptive_pool2d¶

paddle.fluid.layers.
adaptive_pool2d
(input, pool_size, pool_type='max', require_index=False, name=None)[source] This operation calculates the output based on the input, pool_size, pool_type parameters. Input(X) and output(Out) are in NCHW format, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Parameters(pool_size) should contain two elements which represent height and width, respectively. Also the H and W dimensions of output(Out) is same as Parameter(pool_size). The output tensor shape will be [N, C, pool_size[0], pool_size[1]]
For average adaptive pool2d:
\[ \begin{align}\begin{aligned}hstart &= floor(i * H_{in} / H_{out})\\hend &= ceil((i + 1) * H_{in} / H_{out})\\wstart &= floor(j * W_{in} / W_{out})\\wend &= ceil((j + 1) * W_{in} / W_{out})\\Output(i ,j) &= \frac{sum(Input[hstart:hend, wstart:wend])}{(hend  hstart) * (wend  wstart)}\end{aligned}\end{align} \] Parameters
input (Variable) – The input tensor of pooling operator, which is a 4D tensor with shape [N, C, H, W]. The format of input tensor is NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. The data type is float32 or float64.
pool_size (intlisttuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain two integers, (pool_size_Height, pool_size_Width).
pool_type – (string), pooling type, can be “max” for maxpooling and “avg” for averagepooling
require_index (bool) – If true, the index of max pooling point will be returned along with outputs. It cannot be set in average pooling type. Default False.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
 The output tensor of adaptive pooling result. The data type is same
as input tensor.
 Return type
Variable
 Raises
ValueError
– ‘pool_type’ is not ‘max’ nor ‘avg’.ValueError
– invalid setting ‘require_index’ true when ‘pool_type’ is ‘avg’.ValueError
– ‘pool_size’ should be a list or tuple with length as 2.
Examples
# average adaptive pool2d # suppose input data in shape of [N, C, H, W], `pool_size` is [m, n], # output shape is [N, C, m, n], adaptive pool divide H and W dimentions # of input data into m * n grids averagely and performs poolings in each # grid to get output. # adaptive average pool performs calculations as follow: # # for i in range(m): # for j in range(n): # hstart = floor(i * H / m) # hend = ceil((i + 1) * H / m) # wstart = floor(i * W / n) # wend = ceil((i + 1) * W / n) # output[:, :, i, j] = avg(input[:, :, hstart: hend, wstart: wend]) # import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32') pool_out = fluid.layers.adaptive_pool2d( input=data, pool_size=[3, 3], pool_type='avg') # max adaptive pool2d # suppose input data in shape of [N, C, H, W], `pool_size` is [m, n], # output shape is [N, C, m, n], adaptive pool divide H and W dimentions # of input data into m * n grids averagely and performs poolings in each # grid to get output. # adaptive average pool performs calculations as follow: # # for i in range(m): # for j in range(n): # hstart = floor(i * H / m) # hend = ceil((i + 1) * H / m) # wstart = floor(i * W / n) # wend = ceil((i + 1) * W / n) # output[:, :, i, j] = max(input[:, :, hstart: hend, wstart: wend]) # import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32') pool_out = fluid.layers.adaptive_pool2d( input=data, pool_size=[3, 3], pool_type='max')
adaptive_pool3d¶

paddle.fluid.layers.
adaptive_pool3d
(input, pool_size, pool_type='max', require_index=False, name=None)[source] This operation calculates the output based on the input, pool_size, pool_type parameters. Input(X) and output(Out) are in NCDHW format, where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. Parameters(pool_size) should contain three elements which represent height and width, respectively. Also the D, H and W dimensions of output(Out) is same as Parameter(pool_size). The output tensor shape will be [N, C, pool_size[0], pool_size[1], pool_size[2]]
For average adaptive pool3d:
\[ \begin{align}\begin{aligned}dstart &= floor(i * D_{in} / D_{out})\\dend &= ceil((i + 1) * D_{in} / D_{out})\\hstart &= floor(j * H_{in} / H_{out})\\hend &= ceil((j + 1) * H_{in} / H_{out})\\wstart &= floor(k * W_{in} / W_{out})\\wend &= ceil((k + 1) * W_{in} / W_{out})\\Output(i ,j, k) &= \frac{sum(Input[dstart:dend, hstart:hend, wstart:wend])}{(dend  dstart) * (hend  hstart) * (wend  wstart)}\end{aligned}\end{align} \] Parameters
input (Variable) – The input tensor of pooling operator, which is a 5D tensor with shape [N, C, D, H, W]. The format of input tensor is NCDHW, where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. The data type is float32 or float64.
pool_size (intlisttuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain three integers, (Depth, Height, Width).
pool_type – (string) Pooling type, can be “max” for maxpooling and “avg” for averagepooling
require_index (bool) – If true, the index of max pooling point will be returned along with outputs. It cannot be set in average pooling type. Default False.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
The output tensor of adaptive pooling result. The data type is same as input tensor.
 Return type
Variable
 Raises
ValueError
– ‘pool_type’ is not ‘max’ nor ‘avg’.ValueError
– invalid setting ‘require_index’ true when ‘pool_type’ is ‘avg’.ValueError
– ‘pool_size’ should be a list or tuple with length as 2.
Examples
# average adaptive pool3d # suppose input data in shape of [N, C, D, H, W], `pool_size` is [l, m, n], # output shape is [N, C, l, m, n], adaptive pool divide D, H and W dimentions # of input data into l * m * n grids averagely and performs poolings in each # grid to get output. # adaptive average pool performs calculations as follow: # # for i in range(l): # for j in range(m): # for k in range(n): # dstart = floor(i * D / l) # dend = ceil((i + 1) * D / l) # hstart = floor(j * H / m) # hend = ceil((j + 1) * H / m) # wstart = floor(k * W / n) # wend = ceil((k + 1) * W / n) # output[:, :, i, j, k] = # avg(input[:, :, dstart:dend, hstart: hend, wstart: wend]) # import paddle.fluid as fluid data = fluid.data( name='data', shape=[None, 3, 32, 32, 32], dtype='float32') pool_out = fluid.layers.adaptive_pool3d( input=data, pool_size=[3, 3, 3], pool_type='avg') # max adaptive pool3d # suppose input data in shape of [N, C, D, H, W], `pool_size` is [l, m, n], # output shape is [N, C, l, m, n], adaptive pool divide D, H and W dimentions # of input data into l * m * n grids averagely and performs poolings in each # grid to get output. # adaptive average pool performs calculations as follow: # # for i in range(l): # for j in range(m): # for k in range(n): # dstart = floor(i * D / l) # dend = ceil((i + 1) * D / l) # hstart = floor(j * H / m) # hend = ceil((j + 1) * H / m) # wstart = floor(k * W / n) # wend = ceil((k + 1) * W / n) # output[:, :, i, j, k] = # avg(input[:, :, dstart:dend, hstart: hend, wstart: wend]) # import paddle.fluid as fluid data = fluid.data( name='data', shape=[None, 3, 32, 32, 32], dtype='float32') pool_out = fluid.layers.adaptive_pool3d( input=data, pool_size=[3, 3, 3], pool_type='max')
add_position_encoding¶

paddle.fluid.layers.
add_position_encoding
(input, alpha, beta, name=None)[source] This operator performs weighted sum of input feature at each position (position in the sequence) and the corresponding position encoding.
For more details of position encoding, please refer to Attention Is All You Need .
The formula is as follows:
\[\begin{split}PE(pos, 2i) &= \sin{(pos / 10000^{2i / P})} \\ PE(pos, 2i + 1) &= \cos{(pos / 10000^{2i / P})} \\ Out(:, pos, i) &= \alpha * input(:, pos, i) + \beta * PE(pos, i)\end{split}\] Where:
\(PE(pos, 2i)\) : the value at even index 2i for encoding of position pos.
\(PE(pos, 2i + 1)\) : the value at odd index 2i+1 for encoding of position pos
 Parameters
input (Variable) – A Tensor or LoDTensor (lod level is 1). If it is a Tensor, the shape should be [N, M, P], where N stands for batch size, M for sequence length, P for the size of feature dimension. If it is a LoDTensor, the shape should be [N, P], where N stands for the total sequence lengths in this minibatch, P for the size of feature. The data type should be float32 or float64.
alpha (float) – Indicate the weight coefficient for input when performing weighted sum.
beta (float) – Indicate the weight coefficient for position encoding when performing weighted sum.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
A Tensor or LoDTensor. It has the same shape, data type and lod as input.
 Return type
Variable
Examples
import paddle.fluid as fluid tensor = fluid.data( name='tensor', shape=[None, 64, 512], dtype='float32') position_tensor = fluid.layers.add_position_encoding( input=tensor, alpha=1.0, beta=1.0)
affine_channel¶

paddle.fluid.layers.
affine_channel
(x, scale=None, bias=None, data_layout='NCHW', name=None, act=None)[source] Applies a separate affine transformation to each channel of the input. Useful for replacing spatial batch norm with its equivalent fixed transformation. The input also can be 2D tensor and applies a affine transformation in second dimension.
 Parameters
x (Variable) – Feature map input can be a 4D tensor with order NCHW or NHWC. It also can be a 2D tensor and the affine transformation is applied in the second dimension.The data type is float32 or float64.
scale (Variable) – 1D input of shape (C), the cth element is the scale factor of the affine transformation for the cth channel of the input.The data type is float32 or float64.
bias (Variable) – 1D input of shape (C), the cth element is the bias of the affine transformation for the cth channel of the input. The data type is float32 or float64.
data_layout (str, default NCHW) – NCHW or NHWC. If input is 2D tensor, you can ignore data_layout.
name (str, default None) – The name of this layer. For more information, please refer to api_guide_Name .
act (str, default None) – Activation to be applied to the output of this layer.
 Returns
A tensor which has the same shape, data layout and data type with x.
 Return type
Variable
Examples
import numpy as np import paddle.fluid as fluid use_gpu = False place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace() exe = fluid.Executor(place) data = fluid.data(name='data', shape=[None, 1, 2, 2], dtype='float32') input_scale = fluid.layers.create_parameter(shape=[1], dtype="float32", default_initializer=fluid.initializer.Constant(2.0)) input_bias = fluid.layers.create_parameter(shape=[1],dtype="float32", default_initializer=fluid.initializer.Constant(0.5)) out = fluid.layers.affine_channel(data,scale=input_scale, bias=input_bias) exe.run(fluid.default_startup_program()) test_program = fluid.default_main_program().clone(for_test=True) [out_array] = exe.run(test_program, fetch_list=out, feed={'data': np.ones([1,1,2,2]).astype('float32')}) # out_array is [[[[2.5, 2.5], # [2.5, 2.5]]]] with shape: [1, 1, 2, 2]
affine_grid¶

paddle.fluid.layers.
affine_grid
(theta, out_shape, name=None)[source] It generates a grid of (x,y) coordinates using the parameters of the affine transformation that correspond to a set of points where the input feature map should be sampled to produce the transformed output feature map.
 Parameters
theta (Variable) – The data type can be float32 or float64.
out_shape (Variable  list  tuple) – The shape of target output with format [batch_size, channel, height, width].
out_shape
can be a Tensor or a list or tuple. The data type must be int32.name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.
 Returns
A Tensor with shape [batch_size, H, W, 2] while ‘H’ and ‘W’ are the height and width of feature map in affine transformation. The data type is the same as theta.
 Return type
Variable
 Raises
ValueError
– If the type of arguments is not supported.
Examples
import paddle.fluid as fluid import numpy as np place = fluid.CPUPlace() theta = fluid.data(name="x", shape=[None, 2, 3], dtype="float32") out_shape = fluid.data(name="y", shape=[4], dtype="int32") grid_0 = fluid.layers.affine_grid(theta, out_shape) grid_1 = fluid.layers.affine_grid(theta, [5, 3, 28, 28]) batch_size=2 exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) output= exe.run(feed={"x": np.random.rand(batch_size,2,3).astype("float32"), "y": np.array([5, 3, 28, 28]).astype("int32")}, fetch_list=[grid_0.name, grid_1.name]) print(output[0]) print(output[1])
autoincreased_step_counter¶

paddle.fluid.layers.
autoincreased_step_counter
(counter_name=None, begin=1, step=1)[source] Create an autoincrease variable. which will be automatically increased by 1 in every iteration. By default, the first return of this counter is 1, and the step size is 1.
 Parameters
counter_name (str, optional) – The counter name. Default ‘@STEP_COUNTER@’.
begin (int, optional) – The first return value of this counter. Default 1.
step (int, optional) – The step size. Default 1.
 Returns
The autoincreased Variable with data type int64.
 Return type
Variable
Examples
import paddle.fluid as fluid global_step = fluid.layers.autoincreased_step_counter( counter_name='@LR_DECAY_COUNTER@', begin=0, step=1)
batch_norm¶

paddle.fluid.layers.
batch_norm
(input, act=None, is_test=False, momentum=0.9, epsilon=1e05, param_attr=None, bias_attr=None, data_layout='NCHW', in_place=False, name=None, moving_mean_name=None, moving_variance_name=None, do_model_average_for_mean_and_var=True, fuse_with_relu=False, use_global_stats=False)[source] Batch Normalization Layer
Can be used as a normalizer function for convolution or fully_connected operations. The required data format for this layer is one of the following:
NHWC [batch, in_height, in_width, in_channels]
NCHW [batch, in_channels, in_height, in_width]
Refer to Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift for more details.
\(input\) is the input features over a minibatch.
\[ \begin{align}\begin{aligned}\begin{split}\mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\ \ minibatch\ mean \\ \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i  \ \mu_{\beta})^2 \qquad &//\ minibatch\ variance \\ \hat{x_i} &\gets \frac{x_i  \mu_\beta} {\sqrt{\ \sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift\end{split}\\\begin{split}moving\_mean = moving\_mean * momentum + minibatch\_mean * (1.  momentum) \\ moving\_var = moving\_var * momentum + minibatch\_var * (1.  momentum)\end{split}\end{aligned}\end{align} \]moving_mean is global mean and moving_var is global variance.
When use_global_stats = True, the \(\mu_{\beta}\) and \(\sigma_{\beta}^{2}\) are not the statistics of one minibatch. They are global (or running) statistics. (It usually got from the pretrained model.) The training and testing (or inference) have the same behavior:
\[\begin{split}\hat{x_i} &\gets \frac{x_i  \mu_\beta} {\sqrt{\ \sigma_{\beta}^{2} + \epsilon}} \\ y_i &\gets \gamma \hat{x_i} + \beta\end{split}\]Note
if build_strategy.sync_batch_norm=True, the batch_norm in network will use sync_batch_norm automatically.
 Parameters
input (variable) – The rank of input variable can be 2, 3, 4, 5. The data type is float16 or float32 or float64.
act (string, Default None) – Activation type, linearreluprelu…
is_test (bool, Default False) – A flag indicating whether it is in test phrase or not.
momentum (float, Default 0.9) – The value used for the moving_mean and moving_var computation. The updated formula is: \(moving\_mean = moving\_mean * momentum + new\_mean * (1.  momentum)\) \(moving\_var = moving\_var * momentum + new\_var * (1.  momentum)\) Default is 0.9.
epsilon (float, Default 1e05) – A value added to the denominator for numerical stability. Default is 1e5.
param_attr (ParamAttrNone) – The parameter attribute for Parameter scale of batch_norm. If it is set to None or one attribute of ParamAttr, batch_norm will create ParamAttr as param_attr, the name of scale can be set in ParamAttr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.
bias_attr (ParamAttrNone) – The parameter attribute for the bias of batch_norm. If it is set to None or one attribute of ParamAttr, batch_norm will create ParamAttr as bias_attr, the name of bias can be set in ParamAttr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.
data_layout (str, default NCHW) – the data_layout of input, is NCHW or NHWC.
in_place (bool, Default False) – Make the input and output of batch norm reuse memory.
name (strNone) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
moving_mean_name (str, Default None) – The name of moving_mean which store the global Mean. If it is set to None, batch_norm will save global mean with a random name, otherwise, batch_norm will save global mean with the string.
moving_variance_name (str, Default None) – The name of the moving_variance which store the global Variance. If it is set to None, batch_norm will save global variance with a random name, otherwise, batch_norm will save global variance with the string.
do_model_average_for_mean_and_var (bool, Default True) – Whether parameter mean and variance should do model average when model average is enabled.
fuse_with_relu (bool) – if True, this OP performs relu after batch norm.
use_global_stats (bool, Default False) – Whether to use global mean and variance. In inference or test mode, set use_global_stats to true or is_test to true, and the behavior is equivalent. In train mode, when setting use_global_stats True, the global mean and variance are also used during train period.
 Returns
A Variable holding Tensor which is the result after applying batch normalization on the input, has same shape and data type with input.
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[3, 7, 3, 7], dtype='float32') hidden1 = fluid.layers.fc(input=x, size=200, param_attr='fc1.w') hidden2 = fluid.layers.batch_norm(input=hidden1)
beam_search¶

paddle.fluid.layers.
beam_search
(pre_ids, pre_scores, ids, scores, beam_size, end_id, level=0, is_accumulated=True, name=None, return_parent_idx=False)[source] Beam search is a classical algorithm for selecting candidate words in a machine translation task.
Refer to Beam search for more details.
This operator only supports LoDTensor. It is used after finishing scores calculation to perform beam search for one time step. Specifically, after
ids
andscores
have been produced, it selects the topK ( k isbeam_size
) candidate word ids of current step fromids
according to the correspongdingscores
. Additionally,pre_id
andpre_scores
are the output of beam_search at previous step, they are needed for special use to handle ended candidate translations.Note that if
is_accumulated
is True, thescores
passed in should be accumulated scores. Otherwise, thescores
are considered as the probabilities of single step and would be transformed to the log field and added up withpre_scores
for final scores in this operator. Length penalty should be done with extra operators before calculating the accumulated scores if needed.Please see the following demo for a fully beam search usage example:
fluid/tests/book/test_machine_translation.py
 Parameters
pre_ids (Variable) – A LodTensor variable (lod level is 2), representing the selected ids of previous step. It is the output of beam_search at previous step. Its shape is [batch_size, 1] and its lod is [[0, 1, … , batch_size], [0, 1, …, batch_size]] at the first step. The data type should be int64.
pre_scores (Variable) – A LodTensor variable has the same shape and lod with
pre_ids
, representing the accumulated scores corresponding to the selected ids of previous step. It is the output of beam_search at previous step. The data type should be float32.ids (VariableNone) – A LodTensor variable containing the candidates ids. It has the same lod with
pre_ids
and its shape should be [batch_size * beam_size, K], where K supposed to be greater thanbeam_size
and the first dimension size (decrease as samples reach to the end) should be same as that ofpre_ids
. The data type should be int64. It can be None, which use indice inscores
as ids.scores (Variable) – A LodTensor variable containing the accumulated scores corresponding to
ids
. Both its shape and lod are same as thoes ofids
. The data type should be float32.beam_size (int) – The beam width used in beam search.
end_id (int) – The id of end token.
level (int) – It can be ignored and mustn’t change currently. The 2 level lod used in this operator has the following meaning: The first level describes how many beams each sample has, which would change to 0 when beams of the sample all end (batch reduce); The second level describes how many times each beam is selected. Default 0, which shouldn’t be changed currently.
is_accumulated (bool) – Whether the input
score
is accumulated scores. Default True.name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
return_parent_idx (bool, optional) – Whether to return an extra Tensor variable in output, which stores the selected ids’ parent indice in
pre_ids
and can be used to update RNN’s states by gather operator. Default False.
 Returns
The tuple contains two or three LodTensor variables. The two LodTensor, representing the selected ids and the corresponding accumulated scores of current step, have the same shape [batch_size, beam_size] and lod with 2 levels, and have data types int64 and float32. If
return_parent_idx
is True, an extra Tensor variable preserving the selected ids’ parent indice is included, whose shape is [batch_size * beam_size] and data type is int64. Return type
tuple
Examples
import paddle.fluid as fluid # Suppose `probs` contains predicted results from the computation # cell and `pre_ids` and `pre_scores` is the output of beam_search # at previous step. beam_size = 4 end_id = 1 pre_ids = fluid.data( name='pre_id', shape=[None, 1], lod_level=2, dtype='int64') pre_scores = fluid.data( name='pre_scores', shape=[None, 1], lod_level=2, dtype='float32') probs = fluid.data( name='probs', shape=[None, 10000], dtype='float32') topk_scores, topk_indices = fluid.layers.topk(probs, k=beam_size) accu_scores = fluid.layers.elementwise_add( x=fluid.layers.log(x=topk_scores), y=fluid.layers.reshape(pre_scores, shape=[1]), axis=0) selected_ids, selected_scores = fluid.layers.beam_search( pre_ids=pre_ids, pre_scores=pre_scores, ids=topk_indices, scores=accu_scores, beam_size=beam_size, end_id=end_id)
beam_search_decode¶

paddle.fluid.layers.
beam_search_decode
(ids, scores, beam_size, end_id, name=None)[source] This operator is used after beam search has completed. It constructs the full predicted sequences for each sample by walking back along the search paths stored in lod of
ids
. The result sequences are stored in a LoDTensor, which uses the following way to parse:If lod = [[0, 3, 6], [0, 12, 24, 40, 54, 67, 82]] The first level of lod stands for: There are 2 samples each having 3 (beam width) predicted sequence. The second level of lod stands for: The lengths of the first sample's 3 predicted sequences are 12, 12, 16; The lengths of the second sample's 3 predicted sequences are 14, 13, 15.
 Please see the following demo for a fully beam search usage example:
fluid/tests/book/test_machine_translation.py
 Parameters
ids (Variable) – The LoDTensorArray variable containing the selected ids of all steps. Each LoDTensor in it has int64 data type and 2 level lod which can be used to get the search paths.
scores (Variable) – The LodTensorArray variable containing the accumulated scores corresponding to selected ids of all steps. It has the same size as
ids
. Each LoDTensor in it has the same shape and lod as the counterpart inids
, and has a float32 data type.beam_size (int) – The beam width used in beam search.
end_id (int) – The id of end token.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
The tuple contains two LodTensor variables. The two LodTensor, containing the full sequences of ids and the correspongding accumulated scores, have the same shape flattened to 1D and have the same 2 level lod. The lod can be used to get how many predicted sequences each sample has and how many ids each predicted sequence has.
 Return type
tuple
Examples
import paddle.fluid as fluid # Suppose `ids` and `scores` are LodTensorArray variables reserving # the selected ids and scores of all steps ids = fluid.layers.create_array(dtype='int64') scores = fluid.layers.create_array(dtype='float32') finished_ids, finished_scores = fluid.layers.beam_search_decode( ids, scores, beam_size=5, end_id=0)
bilinear_tensor_product¶

paddle.fluid.layers.
bilinear_tensor_product
(x, y, size, act=None, name=None, param_attr=None, bias_attr=None)[source] Bilinear Tensor Product Layer
This layer performs bilinear tensor product on two inputs. For example:
\[out_{i} = x * W_{i} * {y^\mathrm{T}}, i=0,1,...,size1\] In this formula:
\(x\): the first input contains M elements, shape is [batch_size, M].
\(y\): the second input contains N elements, shape is [batch_size, N].
\(W_{i}\): the ith learned weight, shape is [M, N].
\(out_{i}\): the ith element of out, shape is [batch_size, size].
\(y^\mathrm{T}\): the transpose of \(y_{2}\).
 Parameters
x (Variable) – 2D input tensor with shape [batch_size, M]. Data type is float32 or float64.
y (Variable) – 2D input tensor with shape [batch_size, N]. Data type should be same as x.
size (int) – The dimension of this layer.
act (strNone) – Activation to be applied to the output of this layer. Default None.
name (strNone) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.
param_attr (ParamAttrNone) – To specify the weight parameter attribute. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
bias_attr (ParamAttrNone) – To specify the bias parameter attribute. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .
 Returns
A 2D Tensor of shape [batch_size, size]. Data type is the same as input x.
 Return type
Variable
Examples
import paddle.fluid as fluid layer1 = fluid.data("t1", shape=[1, 5], dtype="float32") layer2 = fluid.data("t2", shape=[1, 4], dtype="float32") tensor = fluid.layers.bilinear_tensor_product(x=layer1, y=layer2, size=1000)
bpr_loss¶

paddle.fluid.layers.
bpr_loss
(input, label, name=None)[source] Bayesian Personalized Ranking Loss Operator
This operator belongs to pairwise ranking loss. Label is the desired item. The loss at a given point in one session is defined as:
\[Y[i] = 1/(N[i]  1) * \sum_j{\log(\sigma(X[i, Label[i]]X[i, j]))}\]Learn more details by reading paper <sessionbased recommendations with recurrent neural networks>.
 Parameters
input (Variablelist) – a 2D tensor with shape [N x D], where N is the batch size and D is the number of positive classes and negative classes This input is not probability but logits.
label (Variablelist) – the ground truth which is a 2D tensor. label is a tensor<int64> with shape [N x 1].
name (strNone) – A name for this layer(optional). If set None, the layer will be named automatically. Default: None.
 Returns
A 2D tensor with shape [N x 1], the bpr loss.
Examples
import paddle.fluid as fluid neg_size = 10 label = fluid.data( name="label", shape=[3, 1], dtype="int64") predict = fluid.data( name="predict", shape=[3, neg_size + 1], dtype="float32") cost = fluid.layers.bpr_loss(input=predict, label=label)
brelu¶

paddle.fluid.layers.
brelu
(x, t_min=0.0, t_max=24.0, name=None)[source] BRelu Activation Operator.
\(out = \min(\max(x, t_{min}), t_{max})\)
 Parameters
x (Variable) – The input is a multidimensional Tensor. The data type is float32, float64
t_min (FLOAT0.0) – The min marginal value of BRelu
t_max (FLOAT24.0) – The max marginal value of BRelu
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.
 Returns
The output is a multidimensional Tensor which has same dimension and data type as the
X
 Return type
Variable
Examples:
import paddle.fluid as fluid import numpy as np input_brelu = np.array([[1,6],[1,15.6]]) with fluid.dygraph.guard(): x = fluid.dygraph.to_variable(input_brelu) y = fluid.layers.brelu(x, t_min=1.0, t_max=10.0) print(y.numpy()) #[[ 1. 6.] #[ 1. 10.]]
chunk_eval¶

paddle.fluid.layers.
chunk_eval
(input, label, chunk_scheme, num_chunk_types, excluded_chunk_types=None, seq_length=None)[source] This operator computes the precision, recall and F1score for chunk detection. It is often used in sequence tagging tasks, such as Named Entity Recognition(NER).
For some basics of chunking, please refer to Chunking with Support Vector Machines .
This operator supports IOB, IOE, IOBES and IO (also known as plain) tagging schemes. Here is a NER example for the usage of these tagging schemes:
====== ====== ====== ===== == ============ ===== ===== ===== == ========= Li Ming works at Agricultural Bank of China in Beijing. ====== ====== ====== ===== == ============ ===== ===== ===== == ========= IO IPER IPER O O IORG IORG IORG IORG O ILOC IOB BPER IPER O O BORG IORG IORG IORG O BLOC IOE IPER EPER O O IORG IORG IORG EORG O ELOC IOBES BPER EPER O O IORG IORG IORG EORG O SLOC ====== ====== ====== ===== == ============ ===== ===== ===== == =========
There are three chunk types(named entity types) including PER(person), ORG(organization) and LOC(location), and we can see that the labels have the form <tag type><chunk type> .
Since the implementation of this operator actually uses label ids rather than label strings, to make it work, there should be a way to map label ids to tag types and chunk types. This operator uses the following way to do mapping:
tag_type = label % num_tag_type chunk_type = label / num_tag_type
where num_tag_type is the num of tag types in the tagging scheme, num_chunk_type is the num of chunk types, and tag_type get its value from the following table.
Scheme Begin Inside End Single plain 0    IOB 0 1   IOE  0 1  IOBES 0 1 2 3
Accordingly, in the above NER example, if the tagging scheme is IOB and chunk types are ORG, PER and LOC, then the label ids would be as follows:
BORG 0 IORG 1 BPER 2 IPER 3 BLOC 4 ILOC 5 O 6
With which we can map each label id to the corresponding tag type and chunk type correctly.
 Parameters
input (Variable) – A Tensor or LoDTensor, representing the predicted labels from the network. When it is a Tensor, its shape would be [N, M, 1], where N stands for batch size, M for sequence length; When it is a LoDTensor, its shape would be [N, 1] where N stands for the total sequence lengths in this minibatch. The data type should be int64.
label (Variable) – A Tensor or LoDTensor representing the groundtruth labels. It shoud have the same shape, lod and data type as
input
.chunk_scheme (str) – Indicate the tagging schemes used here. The value must be IOB, IOE, IOBES or plain.
num_chunk_types (int) – The number of chunk types.
excluded_chunk_types (list, optional) – Indicate the chunk types shouldn’t be taken into account. It should be a list of chunk type ids(integer). Default None.
seq_length (Variable, optional) – A 1D Tensor containing the length of each sequence when
input
andlabel
are Tensor. It needn’t be provided ifinput
andlabel
are LoDTensor. Default None.
 Returns
A tuple including precision, recall, F1score, chunk number detected, chunk number in groundtruth, chunk number correctly detected. Each is a Tensor with shape [1]. The data type of precision, recall and F1score all is float32, and the others’ data type all is int64.
 Return type
tuple
Examples
import paddle.fluid as fluid dict_size = 10000 label_dict_len = 7 sequence = fluid.data( name='id', shape=[1, 1], lod_level=1, dtype='int64') embedding = fluid.embedding( input=sequence, size=[dict_size, 512]) hidden = fluid.layers.fc(input=embedding, size=512) label = fluid.layers.data( name='label', shape=[1], lod_level=1, dtype='int32') crf = fluid.layers.linear_chain_crf( input=hidden, label=label, param_attr=fluid.ParamAttr(name="crfw")) crf_decode = fluid.layers.crf_decoding( input=hidden, param_attr=fluid.ParamAttr(name="crfw")) fluid.layers.chunk_eval( input=crf_decode, label=label, chunk_scheme="IOB", num_chunk_types=(label_dict_len  1) / 2)
clip¶

paddle.fluid.layers.
clip
(x, min, max, name=None)[source] Clip Operator.
The clip operator limits the value of given input within an interval [min, max], just as the following equation,
$$ Out = MIN(MAX(x, min), max) $$
 Parameters
x (Variable) – Tensor, the input of clip op, data type should be float32 or float64
min (float) – float number, the minimum value to clip by
max (float) – float number, the maximum value to clip by
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Tensor, the clipped tensor, with the same shape and data type as input(x)
 Return Type:
Variable
Examples
import paddle.fluid as fluid input = fluid.data( name='data', shape=[1], dtype='float32') reward = fluid.layers.clip(x=input, min=1.0, max=1.0)
clip_by_norm¶

paddle.fluid.layers.
clip_by_norm
(x, max_norm, name=None)[source] ClipByNorm Operator.
This operator limits the L2 norm of the input \(X\) within \(max\_norm\). If the L2 norm of \(X\) is less than or equal to \(max\_norm\), \(Out\) will be the same as \(X\). If the L2 norm of \(X\) is greater than \(max\_norm\), \(X\) will be linearly scaled to make the L2 norm of \(Out\) equal to \(max\_norm\), as shown in the following formula:
$$ Out = \frac{max\_norm * X}{norm(X)}, $$
where \(norm(X)\) represents the L2 norm of \(X\).
 Parameters
x (Variable) – (Tensor) The input of clip_by_norm op and data type is float32.The number of dimensions must be between [1, 9]
max_norm (FLOAT) – (float) The maximum norm value
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
out(Variable): (Tensor) The output of clip_by_norm op with shape as input(X)The data type is float32
 Return type
Variable
Examples
import paddle.fluid as fluid input = fluid.data( name='data', shape=[None, 1], dtype='float32') reward = fluid.layers.clip_by_norm(x=input, max_norm=1.0)
continuous_value_model¶

paddle.fluid.layers.
continuous_value_model
(input, cvm, use_cvm=True)[source] continuous_value_model layers
Now, this OP is used in CTR project to remove or dispose show and click value in
input
.input
is an embedding vector including show and click value, whose shape is \([N, D]\) (N is batch size. D is 2 + embedding dim ). Show and click at first two dims of embedding vector D. Ifuse_cvm
is True, it will caculate \(log(show)\) and \(log(click)\) , and output shape is \([N, D]\) . Ifuse_cvm
is False, it will remove show and click frominput
, and output shape is \([N, D  2]\) .cvm
is show_click info, whose shape is \([N, 2]\) . Parameters
input (Variable) – The input variable. A 2D LoDTensor with shape \([N, D]\) , where N is the batch size, D is 2 + the embedding dim . lod level = 1 .
Tensor with type float32, float64. (A) –
cvm (Variable) – Show and click variable. A 2D Tensor with shape \([N, 2]\) , where N is the batch size, 2 is show and click.
Tensor with type float32, float64. –
use_cvm (bool) – Use show_click or not. if use, the output dim is the same as input. if not use, the output dim is input dim  2 (remove show and click)
 Returns
A 2D LodTensor with shape \([N, M]\) . if
use_cvm
= True, M is equal to input dim D. if False, M is equal to D  2. A Tensor with same type as input. Return type
Variable
Examples
import paddle.fluid as fluid input = fluid.data(name="input", shape=[64, 1], dtype="int64") label = fluid.data(name="label", shape=[64, 1], dtype="int64") embed = fluid.layers.embedding( input=input, size=[100, 11], dtype='float32') ones = fluid.layers.fill_constant_batch_size_like(input=label, shape=[1, 1], dtype="int64", value=1) show_clk = fluid.layers.cast(fluid.layers.concat([ones, label], axis=1), dtype='float32') show_clk.stop_gradient = True input_with_cvm = fluid.layers.continuous_value_model(embed, show_clk, True)
conv2d¶

paddle.fluid.layers.
conv2d
(input, num_filters, filter_size, stride=1, padding=0, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCHW')[source] The convolution2D layer calculates the output based on the input, filter and strides, paddings, dilations, groups parameters. Input and Output are in NCHW or NHWC format, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Filter is in MCHW format, where M is the number of output image channels, C is the number of input image channels, H is the height of the filter, and W is the width of the filter. If the groups is greater than 1, C will equal the number of input image channels divided by the groups. Please refer to UFLDL’s convolution for more details. If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.
For each input \(X\), the equation is:
\[Out = \sigma (W \ast X + b)\]Where:
\(X\): Input value, a tensor with NCHW or NHWC format.
\(W\): Filter value, a tensor with MCHW format.
\(\ast\): Convolution operation.
\(b\): Bias value, a 2D tensor with shape [M, 1].
\(\sigma\): Activation function.
\(Out\): Output value, the shape of \(Out\) and \(X\) may be different.
Example
Input:
Input shape: \((N, C_{in}, H_{in}, W_{in})\)
Filter shape: \((C_{out}, C_{in}, H_f, W_f)\)
Output:
Output shape: \((N, C_{out}, H_{out}, W_{out})\)
Where
\[\begin{split}H_{out}&= \frac{(H_{in} + 2 * paddings[0]  (dilations[0] * (H_f  1) + 1))}{strides[0]} + 1 \\ W_{out}&= \frac{(W_{in} + 2 * paddings[1]  (dilations[1] * (W_f  1) + 1))}{strides[1]} + 1\end{split}\] Parameters
input (Variable) – The input is 4D Tensor with shape [N, C, H, W], the data type of input is float16 or float32 or float64.
num_filters (int) – The number of filter. It is as same as the output image channel.
filter_size (inttuple) – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_height, filter_size_width). Otherwise, filter_size_height = filter_size_width = filter_size.
stride (inttuple) – The stride size. It means the stride in convolution. If stride is a tuple, it must contain two integers, (stride_height, stride_width). Otherwise, stride_height = stride_width = stride. Default: stride = 1.
padding (stringintlisttuple) – The padding size. It means the number of zeropaddings on both sides for each dimention.If padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If padding size is a tuple or list, it could be in three forms: [pad_height, pad_width] or [pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCHW”, padding can be in the form [[0,0], [0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NHWC”, pool_padding can be in the form [[0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.
dilation (inttuple) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain two integers, (dilation_height, dilation_width). Otherwise, dilation_height = dilation_width = dilation. Default: dilation = 1.
groups (int) – The groups number of the Conv2d Layer. According to grouped convolution in Alex Krizhevsky’s Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1.
param_attr (ParamAttrNone) – The parameter attribute for learnable parameters/weights of conv2d. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with \(Normal(0.0, std)\), and the \(std\) is \((\frac{2.0 }{filter\_elem\_num})^{0.5}\). Default: None.
bias_attr (ParamAttrboolNone) – The parameter attribute for the bias of conv2d. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.
use_cudnn (bool) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True
act (str) – Activation type, if it is set to None, activation is not appended. Default: None
name (strNone) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
data_format (str) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. The default is “NCHW”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width].
 Returns
A Variable holding Tensor representing the conv2d, whose data type is the same with input. If act is None, the tensor variable storing the convolution result, and if act is not None, the tensor variable storing convolution and nonlinearity activation result.
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32') conv2d = fluid.layers.conv2d(input=data, num_filters=2, filter_size=3, act="relu")
conv2d_transpose¶

paddle.fluid.layers.
conv2d_transpose
(input, num_filters, output_size=None, filter_size=None, padding=0, stride=1, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCHW')[source] The convolution2D transpose layer calculates the output based on the input, filter, and dilations, strides, paddings. Input(Input) and output(Output) are in NCHW or NHWC format. Where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Parameters(dilations, strides, paddings) are two elements. These two elements represent height and width, respectively. The details of convolution transpose layer, please refer to the following explanation and references therein. If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.
For each input \(X\), the equation is:
\[Out = \sigma (W \ast X + b)\]Where:
\(X\): Input value, a 4D Tensor with NCHW or NHWC format.
\(W\): Filter value, a 4D Tensor with MCHW format.
\(\ast\): Convolution operation.
\(b\): Bias value, a 2D Tensor with shape [M, 1].
\(\sigma\): Activation function.
\(Out\): Output value, a 4D Tensor with data format ‘NCHW’ or ‘NHWC’, the shape of \(Out\) and \(X\) may be different.
Example
Input:
Input shape: \((N, C_{in}, H_{in}, W_{in})\)
Filter shape: \((C_{in}, C_{out}, H_f, W_f)\)
Output:
Output shape: \((N, C_{out}, H_{out}, W_{out})\)
Where
\[\begin{split}H^\prime_{out} &= (H_{in}  1) * strides[0]  pad_height_top  pad_height_bottom + dilations[0] * (H_f  1) + 1 \\ W^\prime_{out} &= (W_{in}  1) * strides[1]  pad_width_left  pad_width_right + dilations[1] * (W_f  1) + 1 \\ H_{out} &\in [ H^\prime_{out}, H^\prime_{out} + strides[0] ] \\ W_{out} &\in [ W^\prime_{out}, W^\prime_{out} + strides[1] ]\end{split}\]Note
The conv2d_transpose can be seen as the backward of the conv2d. For conv2d, when stride > 1, conv2d maps multiple input shape to the same output shape, so for conv2d_transpose, when stride > 1, input shape maps multiple output shape. If output_size is None, \(H_{out} = H^\prime_{out}, W_{out} = W^\prime_{out}\); else, the \(H_{out}\) of the output size must between \(H^\prime_{out}\) and \(H^\prime_{out} + strides[0]\), and the \(W_{out}\) of the output size must between \(W^\prime_{out}\) and \(W^\prime_{out} + strides[1]\), conv2d_transpose can compute the kernel size automatically.
 Parameters
input (Variable) – 4D Tensor with [N, C, H, W] or [N, H, W, C] format, its data type is float32 or float64.
num_filters (int) – The number of the filter. It is as same as the output image channel.
output_size (inttuple, optional) – The output image size. If output size is a tuple, it must contain two integers, (image_height, image_width). None if use filter_size, padding, and stride to calculate output_size. If output_size and filter_size are specified at the same time, They should follow the formula above. Default: None. output_size and filter_size should not be None at the same time.
filter_size (inttuple, optional) – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_height, filter_size_width). Otherwise, filter_size_height = filter_size_width = filter_size. None if use output size to calculate filter_size. Default: None. filter_size and output_size should not be None at the same time.
stride (inttuple, optional) – The stride size. It means the stride in transposed convolution. If stride is a tuple, it must contain two integers, (stride_height, stride_width). Otherwise, stride_height = stride_width = stride. Default: stride = 1.
padding (intliststrtuple, optional) –
 The padding size. The padding argument effectively adds
dilation * (kernel  1) amount of zeropadding on both sides of input. If padding is a string, either ‘VALID’ or ‘SAME’ supported, which is the padding algorithm. If padding is a tuple or list, it could be in three forms: [pad_height, pad_width] or
[pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is ‘NCHW’, padding can be in the form [[0,0], [0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is ‘NHWC’, padding can be in the form [[0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.
dilation (inttuple, optional) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain two integers, (dilation_height, dilation_width). Otherwise, dilation_height = dilation_width = dilation. Default: dilation = 1.
filter_size – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_height, filter_size_width). Otherwise, filter_size_height = filter_size_width = filter_size. None if use output size to calculate filter_size. Default: None.
groups (int, optional) – The groups number of the Conv2d transpose layer. Inspired by grouped convolution in Alex Krizhevsky’s Deep CNN paper, in which when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups = 1.
param_attr (ParamAttr, optional) – The parameter attribute for learnable parameters/weights of conv2d_transpose. If it is set to None or one attribute of ParamAttr, conv2d_transpose will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.
bias_attr (ParamAttrbool, optional) – The parameter attribute for the bias of conv2d_transpose. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv2d_transpose will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.
use_cudnn (bool, optional) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True.
act (str, optional) – Activation type, if it is set to None, activation is not appended. Default: None.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
data_format (str, optional) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width]. Default: ‘NCHW’.
 Returns
A Variable holding Tensor representing the conv2d_transpose, whose data type is the same with input and shape is (num_batches, channels, out_h, out_w) or (num_batches, out_h, out_w, channels). If act is None, the tensor variable storing the transposed convolution result, and if act is not None, the tensor variable storing transposed convolution and nonlinearity activation result.
 Raises
ValueError
– If the shapes of output, input, filter_size, stride, padding and groups mismatch.
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32') conv2d_transpose = fluid.layers.conv2d_transpose(input=data, num_filters=2, filter_size=3)
conv3d¶

paddle.fluid.layers.
conv3d
(input, num_filters, filter_size, stride=1, padding=0, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCDHW')[source] The convolution3D layer calculates the output based on the input, filter and strides, paddings, dilations, groups parameters. Input(Input) and Output(Output) are in NCDHW or NDHWC format. Where N is batch size C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. Convlution3D is similar with Convlution2D but adds one dimension(depth). If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.
For each input \(X\), the equation is:
\[Out = \sigma (W \ast X + b)\]In the above equation:
\(X\): Input value, a tensor with NCDHW or NDHWC format.
\(W\): Filter value, a tensor with MCDHW format.
\(\ast\): Convolution operation.
\(b\): Bias value, a 2D tensor with shape [M, 1].
\(\sigma\): Activation function.
\(Out\): Output value, the shape of \(Out\) and \(X\) may be different.
Example
Input:
Input shape: \((N, C_{in}, D_{in}, H_{in}, W_{in})\)
Filter shape: \((C_{out}, C_{in}, D_f, H_f, W_f)\)
Output: Output shape: \((N, C_{out}, D_{out}, H_{out}, W_{out})\)
Where
\[\begin{split}D_{out}&= \frac{(D_{in} + 2 * paddings[0]  (dilations[0] * (D_f  1) + 1))}{strides[0]} + 1 \\ H_{out}&= \frac{(H_{in} + 2 * paddings[1]  (dilations[1] * (H_f  1) + 1))}{strides[1]} + 1 \\ W_{out}&= \frac{(W_{in} + 2 * paddings[2]  (dilations[2] * (W_f  1) + 1))}{strides[2]} + 1\end{split}\] Parameters
input (Variable) – The input is 5D Tensor with shape [N, C, D, H, W], the data type of input is float16 or float32 or float64.
num_filters (int) – The number of filter. It is as same as the output image channel.
filter_size (inttuple) – The filter size. If filter_size is a tuple, it must contain three integers, (filter_size_depth, filter_size_height, filter_size_width). Otherwise, filter_size_depth = filter_size_height = filter_size_width = filter_size.
stride (inttuple) – The stride size. It means the stride in convolution. If stride is a tuple, it must contain three integers, (stride_depth, stride_height, stride_width). Otherwise, stride_depth = stride_height = stride_width = stride. Default: stride = 1.
padding (stringintlisttuple) – The padding size. It means the number of zeropaddings on both sides for each dimention. If padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If padding size is a tuple or list, it could be in three forms: [pad_depth, pad_height, pad_width] or [pad_depth_front, pad_depth_back, pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCDHW”, pool_padding can be in the form [[0,0], [0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NDHWC”, pool_padding can be in the form [[0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.
dilation (inttuple) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain three integers, (dilation_depth, dilation_height, dilation_width). Otherwise, dilation_depth = dilation_height = dilation_width = dilation. Default: dilation = 1.
groups (int) – The groups number of the Conv3d Layer. According to grouped convolution in Alex Krizhevsky’s Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1
param_attr (ParamAttrNone) – The parameter attribute for learnable parameters/weights of conv3d. If it is set to None or one attribute of ParamAttr, conv3d will create ParamAttr as param_attr. If it is set to None, the parameter is initialized with \(Normal(0.0, std)\), and the \(std\) is \((\frac{2.0 }{filter\_elem\_num})^{0.5}\). Default: None.
bias_attr (ParamAttrboolNone) – The parameter attribute for the bias of conv3d. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv3d will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.
use_cudnn (bool) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True
act (str) – Activation type, if it is set to None, activation is not appended. Default: None.
name (strNone) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
data_format (str) – The data format of the input and output data. An optional string from: “NCDHW”, “NDHWC”. The default is “NCDHW”. When it is “NCDHW”, the data is stored in the order of: [batch_size, input_channels, input_depth, input_height, input_width].
 Returns
A Variable holding Tensor representing the conv3d, whose data type is the same with input. If act is None, the tensor variable storing the convolution result, and if act is not None, the tensor variable storing convolution and nonlinearity activation result.
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 12, 32, 32], dtype='float32') conv3d = fluid.layers.conv3d(input=data, num_filters=2, filter_size=3, act="relu")
conv3d_transpose¶

paddle.fluid.layers.
conv3d_transpose
(input, num_filters, output_size=None, filter_size=None, padding=0, stride=1, dilation=1, groups=None, param_attr=None, bias_attr=None, use_cudnn=True, act=None, name=None, data_format='NCDHW')[source] The convolution3D transpose layer calculates the output based on the input, filter, and dilations, strides, paddings. Input(Input) and output(Output) are in NCDHW or NDHWC format. Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature. Parameters(dilations, strides, paddings) are two elements. These two elements represent height and width, respectively. The details of convolution transpose layer, please refer to the following explanation and references therein. If bias attribution and activation type are provided, bias is added to the output of the convolution, and the corresponding activation function is applied to the final result.
For each input \(X\), the equation is:
\[Out = \sigma (W \ast X + b)\]In the above equation:
\(X\): Input value, a Tensor with NCDHW or NDHWC format.
\(W\): Filter value, a Tensor with MCDHW format.
\(\ast\): Convolution operation.
\(b\): Bias value, a 2D Tensor with shape [M, 1].
\(\sigma\): Activation function.
\(Out\): Output value, the shape of \(Out\) and \(X\) may be different.
Example
Input:
Input shape: \((N, C_{in}, D_{in}, H_{in}, W_{in})\)
Filter shape: \((C_{in}, C_{out}, D_f, H_f, W_f)\)
Output:
Output shape: \((N, C_{out}, D_{out}, H_{out}, W_{out})\)
Where
\[\begin{split}D^\prime_{out} &= (D_{in}  1) * strides[0]  2 * paddings[0] + dilations[0] * (D_f  1) + 1 \\ H^\prime_{out} &= (H_{in}  1) * strides[1]  2 * paddings[1] + dilations[1] * (H_f  1) + 1 \\ W^\prime_{out} &= (W_{in}  1) * strides[2]  2 * paddings[2] + dilations[2] * (W_f  1) + 1 \\ D_{out} &\in [ D^\prime_{out}, D^\prime_{out} + strides[0] ] \\ H_{out} &\in [ H^\prime_{out}, H^\prime_{out} + strides[1] ] \\ W_{out} &\in [ W^\prime_{out}, W^\prime_{out} + strides[2] ]\end{split}\]Note
The conv3d_transpose can be seen as the backward of the conv3d. For conv3d, when stride > 1, conv3d maps multiple input shape to the same output shape, so for conv3d_transpose, when stride > 1, input shape maps multiple output shape. If output_size is None, \(H_{out} = H^\prime_{out}, :math:\); else, the \(D_{out}\) of the output size must between \(D^\prime_{out}\) and \(D^\prime_{out} + strides[0]\), the \(H_{out}\) of the output size must between \(H^\prime_{out}\) and \(H^\prime_{out} + strides[1]\), and the \(W_{out}\) of the output size must between \(W^\prime_{out}\) and \(W^\prime_{out} + strides[2]\), conv3d_transpose can compute the kernel size automatically.
 Parameters
input (Variable) – The input is 5D Tensor with shape [N, C, D, H, W] or [N, D, H, W, C], the data type of input is float32 or float64.
num_filters (int) – The number of the filter. It is as same as the output image channel.
output_size (inttuple, optional) – The output image size. If output size is a tuple, it must contain three integers, (image_depth, image_height, image_width). This parameter only works when filter_size is None. If output_size and filter_size are specified at the same time, They should follow the formula above. Default: None. Output_size and filter_size should not be None at the same time.
filter_size (inttuple, optional) – The filter size. If filter_size is a tuple, it must contain three integers, (filter_size_depth, filter_size_height, filter_size_width). Otherwise, filter_size_depth = filter_size_height = filter_size_width = filter_size. None if use output size to calculate filter_size. Default: None. filter_size and output_size should not be None at the same time.
padding (intliststrtuple, optional) –
 The padding size. The padding argument effectively
adds dilation * (kernel  1) amount of zeropadding on both sides of input. If padding is a string, either ‘VALID’ or ‘SAME’ supported, which is the padding algorithm. If padding is a tuple or list, it could be in three forms: [pad_depth, pad_height, pad_width] or
[pad_depth_front, pad_depth_back, pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is ‘NCDHW’, padding can be in the form [[0,0], [0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is ‘NDHWC’, padding can be in the form [[0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Default: padding = 0.
stride (inttuple, optional) – The stride size. It means the stride in transposed convolution. If stride is a tuple, it must contain three integers, (stride_depth, stride_height, stride_width). Otherwise, stride_depth = stride_height = stride_width = stride. Default: stride = 1.
dilation (inttuple, optional) – The dilation size. It means the spacing between the kernel points. If dilation is a tuple, it must contain three integers, (dilation_depth, dilation_height, dilation_width). Otherwise, dilation_depth = dilation_height = dilation_width = dilation. Default: dilation = 1.
groups (int, optional) – The groups number of the Conv3d transpose layer. Inspired by grouped convolution in Alex Krizhevsky’s Deep CNN paper, in which when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1
param_attr (ParamAttr, optional) – The parameter attribute for learnable parameters/weights of conv3d_transpose. If it is set to None or one attribute of ParamAttr, conv3d_transpose will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.
bias_attr (ParamAttrbool, optional) – The parameter attribute for the bias of conv3d_transpose. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv3d_transpose will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.
use_cudnn (bool, optional) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True
act (str, optional) – Activation type, if it is set to None, activation is not appended. Default: None.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
data_format (str, optional) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width]. Default: ‘NCDHW’.
 Returns
A Variable holding Tensor representing the conv3d_transpose, whose data type is the same with input and shape is (num_batches, channels, out_d, out_h, out_w) or (num_batches, out_d, out_h, out_w, channels). If act is None, the tensor variable storing the transposed convolution result, and if act is not None, the tensor variable storing transposed convolution and nonlinearity activation result.
 Raises
ValueError
– If the shapes of output, input, filter_size, stride, padding and groups mismatch.
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 12, 32, 32], dtype='float32') conv3d_transpose = fluid.layers.conv3d_transpose(input=data, num_filters=2, filter_size=3)
cos_sim¶

paddle.fluid.layers.
cos_sim
(X, Y)[source] Cosine Similarity Operator
\(Out = \frac{X^T * Y}{(\sqrt{X^T * X} * \sqrt{Y^T * Y})}\)
The input X and Y must have the same shape, except that the 1st dimension of input Y could be just 1 (different from input X), which will be broadcasted to match the shape of input X before computing their cosine similarity.
Both the input X and Y can carry the LoD (Level of Details) information, or not. But the output only shares the LoD information with input X.
 Parameters
X (Variable) – The 1st input of cos_sim op, LoDTensor with shape
[N_1, N_2, ..., N_k]
, the data type is float32.Y (Variable) – The 2nd input of cos_sim op, Tensor with shape
[N_1 or 1, N_2, ..., N_k]
, the data type is float32.
 Returns
A Variable holding LoDTensor representing the output of cosine(X, Y).
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[3, 7], dtype='float32') y = fluid.data(name='y', shape=[1, 7], dtype='float32') out = fluid.layers.cos_sim(x, y)
crf_decoding¶

paddle.fluid.layers.
crf_decoding
(input, param_attr, label=None, length=None)[source] The crf_decoding operator reads the emission feature weights and the transition feature weights learned by the linear_chain_crf operator and performs decoding. It implements the Viterbi algorithm which is a dynamic programming algorithm for finding the most likely sequence of hidden states, called the Viterbi path, that results in a sequence of observed tags.
The output of this operator changes according to whether Input(Label) is given:
Input(Label) is given: This happens in training. This operator is used to cowork with the chunk_eval operator. When Input(Label) is given, the crf_decoding operator returns tensor with the sampe shape as Input(Label) whose values are fixed to be 0, indicating an incorrect prediction, or 1 indicating a tag is correctly predicted. Such an output is the input to chunk_eval operator.
Input(Label) is not given: This is the standard decoding process.
The crf_decoding operator returns a row vector with shape [N x 1]/[B x S], here the shape depends on the inputs are LoDTensors or common tensors, whose values range from 0 to maximum tag number  1, Each element indicates an index of a predicted tag.
 Parameters
input (Variable) – (Tensor/LoDTensor). For a LoDTensor input, its shape is [N x D] where N is the total sequence length of the minibatch and D is the total tag number. While for a tensor input, its shape is [B X S X D] with B the batch size and S the sequence length of each sample after padding. This input is the unscaled emission weight matrix of the linear_chain_crf operator. The data type is float32 or float64
param_attr (ParamAttrNone) – To specify the weight parameter attribute. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
label (Variable, optional) – (Tensor/LoDTensor). The ground truth with shape [N x 1] (for LoDTensor) or [B x S] (for Tensor). This input is optional. See more details in the operator’s comments. The data type is int64
length (Variable, optional) – (Tensor). The actual length of each sample before padding with shape [B x 1]. It means the Input(Emission), Input(Label) and Output(ViterbiPath) are common tensors with padding when this input is given. The data type is int64
 Returns
(Tensor/LoDTensor). The decoding results. What to return changes depending on whether the Input(Label) (the ground truth) is given. See more details in the operator’s comment. The data type is int64
 Return type
Variable
Examples
import paddle.fluid as fluid # LoDTensorbased example num_labels = 10 feature = fluid.data(name='word_emb', shape=[1, 784], dtype='float32', lod_level=1) label = fluid.data(name='label', shape=[1, 1], dtype='int64', lod_level=1) emission = fluid.layers.fc(input=feature, size=num_labels) crf_cost = fluid.layers.linear_chain_crf(input=emission, label=label, param_attr=fluid.ParamAttr(name="crfw")) crf_decode = fluid.layers.crf_decoding(input=emission, param_attr=fluid.ParamAttr(name="crfw")) # Common tensor example num_labels, max_len = 10, 20 feature = fluid.data(name='word_emb_pad', shape=[1, max_len, 784], dtype='float32') label = fluid.data(name='label_pad', shape=[1, max_len, 1], dtype='int64') length = fluid.data(name='length', shape=[1, 1], dtype='int64') emission = fluid.layers.fc(input=feature, size=num_labels, num_flatten_dims=2) crf_cost = fluid.layers.linear_chain_crf(input=emission, label=label, length=length, param_attr=fluid.ParamAttr(name="crfw_pad")) crf_decode = fluid.layers.crf_decoding(input=emission, length=length, param_attr=fluid.ParamAttr(name="crfw_pad"))
crop¶

paddle.fluid.layers.
crop
(x, shape=None, offsets=None, name=None)[source] Crop input into output, as specified by offsets and shape.
Warning: THIS OP IS DEPRECATED. It will be removed in the future version. Instructions for updating: Use api_fluid_layers_crop_tensor instead.
* Case 1: Given X = [[0, 1, 2, 0, 0] [0, 3, 4, 0, 0] [0, 0, 0, 0, 0]], and shape = [2, 2], offsets = [0, 1], output is: Out = [[1, 2], [3, 4]]. * Case 2: Given X = [[0, 1, 2, 5, 0] [0, 3, 4, 6, 0] [0, 0, 0, 0, 0]], and shape is tensor shape = [[0, 0, 0] [0, 0, 0]] and offsets = [0, 1], output is: Out = [[1, 2, 5], [3, 4, 6]].
 Parameters
x (Variable) – Tensor, data type can be float32 or float64.
shape (Variablelist/tuple of integers) – The output shape is specified by shape, which can be a Tensor or a list/tuple of integers. If it is a Tensor, it’s rank must be the same as x , only it’s shape will be used, and the value of it will be ignored. This way is suitable for the case that the output shape may be changed each iteration. If it is a list/tuple of integers, it’s length must be the same as the rank of x
offsets (Variablelist/tuple of integersNone) – Specifies the cropping offsets at each dimension. It can be a Tensor or a list/tuple of integers. If it is a Tensor, it’s rank must be the same as x. This way is suitable for the case that the offsets may be changed each iteration. If it is a list/tuple of integers, it’s length must be the same as the rank of x. If None, the offsets are 0 at each dimension.
name (str, optional) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.
 Returns
The cropped Tensor, which has the same rank and data type with x
 Return Type:
Variable
 Raises
ValueError
– If shape is not a list, tuple or Variable.
Examples
import paddle.fluid as fluid x = fluid.data(name="x", shape=[3, 3, 5], dtype="float32") y = fluid.data(name="y", shape=[2, 2, 3], dtype="float32") crop = fluid.layers.crop(x, shape=y) # or z = fluid.data(name="z", shape=[3, 3, 5], dtype="float32") crop = fluid.layers.crop(z, shape=[2, 2, 3])
cross_entropy¶

paddle.fluid.layers.
cross_entropy
(input, label, soft_label=False, ignore_index=100)[source] This operator computes the cross entropy between input and label. It supports both hardlabel and and softlabel cross entropy computation.
Hardlabel cross entropy: if soft_label=False, \(label[i_1, i_2, ..., i_k]\) is the hard label of each sample.
\[output[i_1, i_2, ..., i_k]=log(input[i_1, i_2, ..., i_k, j]), label[i_1, i_2, ..., i_k] = j, j != ignore\_index\]Softlabel cross entropy: if soft_label=True, \(label[i_1, i_2, ..., i_k, j]\) is the soft label of each sample corresponding to the jth class.
\[output[i_1, i_2, ..., i_k]= \sum_{j}label[i_1,i_2,...,i_k,j]*log(input[i_1, i_2, ..., i_k,j])\]
 Parameters
input (Variable) – a multidimensional Tensor with shape \([N_1, N_2, ..., N_k, D]\), where the last dimension D is the class number. The data type should be float32 or float64.
label (Variable) – label value corresponding to input. If soft_label=False, the dimension of label should be \([N_1, N_2, ..., N_k]\) or \([N_1, N_2, ..., N_k, 1]\) , and its data type should be int64, and the value must be inside [0, D). If soft_label=True, the shape, data type of label should be the same with input, and the sum of soft label value of each sample should be 1.
soft_label (bool) – indicate whether label is soft. Default False, meaning that the label is hard. If soft_label=True, the label is soft.
ignore_index (int) – specify an ignorable label value. The ignored label would be omitted when computing. If it is a negative integer, no label would be ignored. Only valid when soft_label=False. Default 100.
 Returns
A Variable holding Tensor representing the cross entropy, whose data type is the same with input. If soft_label=False, the shape of output is the same with label. If soft_label=True, the shape of output is \([N_1, N_2, ..., N_k, 1]\) .
Examples
import paddle.fluid as fluid class_num = 7 x = fluid.data(name='x', shape=[None, 3, 10], dtype='float32') label = fluid.data(name='label', shape=[None, 1], dtype='int64') predict = fluid.layers.fc(input=x, size=class_num, act='softmax') cost = fluid.layers.cross_entropy(input=predict, label=label)
ctc_greedy_decoder¶

paddle.fluid.layers.
ctc_greedy_decoder
(input, blank, input_length=None, padding_value=0, name=None)[source] This op is used to decode sequences by greedy policy by the following steps:
Get the indexes of maximum value for each row in input. a.k.a. numpy.argmax(input, axis=0).
For each sequence in result of step1, merge repeated tokens between two blanks and delete all blanks.
This op is implemented in two modes: lod and padding, either of them can be used. The input can be either LoDTensor or Tensor, corresponding to lod and padding mode respectively.
A simple example as below:
Given: (1) for lod mode: input.data = [[0.6, 0.1, 0.3, 0.1], [0.3, 0.2, 0.4, 0.1], [0.1, 0.5, 0.1, 0.3], [0.5, 0.1, 0.3, 0.1], [0.5, 0.1, 0.3, 0.1], [0.2, 0.2, 0.2, 0.4], [0.2, 0.2, 0.1, 0.5], [0.5, 0.1, 0.3, 0.1]] input.lod = [[4, 4]] Computation: step1: Apply argmax to first input sequence which is input.data[0:4]. Then we get: [[0], [2], [1], [0]] step2: merge repeated tokens and remove blank which is 0. Then we get first output sequence: [[2], [1]] Finally: output.data = [[2], [1], [3]] output.lod = [[2, 1]] (2) for padding mode: input.data = [[[0.6, 0.1, 0.3, 0.1], [0.3, 0.2, 0.4, 0.1], [0.1, 0.5, 0.1, 0.3], [0.5, 0.1, 0.3, 0.1]], [[0.5, 0.1, 0.3, 0.1], [0.2, 0.2, 0.2, 0.4], [0.2, 0.2, 0.1, 0.5], [0.5, 0.1, 0.3, 0.1]]] input_length.data = [[4], [4]] input.shape = [2, 4, 4] step1: Apply argmax to first input sequence which is input.data[0:4]. Then we get: [[0], [2], [1], [0]], for input.data[4:8] is [[0], [3], [3], [0]], shape is [2,4,1] step2: Change the argmax result to use padding mode, then argmax result is [[0, 2, 1, 0], [0, 3, 3, 0]], shape is [2, 4], lod is [], input_length is [[4], [4]] step3: Apply ctc_align to padding argmax result, padding_value is 0 Finally: output.data = [[2, 1, 0, 0], [3, 0, 0, 0]] output_length.data = [[2], [1]]
 Parameters
input (Variable) – the probabilities of variablelength sequences. When in lod mode, it is a 2D LoDTensor with LoD information. It’s shape is [Lp, num_classes + 1] where Lp is the sum of all input sequences’ length and num_classes is the true number of classes. When in padding mode, it is a 3D Tensor with padding, It’s shape is [batch_size, N, num_classes + 1]. (not including the blank label). The data type can be float32 or float64.
blank (int) – the blank label index of Connectionist Temporal Classification (CTC) loss, which is in the halfopened interval [0, num_classes + 1).
input_length (Variable, optional) – 2D LoDTensor, shape is [batch_size, 1], data type is int64. It is used for padding mode. In lod mode, input_length is None.
padding_value (int) – padding value.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
For lod mode, returns the result of CTC greedy decoder, 2D LoDTensor, shape is [Lp, 1], data type is int64. ‘Lp’ is the sum of all output sequences’ length. If all the sequences in result were empty, the result LoDTensor will be [1] with empty LoD [[]].
For padding mode, returns a tuple of (output, output_length), which was describled as below:
output, 2D Tensor, shape is [batch_size, N], data type is int64.
output_length, 2D Tensor, shape is [batch_size, 1], data type is int64. It is the length of each sequence of output for padding mode.
 Return type:
For lod mode: Variable
For padding mode: tuple of two Variables (output, output_length).
Examples
# for lod mode import paddle.fluid as fluid x = fluid.data(name='x', shape=[None, 8], dtype='float32', lod_level=1) cost = fluid.layers.ctc_greedy_decoder(input=x, blank=0) # for padding mode x_pad = fluid.data(name='x_pad', shape=[10, 4, 8], dtype='float32') x_pad_len = fluid.data(name='x_pad_len', shape=[10, 1], dtype='int64') out, out_len = fluid.layers.ctc_greedy_decoder(input=x_pad, blank=0, input_length=x_pad_len)
data_norm¶

paddle.fluid.layers.
data_norm
(input, act=None, epsilon=1e05, param_attr=None, data_layout='NCHW', in_place=False, name=None, moving_mean_name=None, moving_variance_name=None, do_model_average_for_mean_and_var=True)[source] Data Normalization Layer
This op can be used as a normalizer function for conv2d and fully_connected operations. The required data format for this layer is one of the following:
NHWC [batch, in_height, in_width, in_channels]
NCHW [batch, in_channels, in_height, in_width]
\(input\) is the input features over a minibatch.
\[\begin{split}\mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\ \ minibatch\ mean \\ \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i  \ \mu_{\beta})^2 \qquad &//\ minibatch\ variance \\ \hat{x_i} &\gets \frac{x_i  \mu_\beta} {\sqrt{\ \sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift\end{split}\] Parameters
input (variable) – The input variable which is a LoDTensor.
act (string, Default None) – Activation type, linearreluprelu…
epsilon (float, Default 1e05) –
param_attr (ParamAttr) – The parameter attribute for Parameter scale.
data_layout (string, default NCHW) – NCHWNHWC
in_place (bool, Default False) – Make the input and output of batch norm reuse memory.
name (string, Default None) – A name for this layer(optional). If set None, the layer will be named automatically.
moving_mean_name (string, Default None) – The name of moving_mean which store the global Mean.
moving_variance_name (string, Default None) – The name of the moving_variance which store the global Variance.
do_model_average_for_mean_and_var (bool, Default True) – Whether parameter mean and variance should do model average when model average is enabled.
 Returns
A tensor variable which is the result after applying data normalization on the input.
 Return type
Variable
Examples
import paddle.fluid as fluid hidden1 = fluid.data(name="hidden1", shape=[64, 200]) hidden2 = fluid.layers.data_norm(name="hidden2", input=hidden1)
deformable_conv¶

paddle.fluid.layers.
deformable_conv
(input, offset, mask, num_filters, filter_size, stride=1, padding=0, dilation=1, groups=None, deformable_groups=None, im2col_step=None, param_attr=None, bias_attr=None, modulated=True, name=None)[source] Deformable Convolution op
Compute 2D deformable convolution on 4D input. Given input image x, output feature map y, the deformable convolution operation can be expressed as follow:
Deformable Convolution v2:
\[y(p) = \sum_{k=1}^{K}{w_k * x(p + p_k + \Delta p_k) * \Delta m_k}\]Deformable Convolution v1:
\[y(p) = \sum_{k=1}^{K}{w_k * x(p + p_k + \Delta p_k)}\]Where \(\Delta p_k\) and \(\Delta m_k\) are the learnable offset and modulation scalar for the kth location, Which \(\Delta m_k\) is one in deformable convolution v1. Please refer to Deformable ConvNets v2: More Deformable, Better Results and Deformable Convolutional Networks.
Example
Input:
Input shape: \((N, C_{in}, H_{in}, W_{in})\)
Filter shape: \((C_{out}, C_{in}, H_f, W_f)\)
Offset shape: \((N, 2 * deformable\_groups * H_f * H_w, H_{in}, W_{in})\)
Mask shape: \((N, deformable\_groups * H_f * H_w, H_{in}, W_{in})\)
Output:
Output shape: \((N, C_{out}, H_{out}, W_{out})\)
Where
\[\begin{split}H_{out}&= \frac{(H_{in} + 2 * paddings[0]  (dilations[0] * (H_f  1) + 1))}{strides[0]} + 1 \\ W_{out}&= \frac{(W_{in} + 2 * paddings[1]  (dilations[1] * (W_f  1) + 1))}{strides[1]} + 1\end{split}\] Parameters
input (Variable) – The input image with [N, C, H, W] format. A Tensor with type float32, float64.
offset (Variable) – The input coordinate offset of deformable convolution layer. A Tensor with type float32, float64.
Mask (Variable, Optional) – The input mask of deformable covolution layer. A Tensor with type float32, float64.It should be None when you use deformable_conv_v2.
num_filters (int) – The number of filter. It is as same as the output image channel.
filter_size (inttuple) – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_H, filter_size_W). Otherwise, the filter will be a square.
stride (inttuple) – The stride size. If stride is a tuple, it must contain two integers, (stride_H, stride_W). Otherwise, the stride_H = stride_W = stride. Default: stride = 1.
padding (inttuple) – The padding size. If padding is a tuple, it must contain two integers, (padding_H, padding_W). Otherwise, the padding_H = padding_W = padding. Default: padding = 0.
dilation (inttuple) – The dilation size. If dilation is a tuple, it must contain two integers, (dilation_H, dilation_W). Otherwise, the dilation_H = dilation_W = dilation. Default: dilation = 1.
groups (int) – The groups number of the deformable conv layer. According to grouped convolution in Alex Krizhevsky’s Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1.
deformable_groups (int) – The number of deformable group partitions. Default: deformable_groups = 1.
im2col_step (int) – Maximum number of images per im2col computation; The total batch size should be divisable by this value or smaller than this value; if you face out of memory problem, you can try to use a smaller value here. Default: im2col_step = 64.
param_attr (ParamAttr, Optional) – The parameter attribute for learnable parameters/weights of deformable conv. If it is set to None or one attribute of ParamAttr, deformable conv will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with \(Normal(0.0, std)\), and the \(std\) is \((\frac{2.0 }{filter\_elem\_num})^{0.5}\). Default: None.
bias_attr (ParamAttrbool, Optional) – The parameter attribute for the bias of deformable conv layer. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.
modulated (bool) – Make sure which version should be used between v1 and v2, where v2 is used while True. Default: True.
name (str, Optional) – For details, please refer to api_guide_Name. Generally, no setting is required. Default: None.
 Returns
The tensor variable storing the deformable convolution result. A Tensor with type float32, float64.
 Return type
Variable
 Raises
ValueError
– If the shapes of input, filter_size, stride, padding and groups mismatch.
Examples
#deformable conv v2: import paddle.fluid as fluid C_in, H_in, W_in = 3, 32, 32 filter_size, deformable_groups = 3, 1 data = fluid.data(name='data', shape=[None, C_in, H_in, W_in], dtype='float32') offset = fluid.data(name='offset', shape=[None, 2*deformable_groups*filter_size**2, H_in, W_in], dtype='float32') mask = fluid.data(name='mask', shape=[None, deformable_groups*filter_size**2, H_in, W_in], dtype='float32') out = fluid.layers.deformable_conv(input=data, offset=offset, mask=mask, num_filters=2, filter_size=filter_size, padding=1, modulated=True) #deformable conv v1: import paddle.fluid as fluid C_in, H_in, W_in = 3, 32, 32 filter_size, deformable_groups = 3, 1 data = fluid.data(name='data', shape=[None, C_in, H_in, W_in], dtype='float32') offset = fluid.data(name='offset', shape=[None, 2*deformable_groups*filter_size**2, H_in, W_in], dtype='float32') out = fluid.layers.deformable_conv(input=data, offset=offset, mask=None, num_filters=2, filter_size=filter_size, padding=1, modulated=False)
deformable_roi_pooling¶

paddle.fluid.layers.
deformable_roi_pooling
(input, rois, trans, no_trans=False, spatial_scale=1.0, group_size=[1, 1], pooled_height=1, pooled_width=1, part_size=None, sample_per_part=1, trans_std=0.1, position_sensitive=False, name=None)[source] Deformable ROI Pooling Layer
Performs deformable regionofinterest pooling on inputs. As described in Deformable Convolutional Networks, it will get offset for each bin after roi pooling so that pooling at correct region. Batch_size will change to the number of region bounding boxes after deformable_roi_pooling.
The operation has three steps:
Dividing each region proposal into equalsized sections with the pooled_width and pooled_height.
Add offset to pixel in ROI to get new location and the new value which are computed directly through bilinear interpolation with four nearest pixel.
Sample several points in each bin to get average values as output.
 Parameters
input (Variable) – The input of deformable roi pooling and it is tensor which value type is float32. The shape of input is [N, C, H, W]. Where N is batch size, C is number of input channels, H is height of the feature, and W is the width of the feature.
rois (Variable) – ROIs (Regions of Interest) with type float32 to pool over. It should be a 2D LoDTensor of shape (num_rois, 4), and the lod level is 1. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates, which value type is float32.
trans (Variable) – Offset of features on ROIs while pooling which value type is float32. The format is [N, C, H, W], where N is number of ROIs, C is number of channels, which indicate the offset distance in the x and y directions, H is pooled height, and W is pooled width.
no_trans (bool) – Whether to add offset to get new value or not while roi pooling, which value with type bool is True or False. If value is True, no offset will be added in operation. Default: False.
spatial_scale (float) – Ratio of input feature map height (or width) to raw image height (or width), which value type is float32. Equals the reciprocal of total stride in convolutional layers, Default: 1.0.
group_size (listtuple) – The number of groups which input channels are divided and the input is list or tuple, which value type is int32. (eg.number of input channels is k1 * k2 * (C + 1), which k1 and k2 are group width and height and C+1 is number of output chanels.) eg.(4, 6), which 4 is height of group and 6 is width of group. Default: [1, 1].
pooled_height (int) – The pooled output height which value type is int32. Default: 1.
pooled_width (int) – The pooled output width which value type is int32. Default: 1.
part_size (listtuple) – The height and width of offset which values in list or tuple is int32, eg.(4, 6), which height is 4 and width is 6, and values always equal to pooled_height and pooled_width. Default: if None, default value is [pooled_height, pooled_width].
sample_per_part (int) – The number of samples in each bin which value type is int32. If value is bigger, it will consume more performance. Default: 1.
trans_std (float) – Coefficient of offset which value type is float32. It controls weight of offset. Default: 0.1.
position_sensitive (bool) – Whether to choose deformable psroi pooling mode or not, and value type is bool(True or False). If value is False, input dimension equals to output dimension. If value is True, input dimension shoule be output dimension * pooled_height * pooled_width. Default: False.
name (strNone) – Name of layer. Default: None.
 Returns
Output of deformable roi pooling is that, if position sensitive is False, input dimension equals to output dimension. If position sensitive is True, input dimension should be the result of output dimension divided by pooled height and pooled width.
 Return type
Variable
Examples
# position_sensitive=True import paddle.fluid as fluid input = fluid.data(name="input", shape=[2, 192, 64, 64], dtype='float32') rois = fluid.data(name="rois", shape=[1, 4], dtype='float32', lod_level=1) trans = fluid.data(name="trans", shape=[2, 384, 64, 64], dtype='float32') x = fluid.layers.deformable_roi_pooling(input=input, rois=rois, trans=trans, no_trans=False, spatial_scale=1.0, group_size=(1, 1), pooled_height=8, pooled_width=8, part_size=(8, 8), sample_per_part=4, trans_std=0.1, position_sensitive=True) # position_sensitive=False import paddle.fluid as fluid input = fluid.data(name="input", shape=[2, 192, 64, 64], dtype='float32') rois = fluid.data(name="rois", shape=[1, 4], dtype='float32', lod_level=1) trans = fluid.data(name="trans", shape=[2, 384, 64, 64], dtype='float32') x = fluid.layers.deformable_roi_pooling(input=input, rois=rois, trans=trans, no_trans=False, spatial_scale=1.0, group_size=(1, 1), pooled_height=8, pooled_width=8, part_size=(8, 8), sample_per_part=4, trans_std=0.1, position_sensitive=False)
dice_loss¶

paddle.fluid.layers.
dice_loss
(input, label, epsilon=1e05, name=None)[source] Dice loss for comparing the similarity between the input predictions and the label. This implementation is for binary classification, where the input is sigmoid predictions of each pixel, usually used for segmentation task. The dice loss can be defined as the following equation:
\[\begin{split}dice\_loss &= 1  \frac{2 * intersection\_area}{total\_area} \\ &= \frac{(total\_area  intersection\_area)  intersection\_area}{total\_area} \\ &= \frac{(union\_area  intersection\_area)}{total\_area}\end{split}\] Parameters
input (Variable) – Tensor, rank>=2, shape is \([N_1, N_2, ..., N_D]\), where \(N_1\) is the batch_size, \(N_D\) is 1. It is usually the output predictions of sigmoid activation. The data type can be float32 or float64.
label (Variable) – Tensor, the groud truth with the same rank as input, shape is \([N_1, N_2, ..., N_D]\). where \(N_1\) is the batch_size, \(N_D\) is 1. The data type can be float32 or float64.
epsilon (float) – The epsilon will be added to the numerator and denominator. If both input and label are empty, it makes sure dice is 1. Default: 0.00001
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
The dice loss with shape [1], data type is the same as input .
 Return Type:
Varaible
Example
import paddle.fluid as fluid x = fluid.data(name='data', shape = [3, 224, 224, 1], dtype='float32') label = fluid.data(name='label', shape=[3, 224, 224, 1], dtype='float32') predictions = fluid.layers.sigmoid(x) loss = fluid.layers.dice_loss(input=predictions, label=label)
dropout¶

paddle.fluid.layers.
dropout
(x, dropout_prob, is_test=False, seed=None, name=None, dropout_implementation='downgrade_in_infer')[source] Computes dropout.
Drop or keep each element of x independently. Dropout is a regularization technique for reducing overfitting by preventing neuron coadaption during training. The dropout operator randomly sets (according to the given dropout probability) the outputs of some units to zero, while others are remain unchanged.
dropout op can be removed from the program to make the program more efficient.
 Parameters
x (Variable) – The input tensor variable. The data type is float16 or float32 or float64.
dropout_prob (float) – Probability of setting units to zero.
is_test (bool) – A flag indicating whether it is in test phrase or not.
seed (int) – A Python integer used to create random seeds. If this parameter is set to None, a random seed is used. NOTE: If an integer seed is given, always the same output units will be dropped. DO NOT use a fixed seed in training.Default: None.
name (strNone) – A name for this layer(optional). If set None, the layer will be named automatically.
dropout_implementation (string) –
[‘downgrade_in_infer’(default)’upscale_in_train’]
downgrade_in_infer(default), downgrade the outcome at inference
train: out = input * mask
inference: out = input * (1.0  dropout_prob)
(mask is a tensor same shape with input, value is 0 or 1 ratio of 0 is dropout_prob)
upscale_in_train, upscale the outcome at training time
train: out = input * mask / ( 1.0  dropout_prob )
inference: out = input
(mask is a tensor same shape with input, value is 0 or 1 ratio of 0 is dropout_prob)
 Returns
A Variable holding Tensor representing the dropout, has same shape and data type with x.
Examples
import paddle.fluid as fluid x = fluid.data(name="data", shape=[None, 32, 32], dtype="float32") droped = fluid.layers.dropout(x, dropout_prob=0.5)
dynamic_gru¶

paddle.fluid.layers.
dynamic_gru
(input, size, param_attr=None, bias_attr=None, is_reverse=False, gate_activation='sigmoid', candidate_activation='tanh', h_0=None, origin_mode=False)[source] Note: The input type of this must be LoDTensor. If the input type to be processed is Tensor, use StaticRNN .
This operator is used to perform the calculations for a single layer of Gated Recurrent Unit (GRU) on full sequences step by step. The calculations in one time step support these two modes:
If
origin_mode
is True, then the formula used is from paper Learning Phrase Representations using RNN Encoder Decoder for Statistical Machine Translation .\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t1}) + b_c)\\h_t & = u_t \odot h_{t1} + (1u_t) \odot \tilde{h_t}\end{aligned}\end{align} \]if
origin_mode
is False, then the formula used is from paper Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t1}) + b_c)\\h_t & = (1u_t) \odot h_{t1} + u_t \odot \tilde{h_t}\end{aligned}\end{align} \]\(x_t\) is the input of current time step, but it is not from
input
. This operator does not include the calculations \(W_{ux}x_{t}, W_{rx}x_{t}, W_{cx}x_{t}\) , Note thus a fullyconnect layer whose size is 3 times ofsize
should be used before this operator, and the output should be used asinput
here. \(h_{t1}\) is the hidden state from previous time step. \(u_t\) , \(r_t\) , \(\tilde{h_t}\) and \(h_t\) stand for update gate, reset gate, candidate hidden and hidden output separately. \(W_{uh}, b_u\) , \(W_{rh}, b_r\) and \(W_{ch}, b_c\) stand for the weight matrix and bias used in update gate, reset gate, candidate hidden calculations. For implementation, the three weight matrix are merged into a tensor shaped \([D, D \times 3]\) , the three bias are concatenated as a tensor shaped \([1, D \times 3]\) , where \(D\) stands for the hidden size; The data layout of weight tensor is: \(W_{uh}\) and \(W_{rh}\) are concatenated with shape \([D, D \times 2]\) lying on the first part, and \(W_{ch}\) lying on the latter part with shape \([D, D]\) . Parameters
input (Variable) – A LoDTensor whose lod level is 1, representing the input after linear projection. Its shape should be \([T, D \times 3]\) , where \(T\) stands for the total sequence lengths in this minibatch, \(D\) for the hidden size. The data type should be float32 or float64.
size (int) – Indicate the hidden size.
param_attr (ParamAttr, optional) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
bias_attr (ParamAttr, optional) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .
is_reverse (bool, optional) – Whether to compute in the reversed order of input sequences. Default False.
gate_activation (str, optional) – The activation fuction corresponding to \(act_g\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “sigmoid”.
candidate_activation (str, optional) – The activation fuction corresponding to \(act_c\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “tanh”.
h_0 (Variable, optional) – A Tensor representing the initial hidden state. It not provided, the default initial hidden state is 0. The shape is \([N, D]\) , where \(N\) is the number of sequences in the minibatch, \(D\) for the hidden size. The data type should be same as
input
. Default None.
 Returns
A LoDTensor whose lod level is 1 and shape is \([T, D]\) , where \(T\) stands for the total sequence lengths in this minibatch \(D\) for the hidden size. It represents GRU transformed sequence output, and has the same lod and data type with
input
. Return type
Variable
Examples
import paddle.fluid as fluid dict_dim, emb_dim = 128, 64 data = fluid.data(name='sequence', shape=[None], dtype='int64', lod_level=1) emb = fluid.embedding(input=data, size=[dict_dim, emb_dim]) hidden_dim = 512 x = fluid.layers.fc(input=emb, size=hidden_dim * 3) hidden = fluid.layers.dynamic_gru(input=x, size=hidden_dim)
dynamic_lstm¶

paddle.fluid.layers.
dynamic_lstm
(input, size, h_0=None, c_0=None, param_attr=None, bias_attr=None, use_peepholes=True, is_reverse=False, gate_activation='sigmoid', cell_activation='tanh', candidate_activation='tanh', dtype='float32', name=None)[source]  Note:
This OP only supports LoDTensor as inputs. If you need to deal with Tensor, please use lstm .
In order to improve efficiency, users must first map the input of dimension [T, hidden_size] to input of [T, 4 * hidden_size], and then pass it to this OP.
The implementation of this OP include diagonal/peephole connections. Please refer to Gers, F. A., & Schmidhuber, J. (2000) . If you do not need peephole connections, please set use_peepholes to False .
This OP computes each timestep as follows:
\[i_t = \sigma(W_{ix}x_{t} + W_{ih}h_{t1} + b_{x_i} + b_{h_i})\]\[f_t = \sigma(W_{fx}x_{t} + W_{fh}h_{t1} + b_{x_f} + b_{h_f})\]\[o_t = \sigma(W_{ox}x_{t} + W_{oh}h_{t1} + b_{x_o} + b_{h_o})\]\[\widetilde{c_t} = tanh(W_{cx}x_t + W_{ch}h_{t1} + b{x_c} + b_{h_c})\]\[c_t = f_t \odot c_{t1} + i_t \odot \widetilde{c_t}\]\[h_t = o_t \odot tanh(c_t)\]The symbolic meanings in the formula are as follows:
\(x_{t}\) represents the input at timestep \(t\)
\(h_{t}\) represents the hidden state at timestep \(t\)
\(h_{t1}, c_{t1}\) represent the hidden state and cell state at timestep \(t1\) , respectively
\(\widetilde{c_t}\) represents the candidate cell state
\(i_t\) , \(f_t\) and \(o_t\) represent input gate, forget gate, output gate, respectively
\(W\) represents weight (e.g., \(W_{ix}\) is the weight of a linear transformation of input \(x_{t}\) when calculating input gate \(i_t\) )
\(b\) represents bias (e.g., \(b_{i}\) is the bias of input gate)
\(\sigma\) represents nonlinear activation function for gate, default sigmoid
\(\odot\) represents the Hadamard product of a matrix, i.e. multiplying the elements of the same position for two matrices with the same dimension to get another matrix with the same dimension
 Parameters
input (api_guide_Variable_en) – LSTM input tensor, multidimensional LODTensor of shape \([T, 4*hidden\_size]\) . Data type is float32 or float64.
size (int) – must be 4 * hidden_size.
h_0 (api_guide_Variable_en , optional) – The initial hidden state of the LSTM, multidimensional Tensor of shape \([batch\_size, hidden\_size]\) . Data type is float32 or float64. If set to None, it will be a vector of all 0. Default: None.
c_0 (api_guide_Variable_en , optional) – The initial hidden state of the LSTM, multidimensional Tensor of shape \([batch\_size, hidden\_size]\) . Data type is float32 or float64. If set to None, it will be a vector of all 0. h_0 and c_0 can be None but only at the same time. Default: None.
param_attr (ParamAttr, optional) –
Parameter attribute of weight. If it is None, the default weight parameter attribute is used. Please refer to ref:api_fluid_ParamAttr’ . If the user needs to set this parameter, the dimension must be :math:`[hidden_size, 4*hidden_size] . Default: None.
Weights = \(\{ W_{cr},W_{ir},W_{fr},W_{or} \}\) , the shape is [hidden_size, 4*hidden_size].
bias_attr (ParamAttr, optional) –
The bias attribute for the learnable bias weights, which contains two parts, inputhidden bias weights and peephole connections weights if setting use_peepholes to True. Please refer to ref:`api_fluid_ParamAttr’ . Default: None.
use_peepholes = False  Biases = {\(b_c, b_i, b_f, b_o\)}.  The shape is [1, 4*hidden_size].
use_peepholes = True  Biases = { \(b_c, b_i, b_f, b_o, W_{ic}, W_{fc}, W_{oc}\)}.  The shape is [1, 7*hidden_size].
use_peepholes (bool, optional) – Whether to use peephole connection or not. Default: True.
is_reverse (bool, optional) – Whether to calculate reverse LSTM. Default: False.
gate_activation (str, optional) – The activation for input gate, forget gate and output gate. Default: “sigmoid”.
cell_activation (str, optional) – The activation for cell output. Default: “tanh”.
candidate_activation (str, optional) – The activation for candidate hidden state. Default: “tanh”.
dtype (str, optional) – Data type, can be “float32” or “float64”. Default: “float32”.
name (str, optional) – A name for this layer. Please refer to api_guide_Name . Default: None.
 Returns
The hidden state and cell state of LSTM
hidden: LoDTensor with shape of \([T, hidden\_size]\) , and its lod and dtype is the same as the input.
cell: LoDTensor with shape of \([T, hidden\_size]\) , and its lod and dtype is the same as the input.
 Return type
tuple ( api_guide_Variable , api_guide_Variable )
Examples
import paddle.fluid as fluid emb_dim = 256 vocab_size = 10000 hidden_dim = 512 data = fluid.data(name='x', shape=[None], dtype='int64', lod_level=1) emb = fluid.embedding(input=data, size=[vocab_size, emb_dim], is_sparse=True) forward_proj = fluid.layers.fc(input=emb, size=hidden_dim * 4, bias_attr=False) forward, cell = fluid.layers.dynamic_lstm( input=forward_proj, size=hidden_dim * 4, use_peepholes=False) forward.shape # (1, 512) cell.shape # (1, 512)
dynamic_lstmp¶

paddle.fluid.layers.
dynamic_lstmp
(input, size, proj_size, param_attr=None, bias_attr=None, use_peepholes=True, is_reverse=False, gate_activation='sigmoid', cell_activation='tanh', candidate_activation='tanh', proj_activation='tanh', dtype='float32', name=None, h_0=None, c_0=None, cell_clip=None, proj_clip=None)[source]  Note:
In order to improve efficiency, users must first map the input of dimension [T, hidden_size] to input of [T, 4 * hidden_size], and then pass it to this OP.
This OP implements the LSTMP (LSTM Projected) layer. The LSTMP layer has a separate linear mapping layer behind the LSTM layer. – Sak, H., Senior, A., & Beaufays, F. (2014) .
Compared with the standard LSTM layer, LSTMP has an additional linear mapping layer, which is used to map from the original hidden state \(h_t\) to the lower dimensional state \(r_t\) . This reduces the total number of parameters and computational complexity, especially when the output unit is relatively large.
The default implementation of the OP contains diagonal/peephole connections, please refer to Gers, F. A., & Schmidhuber, J. (2000) . If you need to disable the peephole connections, set use_peepholes to False.
This OP computes each timestep as follows:
\[i_t = \sigma(W_{ix}x_{t} + W_{ir}r_{t1} + W_{ic}c_{t1} + b_i)\]\[f_t = \sigma(W_{fx}x_{t} + W_{fr}r_{t1} + W_{fc}c_{t1} + b_f)\]\[o_t = \sigma(W_{ox}x_{t} + W_{or}r_{t1} + W_{oc}c_{t1} + b_o)\]\[\widetilde{c_t} = act_g(W_{cx}x_t + W_{cr}r_{t1} + b_c)\]\[c_t = f_t \odot c_{t1} + i_t \odot \widetilde{c_t}\]\[h_t = o_t \odot act_h(c_t)\]\[r_t = \overline{act_h}(W_{rh}h_t)\]The symbolic meanings in the formula are as follows:
\(x_{t}\) represents the input at timestep \(t\)
\(h_{t}\) represents the hidden state at timestep \(t\)
\(r_{t}\) : represents the state of the projected output of the hidden state \(h_{t}\)
\(h_{t1}, c_{t1}, r_{t1}\) represent the hidden state, cell state and projected output at timestep \(t1\) , respectively
\(\widetilde{c_t}\) represents the candidate cell state
\(i_t\) , \(f_t\) and \(o_t\) represent input gate, forget gate, output gate, respectively
\(W\) represents weight (e.g., \(W_{ix}\) is the weight of a linear transformation of input \(x_{t}\) when calculating input gate \(i_t\) )
\(b\) represents bias (e.g., \(b_{i}\) is the bias of input gate)
\(\sigma\) represents nonlinear activation function for gate, default sigmoid
\(\odot\) represents the Hadamard product of a matrix, i.e. multiplying the elements of the same position for two matrices with the same dimension to get another matrix with the same dimension
 Parameters
input (api_guide_Variable_en) – The input of dynamic_lstmp layer, which supports variabletime length input sequence. It is a multidimensional LODTensor of shape \([T, 4*hidden\_size]\) . Data type is float32 or float64.
size (int) – must be 4 * hidden_size.
proj_size (int) – The size of projection output.
param_attr (ParamAttr, optional) –
Parameter attribute of weight. If it is None, the default weight parameter attribute is used. Please refer to ref:api_fluid_ParamAttr’ . If the user needs to set this parameter, the dimension must be :math:`[hidden_size, 4*hidden_size] . Default: None.
Weights = \(\{ W_{cr},W_{ir},W_{fr},W_{or} \}\) , the shape is [P, 4*hidden_size] , where P is the projection size.
Projection weight = \(\{ W_{rh} \}\) , the shape is [hidden_size, P].
bias_attr (ParamAttr, optional) –
The bias attribute for the learnable bias weights, which contains two parts, inputhidden bias weights and peephole connections weights if setting use_peepholes to True. Please refer to ref:`api_fluid_ParamAttr’ . Default: None.
use_peepholes = False  Biases = {\(b_c, b_i, b_f, b_o\)}.  The shape is [1, 4*hidden_size].
use_peepholes = True  Biases = { \(b_c, b_i, b_f, b_o, W_{ic}, W_{fc}, W_{oc}\)}.  The shape is [1, 7*hidden_size].
use_peepholes (bool, optional) – Whether to use peephole connection or not. Default True.
is_reverse (bool, optional) – Whether to calculate reverse LSTM. Default False.
gate_activation (str, optional) – The activation for input gate, forget gate and output gate. Default “sigmoid”.
cell_activation (str, optional) – The activation for cell output. Default “tanh”.
candidate_activation (str, optional) – The activation for candidate hidden state. Default “tanh”.
proj_activation (str, optional) – The activation for projection output. Default “tanh”.
dtype (str, optional) – Data type, can be “float32” or “float64”. Default “float32”.
name (str, optional) – A name for this layer. Please refer to api_guide_Name . Default: None.
h_0 (api_guide_Variable , optional) – The initial hidden state is an optional input, default is zero. This is a tensor with shape \([batch\_size, P]\) , where P is the projection size. Default: None.
c_0 (api_guide_Variable , optional) – The initial cell state is an optional input, default is zero. This is a tensor with shape \([batch\_size, P]\) , where P is the projection size. h_0 and c_0 can be None but only at the same time. Default: None.
cell_clip (float, optional) – If not None, the cell state is clipped by this value prior to the cell output activation. Default: None.
proj_clip (float, optional) – If num_proj > 0 and proj_clip is provided, then the projected values are clipped elementwise to within [proj_clip, proj_clip]. Default: None.
 Returns
The hidden state and cell state of LSTMP
hidden: LoDTensor with shape of \([T, P]\) , and its lod and dtype is the same as the input.
cell: LoDTensor with shape of \([T, hidden\_size]\) , and its lod and dtype is the same as the input.
 Return type
tuple ( api_guide_Variable , api_guide_Variable )
Examples
import paddle.fluid as fluid dict_dim, emb_dim = 128, 64 data = fluid.data(name='sequence', shape=[None], dtype='int64', lod_level=1) emb = fluid.embedding(input=data, size=[dict_dim, emb_dim]) hidden_dim, proj_dim = 512, 256 fc_out = fluid.layers.fc(input=emb, size=hidden_dim * 4, act=None, bias_attr=None) proj_out, last_c = fluid.layers.dynamic_lstmp(input=fc_out, size=hidden_dim * 4, proj_size=proj_dim, use_peepholes=False, is_reverse=True, cell_activation="tanh", proj_activation="tanh") proj_out.shape # (1, 256) last_c.shape # (1, 512)
edit_distance¶

paddle.fluid.layers.
edit_distance
(input, label, normalized=True, ignored_tokens=None, input_length=None, label_length=None)[source] This op computes the edit distances between a batch of hypothesis strings and their references. Edit distance, also called Levenshtein distance, measures how dissimilar two strings are by counting the minimum number of operations to transform one string into anthor. Here the operations include insertion, deletion, and substitution.
For example, given hypothesis string A = “kitten” and reference B = “sitting”, the edit distance is 3 for A will be transformed into B at least after two substitutions and one insertion:
“kitten” > “sitten” > “sittin” > “sitting”
The input is a LoDTensor/Tensor consisting of all the hypothesis strings with the total number denoted by batch_size, and the separation is specified by the LoD information or input_length. And the batch_size reference strings are arranged in order in the same way as input.
The output contains the batch_size results and each stands for the edit distance for a pair of strings respectively. If Attr(normalized) is true, the edit distance will be divided by the length of reference string.
 Parameters
input (Variable) – The indices for hypothesis strings, its rank should equals to 2 and its data type should be int64.
label (Variable) – The indices for reference strings, its rank should equals to 2 and its data type should be int64.
normalized (bool, default True) – Indicated whether to normalize the edit distance by the length of reference string.
ignored_tokens (list<int>, default None) – Tokens that should be removed before calculating edit distance.
input_length (Variable) – The length for each sequence in input if it’s of Tensor type, it should have shape [batch_size] and dtype int64.
label_length (Variable) – The length for each sequence in label if it’s of Tensor type, it should have shape [batch_size] and dtype int64.
 Returns
edit_distance_out(Variable): edit distance result in shape [batch_size, 1]. sequence_num(Variable): sequence number in shape [].
 Return type
Tuple
Examples
import paddle.fluid as fluid # using LoDTensor x_lod = fluid.data(name='x_lod', shape=[None,1], dtype='int64', lod_level=1) y_lod = fluid.data(name='y_lod', shape=[None,1], dtype='int64', lod_level=1) distance_lod, seq_num_lod = fluid.layers.edit_distance(input=x_lod, label=y_lod) # using Tensor x_seq_len = 5 y_seq_len = 6 x_pad = fluid.data(name='x_pad', shape=[None,x_seq_len], dtype='int64') y_pad = fluid.data(name='y_pad', shape=[None,y_seq_len], dtype='int64') x_len = fluid.data(name='x_len', shape=[None], dtype='int64') y_len = fluid.data(name='y_len', shape=[None], dtype='int64') distance_pad, seq_num_pad = fluid.layers.edit_distance(input=x_pad, label=y_pad, input_length=x_len, label_length=y_len)
elementwise_add¶

paddle.fluid.layers.
elementwise_add
(x, y, axis=1, act=None, name=None)[source] Elementwise Add Operator.
Add two tensors elementwise
The equation is:
$$Out = X + Y$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([2, 3, 4]).astype('float32'), "y": np.array([1, 5, 2]).astype('float32') } x = fluid.data(name="x", shape=[3], dtype='float32') y = fluid.data(name="y", shape=[3], dtype='float32') z = fluid.layers.elementwise_add(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[3., 8., 6.]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.ones((2, 3, 4, 5)).astype('float32'), "y": np.zeros((3, 4)).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[3,4], dtype='float32') z = fluid.layers.elementwise_add(x, y, axis=1) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'), "y": np.random.randint(1, 5, size=[5]).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[5], dtype='float32') z = fluid.layers.elementwise_add(x, y, axis=3) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
elementwise_div¶

paddle.fluid.layers.
elementwise_div
(x, y, axis=1, act=None, name=None)[source] Elementwise Div Operator.
Divide two tensors elementwise
The equation is:
$$Out = X / Y$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([2, 3, 4]).astype('float32'), "y": np.array([1, 5, 2]).astype('float32') } x = fluid.data(name="x", shape=[3], dtype='float32') y = fluid.data(name="y", shape=[3], dtype='float32') z = fluid.layers.elementwise_div(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[2., 0.6, 2.]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.ones((2, 3, 4, 5)).astype('float32'), "y": np.zeros((3, 4)).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[3,4], dtype='float32') z = fluid.layers.elementwise_div(x, y, axis=1) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'), "y": np.random.randint(1, 5, size=[5]).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[5], dtype='float32') z = fluid.layers.elementwise_div(x, y, axis=3) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
elementwise_floordiv¶

paddle.fluid.layers.
elementwise_floordiv
(x, y, axis=1, act=None, name=None)[source] Elementwise FloorDiv Operator.
Floor divide two tensors elementwise
The equation is:
$$Out = X // Y$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.
y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([10, 15, 8]).astype('int32'), "y": np.array([3, 7, 5]).astype('int32') } x = fluid.data(name="x", shape=[3], dtype='int32') y = fluid.data(name="y", shape=[3], dtype='int32') z = fluid.layers.elementwise_floordiv(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[3, 2, 1]
elementwise_max¶

paddle.fluid.layers.
elementwise_max
(x, y, axis=1, act=None, name=None)[source] Elementwise Max Operator.
Compare two tensors and returns a new tensor containing the elementwise maxima.
The equation is:
$$Out = max(X, Y)$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), The first tensor holding the elements to be compared.
y – (Variable), The second tensor holding the elements to be compared.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([2, 3, 4]).astype('float32'), "y": np.array([1, 5, 2]).astype('float32') } x = fluid.data(name="x", shape=[3], dtype='float32') y = fluid.data(name="y", shape=[3], dtype='float32') z = fluid.layers.elementwise_max(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[2, 5, 4]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.ones((2, 3, 4, 5)).astype('float32'), "y": np.zeros((3, 4)).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[3,4], dtype='float32') z = fluid.layers.elementwise_max(x, y, axis=1) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value)#[[[[1., 1., 1., 1., 1.] .... [1., 1., 1., 1., 1.]]]]
elementwise_min¶

paddle.fluid.layers.
elementwise_min
(x, y, axis=1, act=None, name=None)[source] Elementwise Min Operator.
Compare two tensors and returns a new tensor containing the elementwise minima.
The equation is:
$$Out = min(X, Y)$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), The first tensor holding the elements to be compared.
y – (Variable), The second tensor holding the elements to be compared.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([2, 3, 4]).astype('float32'), "y": np.array([1, 5, 2]).astype('float32') } x = fluid.data(name="x", shape=[3], dtype='float32') y = fluid.data(name="y", shape=[3], dtype='float32') z = fluid.layers.elementwise_max(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[1, 3, 2]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.ones((2, 3, 4, 5)).astype('float32'), "y": np.zeros((3, 4)).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[3,4], dtype='float32') z = fluid.layers.elementwise_max(x, y, axis=1) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value)#[[[[0., 0., 0., 0., 0.] .... [0., 0., 0., 0., 0.]]]]
elementwise_mod¶

paddle.fluid.layers.
elementwise_mod
(x, y, axis=1, act=None, name=None)[source] Elementwise Mod Operator.
Mod two tensors elementwise
The equation is:
$$Out = X \% Y$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.
y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([10, 15, 8]).astype('int32'), "y": np.array([3, 6, 5]).astype('int32') } x = fluid.data(name="x", shape=[3], dtype='int32') y = fluid.data(name="y", shape=[3], dtype='int32') z = fluid.layers.elementwise_mod(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[1, 3, 3]
elementwise_mul¶

paddle.fluid.layers.
elementwise_mul
(x, y, axis=1, act=None, name=None)[source] Elementwise Mul Operator.
Multiply two tensors elementwise
The equation is:
$$Out = X \odot Y$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([2, 3, 4]).astype('float32'), "y": np.array([1, 5, 2]).astype('float32') } x = fluid.data(name="x", shape=[3], dtype='float32') y = fluid.data(name="y", shape=[3], dtype='float32') z = fluid.layers.elementwise_mul(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[2., 15., 8.]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.ones((2, 3, 4, 5)).astype('float32'), "y": np.zeros((3, 4)).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[3,4], dtype='float32') z = fluid.layers.elementwise_mul(x, y, axis=1) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'), "y": np.random.randint(1, 5, size=[5]).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[5], dtype='float32') z = fluid.layers.elementwise_mul(x, y, axis=3) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
elementwise_pow¶

paddle.fluid.layers.
elementwise_pow
(x, y, axis=1, act=None, name=None)[source] Elementwise Pow Operator.
First tensor elements raised to powers from the second tensor, elementwise.
The equation is:
$$Out = X ^ Y$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), The Base.
y – (Variable), The exponents.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([2, 3, 4]).astype('float32'), "y": np.array([1, 5, 2]).astype('float32') } x = fluid.data(name="x", shape=[3], dtype='float32') y = fluid.data(name="y", shape=[3], dtype='float32') z = fluid.layers.elementwise_pow(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[2, 243, 16]
elementwise_sub¶

paddle.fluid.layers.
elementwise_sub
(x, y, axis=1, act=None, name=None)[source] Elementwise Sub Operator.
Substract two tensors elementwise
The equation is:
$$Out = X  Y$$
\(X\): a tensor of any dimension.
\(Y\): a tensor whose dimensions must be less than or equal to the dimensions of \(X\).
There are two cases for this operator:
The shape of \(Y\) is the same with \(X\).
The shape of \(Y\) is a continuous subsequence of \(X\).
For case 2:
Broadcast \(Y\) to match the shape of \(X\), where \(axis\) is the start dimension index for broadcasting \(Y\) onto \(X\).
If \(axis\) is 1 (default), \(axis = rank(X)  rank(Y)\).
The trailing dimensions of size 1 for \(Y\) will be ignored for the consideration of subsequence, such as shape(Y) = (2, 1) => (2).
For example:
shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5), with axis=1(default) or axis=2 shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0 shape(X) = (2, 3, 4, 5), shape(Y) = (2, 1), with axis=0
 Parameters
x – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
y – (Variable), Tensor or LoDTensor of any dimensions. Its dtype should be int32, int64, float32, float64.
axis (int32, optional) – If X.dimension != Y.dimension, Y.dimension must be a subsequence of x.dimension. And axis is the start dimension index for broadcasting Y onto X.
act (string, optional) – Activation applied to the output. Default is None. Details: Activation Function
name (string, optional) – Name of the output. Default is None. It’s used to print debug info for developers. Details: api_guide_Name
 Returns
Ndimension tensor. A location into which the result is stored. It’s dimension equals with x
Examples
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.array([2, 3, 4]).astype('float32'), "y": np.array([1, 5, 2]).astype('float32') } x = fluid.data(name="x", shape=[3], dtype='float32') y = fluid.data(name="y", shape=[3], dtype='float32') z = fluid.layers.elementwise_sub(x, y) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) #[1., 2., 2.]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.ones((2, 3, 4, 5)).astype('float32'), "y": np.zeros((3, 4)).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[3,4], dtype='float32') z = fluid.layers.elementwise_sub(x, y, axis=1) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
import paddle.fluid as fluid import numpy as np def gen_data(): return { "x": np.random.randint(1, 5, size=[2, 3, 4, 5]).astype('float32'), "y": np.random.randint(1, 5, size=[5]).astype('float32') } x = fluid.data(name="x", shape=[2,3,4,5], dtype='float32') y = fluid.data(name="y", shape=[5], dtype='float32') z = fluid.layers.elementwise_sub(x, y, axis=3) place = fluid.CPUPlace() exe = fluid.Executor(place) z_value = exe.run(feed=gen_data(), fetch_list=[z.name]) print(z_value) # z.shape=[2,3,4,5]
elu¶

paddle.fluid.layers.
elu
(x, alpha=1.0, name=None)[source] ELU Activation Operator.
Applies the following elementwise computation on the input according to https://arxiv.org/abs/1511.07289.
\(out = \max(0, x) + \min(0, \alpha * (e^x  1))\)
 Parameters
x (Variable) – The input is a multidimensional Tensor. The data type is float32 or float64
alpha (FLOAT1.0) – The alpha value of ELU
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.
 Returns
The output is a multidimensional Tensor which has same dimension and data type as the
x
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np input_elu = np.array([[1,6],[1,15.6]]) with fluid.dygraph.guard(): x = fluid.dygraph.to_variable(input_elu) y = fluid.layers.elu(x, alpha=0.2) print(y.numpy()) # [[0.12642411 6. ] # [ 1. 15.6 ]]
embedding¶

paddle.fluid.layers.
embedding
(input, size, is_sparse=False, is_distributed=False, padding_idx=None, param_attr=None, dtype='float32')[source] WARING: This OP will be deprecated in a future release. This OP requires the last dimension of Tensor shape must be equal to 1. It is recommended to use fluid. api_fluid_embedding .
The operator is used to lookup embeddings vector of ids provided by
input
. It automatically constructs a 2D embedding matrix based on the inputsize
(vocab_size, emb_size) anddtype
.This OP requires the last dimension of Tensor shape must be equal to 1. The shape of output Tensor is generated by replacing the last dimension of the input Tensor shape with emb_size.
Note: The id in
input
must satisfy \(0 =< id < size[0]\) , otherwise the program will throw an exception and exit.Case 1: input is a Tensor. padding_idx = 1 input.data = [[[1], [3]], [[2], [4]], [[4], [127]]] input.shape = [3, 2, 1] Given size = [128, 16] output is a Tensor: out.shape = [3, 2, 16] out.data = [[[0.129435295, 0.244512452, ..., 0.436322452], [0.345421456, 0.524563927, ..., 0.144534654]], [[0.345249859, 0.124939536, ..., 0.194353745], [0.945345345, 0.435394634, ..., 0.435345365]], [[0.945345345, 0.435394634, ..., 0.435345365], [0.0, 0.0, ..., 0.0 ]]] # padding data The input padding_idx is less than 0, it is automatically converted to padding_idx = 1 + 128 = 127 It will pad allzero data when ids is 127. Case 2: input is a LoDTensor with 1level LoD. padding_idx = 0 input.lod = [[2, 3]] input.data = [[1], [3], [2], [4], [0]] input.shape = [5, 1] Given size = [128, 16] output is a LoDTensor: out.lod = [[2, 3]] out.shape = [5, 16] out.data = [[0.129435295, 0.244512452, ..., 0.436322452], [0.345421456, 0.524563927, ..., 0.144534654], [0.345249859, 0.124939536, ..., 0.194353745], [0.945345345, 0.435394634, ..., 0.435345365], [0.0, 0.0, ..., 0.0 ]] # padding data It will pad allzero data when ids is 0.
 Parameters
input (Variable) – A Tensor or LoDTensor with type int64, which contains the id information. The last dimension of Tensor shape must be equal to 1. The value of the input id should satisfy \(0<= id < size[0]\) .
size (tuplelist) – The shape of lookup table parameter. It should have two elements which indicates the size of the dictionary of embeddings and the size of each embedding vector respectively.
is_sparse (bool) – The flag indicating whether to use sparse update. This parameter only affects the performance of the backwards gradient update. It is recommended to set True because sparse update is faster. But some optimizer does not support sparse update, such as api_fluid_optimizer_AdadeltaOptimizer , AdamaxOptimizer , DecayedAdagradOptimizer , FtrlOptimizer , LambOptimizer and LarsMomentumOptimizer . In these case, is_sparse must be False. Default: False.
is_distributed (bool) – Whether to store the embedding matrix in a distributed manner. Only used in multimachine distributed CPU training. Default: False.
padding_idx (intlongNone) – padding_idx needs to be in the interval [vocab_size, vocab_size). If \(padding\_idx < 0\), the \(padding\_idx\) will automatically be converted to \(vocab\_size + padding\_idx\) . It will output allzero padding data whenever lookup encounters \(padding\_idx\) in id. And the padding data will not be updated while training. If set None, it makes no effect to output. Default: None.
param_attr (ParamAttr) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr . In addition, userdefined or pretrained word vectors can be loaded with the
param_attr
parameter. The local word vector needs to be transformed into numpy format, and the shape of local word vector shoud be consistent withsize
. Then NumpyArrayInitializer is used to load custom or pretrained word vectors. See code example 2 for details.dtype (strcore.VarDesc.VarType) – It refers to the data type of output Tensor. It must be float32 or float64. Default: float32.
 Returns
Embedding Tensor or LoDTensor mapped by input. The data type is the same as
dtype
. Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np data = fluid.data(name='x', shape=[None, 1], dtype='int64') # exampel 1 emb_1 = fluid.embedding(input=data, size=[128, 64]) # example 2: load custom or pretrained word vectors weight_data = np.random.random(size=(128, 100)) # word vectors with numpy format w_param_attrs = fluid.ParamAttr( name="emb_weight", learning_rate=0.5, initializer=fluid.initializer.NumpyArrayInitializer(weight_data), trainable=True) emb_2 = fluid.layers.embedding(input=data, size=(128, 100), param_attr=w_param_attrs, dtype='float32')
expand¶

paddle.fluid.layers.
expand
(x, expand_times, name=None)[source] This operation tiles
x
multiple times according to the parameterexpand_times
. The times number for each dimension ofx
is set by the parameterexpand_times
. The rank ofx
should be less than or equal to 6. Please note that size ofexpand_times
must be the same with X’s rank. Following is a using case:Input(X) is a 3D tensor with shape [2, 3, 1]: [ [[1], [2], [3]], [[4], [5], [6]] ] Attr(expand_times): [1, 2, 2] Output(Out) is a 3D tensor with shape [2, 6, 2]: [ [[1, 1], [2, 2], [3, 3], [1, 1], [2, 2], [3, 3]], [[4, 4], [5, 5], [6, 6], [4, 4], [5, 5], [6, 6]] ]
 Parameters
x (Variable) – A
Tensor
orLoDTensor
with dimension in [1, 6]. The data type isbool
,float32
,float64
orint32
.expand_times (listtupleVariable) – The data type is
int32
. Ifexpand_times
is a list or tuple, the elements of it should be integers or Tensors with shape [1]. Ifexpand_times
is an Variable, it should be an 1D Tensor. Expand times number for each dimension ofx
.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
A
Tensor
orLoDTensor
. The data type is same asx
. After expanding, size of each dimension of output is equal to the size of the corresponding dimension ofx
multiplying the corresponding value given byexpand_times
. Return type
Variable
 Raises
TypeError
– The type ofexpand_times
must be list, tuple or Variable.ValueError
– The elements ofexpand_times
cannot be negative.
Examples
import paddle.fluid as fluid # example 1: data_1 = fluid.layers.fill_constant(shape=[2, 3, 1], dtype='int32', value=0) expanded_1 = fluid.layers.expand(data_1, expand_times=[1, 2, 2]) # the shape of expanded_1 is [2, 6, 2]. # example 2: data_2 = fluid.layers.fill_constant(shape=[12, 14], dtype="int32", value=3) expand_times = fluid.layers.fill_constant(shape=[2], dtype="int32", value=4) expanded_2 = fluid.layers.expand(data_2, expand_times=expand_times) # the shape of expanded_2 is [48, 56].
fc¶

paddle.fluid.layers.
fc
(input, size, num_flatten_dims=1, param_attr=None, bias_attr=None, act=None, name=None)[source] Fully Connected Layer
This operator creates a fully connected layer in the network. It can take a Tensor(or LoDTensor) or a list of Tensor(or LoDTensor) as its inputs(see Args in detail). It creates a variable called weight for each input Tensor, which represents a fully connected weight matrix from each input unit to each output unit. The fully connected layer multiplies each input Tensor with its corresponding weight to produce an output Tensor with shape \([M, size]\) , where M is batch size. If a list of Tensor is given, the results of multiple output Tensors with shape \([M, size]\) will be summed up. If
bias_attr
is not None, a bias variable will be created and added to the output. Finally, ifact
is not None, it will be applied to the output as well.When the input is a single Tensor(or LoDTensor):
\[Out = Act({XW + b})\]When the input is a list of Tensor(or LoDTensor):
\[Out = Act({\sum_{i=0}^{N1}X_iW_i + b})\]In the above equation:
\(N\): Number of the input. N equals to len(input) if input is list of Variable.
\(X_i\): The ith input tensor.
\(W_i\): The ith weights matrix corresponding ith input tensor.
\(b\): The bias parameter created by this layer (if needed).
\(Act\): The activation function.
\(Out\): The output Tensor.
Case 1: Given a single Tensor data_1, and num_flatten_dims = 2: data_1.data = [[[0.1, 0.2], [0.3, 0.4]]] data_1.shape = (1, 2, 2) # 1 is batch_size out = fluid.layers.fc(input=data_1, size=1, num_flatten_dims=2) Then output is: out.data = [[0.83234344], [0.34936576]] out.shape = (1, 2, 1) Case 2: Given a list of Tensor: data_1.data = [[[0.1, 0.2], [0.3, 0.4]]] data_1.shape = (1, 2, 2) # 1 is batch_size data_2 = [[[0.1, 0.2, 0.3]]] data_2.shape = (1, 1, 3) out = fluid.layers.fc(input=[data_1, data_2], size=2) Then: out.data = [[0.18669507, 0.1893476]] out.shape = (1, 2)
 Parameters
input (Variablelist of Variable) – A Tensor(or LoDTensor) with shape \([N_1, N_2,..., N_k]\) or a list of Tensor(or LoDTensor). The dimensions of the input Tensor is at least 2 and the data type should be float32 or float64.
size (int) – The number of output units in this layer, which also means the feature size of ouput Tensor(or LoDTensor).
num_flatten_dims (int) – The fc layer can accept an input Tensor with more than two dimensions. If this happens, the multidimensional tensor will first be flattened into a 2D matrix. The parameter
num_flatten_dims
determines how the input Tensor is flattened: the firstnum_flatten_dims
(inclusive, index starts from 1) dimensions will be flatten to form the first dimension of the final matrix (height of the matrix), and the rest \(rank(X)  num\_flatten\_dims\) dimensions are flattened to form the second dimension of the final matrix (width of the matrix). For example, assuming that X is a 5dimensional Tensor with a shape [2, 3, 4, 5, 6], andnum_flatten_dims
= 3. Then, the flattened matrix will have a shape [2 x 3 x 4, 5 x 6] = [24, 30]. Default: 1.param_attr (ParamAttr) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
bias_attr (ParamAttr) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .
act (str) – Activation to be applied to the output of this layer, such as tanh, softmax, sigmoid, relu. For more information, please refer to Activation Function . Default: None.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
Tensor or LoDTensor calculated by fc layer. The data type is same with input.
 Return type
Variable
 Raises
ValueError
– If dimensions of the input Tensor is less than 2.
Examples
import paddle.fluid as fluid # when input is single tensor data = fluid.data(name="data", shape=[1, 32], dtype="float32") fc = fluid.layers.fc(input=data, size=1000, act="tanh") # when input are multiple tensors data_1 = fluid.data(name="data_1", shape=[1, 32], dtype="float32") data_2 = fluid.data(name="data_2", shape=[1, 36], dtype="float32") fc = fluid.layers.fc(input=[data_1, data_2], size=1000, act="tanh")
flatten¶

paddle.fluid.layers.
flatten
(x, axis=1, name=None)[source] Flatten op
Flatten the input tensor into a 2D matrix.
For Example:
Case 1: Given X.shape = (3, 100, 100, 4) and axis = 2 We get: Out.shape = (3 * 100, 4 * 100) Case 2: Given X.shape = (3, 100, 100, 4) and axis = 0 We get: Out.shape = (1, 3 * 100 * 100 * 4)
 Parameters
x (Variable) – A tensor of rank >= axis. A tensor with type float32, float64, int8, int32, int64.
axis (int) – Indicate up to which input dimensions (exclusive) should be flattened to the outer dimension of the output. The value for axis must be in the range [0, R], where R is the rank of the input tensor. Default: 1.
name (str, Optional) – For details, please refer to api_guide_Name. Generally, no setting is required. Default: None.
 Returns
A 2D tensor with the contents of the input tensor, with input dimensions up to axis flattened to the outer dimension of the output and remaining input dimensions flattened into the inner dimension of the output. A Tensor with type same as input x.
 Return type
Variable
 Raises
ValueError
– If x is not a variable.ValueError
– If axis is not in range [0, rank(x)].
Examples
import paddle.fluid as fluid x = fluid.data(name="x", shape=[4, 4, 3], dtype="float32") # x shape is [4, 4, 3] out = fluid.layers.flatten(x=x, axis=2) # out shape is [16, 3]
fsp_matrix¶

paddle.fluid.layers.
fsp_matrix
(x, y)[source] FSP matrix op
This op is used to calculate the flow of solution procedure (FSP) matrix of two 4D Tensor feature maps. Given feature map x with shape [x_channel, h, w] and feature map y with shape [y_channel, h, w], we can get the fsp matrix of x and y in two steps:
reshape x into matrix with shape [x_channel, h * w] and reshape and transpose y into matrix with shape [h * w, y_channel].
multiply x and y to get fsp matrix with shape [x_channel, y_channel].
The output is a batch of fsp matrices.
 Parameters
x (Variable) – A 4D Tensor feature map with shape [batch_size, x_channel, height, width]. A Tensor with type float32, float64.
y (Variable) – A 4D Tensor feature map with shape [batch_size, y_channel, height, width]. The y_channel can be different with the x_channel of Input(X) while the other dimensions must be the same with Input(X)’s. A Tensor with type float32, float64.
 Returns
The output of FSP op with shape [batch_size, x_channel, y_channel]. The x_channel is the channel of x and the y_channel is the channel of y. A Tensor with type float32, float64.
 Return type
fsp matrix (Variable)
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32]) feature_map_0 = fluid.layers.conv2d(data, num_filters=2, filter_size=3) feature_map_1 = fluid.layers.conv2d(feature_map_0, num_filters=2, filter_size=1) loss = fluid.layers.fsp_matrix(feature_map_0, feature_map_1)
gather¶

paddle.fluid.layers.
gather
(input, index, overwrite=True)[source] Gather Layer
Output is obtained by gathering entries of the outermost dimension of X indexed by index and concatenate them together.
\[Out = X[Index]\]Given: X = [[1, 2], [3, 4], [5, 6]] Index = [1, 2] Then: Out = [[3, 4], [5, 6]]
 Parameters
input (Variable) – The source input tensor with rank>=1. Supported data type is int32, int64, float32, float64 and uint8 (only for CPU), float16 (only for GPU).
index (Variable) – The index input tensor with rank=1. Data type is int32 or int64.
overwrite (bool, optional) – The mode that updating the grad when has same index. If True, use the overwrite mode to update the grad of the same index, if False, use the accumulate mode to update the grad of the same index. Default value is True.
 Returns
The output is a tensor with the same rank as input.
 Return type
output (Variable)
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[1, 5], dtype='float32') index = fluid.data(name='index', shape=[1, 1], dtype='int32') output = fluid.layers.gather(x, index)
gaussian_random¶

paddle.fluid.layers.
gaussian_random
(shape, mean=0.0, std=1.0, seed=0, dtype='float32')[source] Generate a random tensor whose data is drawn from a Gaussian distribution.
 Parameters
shape (Tuple[int]  List[int]) – Shape of the generated random tensor.
mean (float) – Mean of the random tensor, defaults to 0.0.
std (float) – Standard deviation of the random tensor, defaults to 1.0.
seed (int) – (int, default 0) Random seed of generator.0 means use system wide seed.Note that if seed is not 0, this operator will always generate the same random numbers every time
dtype (np.dtype  core.VarDesc.VarType  str) – Output data type, float32 or float64.
 Returns
Random tensor whose data is drawn from a Gaussian distribution, dtype: flaot32 or float64 as specified.
 Return type
Variable
Examples
# declarative mode import numpy as np from paddle import fluid x = fluid.layers.gaussian_random((2, 3), std=2., seed=10) place = fluid.CPUPlace() exe = fluid.Executor(place) start = fluid.default_startup_program() main = fluid.default_main_program() exe.run(start) x_np, = exe.run(main, feed={}, fetch_list=[x]) x_np # array([[2.3060477, 2.676496 , 3.9911983], # [0.9990833, 2.8675377, 2.2279181]], dtype=float32)
# imperative mode import numpy as np from paddle import fluid import paddle.fluid.dygraph as dg place = fluid.CPUPlace() with dg.guard(place) as g: x = fluid.layers.gaussian_random((2, 4), mean=2., dtype="float32", seed=10) x_np = x.numpy() x_np # array([[2.3060477 , 2.676496 , 3.9911983 , 0.9990833 ], # [2.8675377 , 2.2279181 , 0.79029655, 2.8447366 ]], dtype=float32)
gaussian_random_batch_size_like¶

paddle.fluid.layers.
gaussian_random_batch_size_like
(input, shape, input_dim_idx=0, output_dim_idx=0, mean=0.0, std=1.0, seed=0, dtype='float32')[source] Used to initialize tensors with gaussian random generator. The default mean of the distribution is 0, and default standard deviation (std) of the distribution is 1.0. Uers can set mean and std via input arguments.
 Parameters
input (Variable) – Tensor whose input_dim_idx’th dimension specifies the batch_size
shape (tuplelist) – The shape of the output
input_dim_idx (int) – default 0. The index of input’s batch size dimension
output_dim_idx (int) – default 0. The index of output’s batch size dimension
mean (float) – (float, default 0.0) The mean (or center) of the gaussian distribution
std (float) – (float, default 1.0) The standard deviation (std, or spread) of the gaussian distribution
seed (int) – (int, default 0) Random seed of generator.0 means don’t specify random seed.Note that if seed is not 0, this operator will always generate the same random numbers every time
dtype (np.dtypecore.VarDesc.VarTypestr) – The type of output data, float32 or float_64.
 Returns
Tensor of specified shape will be filled with the specified value
 Return type
out (Variable)
Examples
import paddle.fluid as fluid input = fluid.data(name="input", shape=[13, 11], dtype='float32') out = fluid.layers.gaussian_random_batch_size_like( input, shape=[1, 11], mean=1.0, std=2.0)
get_tensor_from_selected_rows¶

paddle.fluid.layers.
get_tensor_from_selected_rows
(x, name=None)[source] This operator gets tensor data from input with SelectedRows type, and outputs a LoDTensor.
input x is SelectedRows: x.rows = [0, 5, 5, 4, 19] x.height = 20 x.value = [[1, 1] [2, 2] [2, 2] [3, 3] [6, 6]] Ouput is LoDTensor: out.shape = [5, 2] out.data = [[1, 1], [2, 2], [2, 2], [3, 3], [6, 6]]
 Parameters
x (SelectedRows) – Input with SelectedRows type. The data type is float32, float64, int32 or int64.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
LoDTensor transformed from SelectedRows. The data type is same with input.
 Return type
Variable
Examples
import paddle.fluid as fluid b = fluid.default_main_program().global_block() input = b.create_var(name="X", dtype="float32", persistable=True, type=fluid.core.VarDesc.VarType.SELECTED_ROWS) out = fluid.layers.get_tensor_from_selected_rows(input)
grid_sampler¶

paddle.fluid.layers.
grid_sampler
(x, grid, name=None)[source] This operation samples input X by using bilinear interpolation based on flow field grid, which is usually gennerated by
affine_grid
. The grid of shape [N, H, W, 2] is the concatenation of (x, y) coordinates with shape [N, H, W] each, where x is indexing the 4th dimension (in width dimension) of input data x and y is indexng the 3rd dimention (in height dimension), finally results is the bilinear interpolation value of 4 nearest corner points. The output tensor shape will be [N, C, H, W].Step 1: Get (x, y) grid coordinates and scale to [0, H1/W1]. .. codeblock:: text grid_x = 0.5 * (grid[:, :, :, 0] + 1) * (W  1) grid_y = 0.5 * (grid[:, :, :, 1] + 1) * (H  1) Step 2: Indices input data X with grid (x, y) in each [H, W] area, and bilinear interpolate point value by 4 nearest points. wn  y_n  en     d_n     x_w d_w gridd_e x_e     d_s     ws  y_s  wn x_w = floor(x) // west side x coord x_e = x_w + 1 // east side x coord y_n = floor(y) // north side y coord y_s = y_s + 1 // south side y coord d_w = grid_x  x_w // distance to west side d_e = x_e  grid_x // distance to east side d_n = grid_y  y_n // distance to north side d_s = y_s  grid_y // distance to south side wn = X[:, :, y_n, x_w] // northwest point value en = X[:, :, y_n, x_e] // northeast point value ws = X[:, :, y_s, x_w] // southeast point value es = X[:, :, y_s, x_w] // northeast point value output = wn * d_e * d_s + en * d_w * d_s + ws * d_e * d_n + es * d_w * d_n
 Parameters
x (Variable) – The input tensor, which is a 4D tensor with shape [N, C, H, W], N is the batch size, C is the channel number, H and W is the feature height and width. The data type is float32 or float64.
grid (Variable) – Input grid tensor of shape [N, H, W, 2]. The data type is float32 or float64.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
 Output of shape [N, C, H, W] data samples input X
using bilnear interpolation based on input grid. The data type is same as input tensor.
 Return type
Variable
Examples
import paddle.fluid as fluid # use with affine_grid x = fluid.data(name='x', shape=[None, 10, 32, 32], dtype='float32') theta = fluid.layers.data(name='theta', shape=[2, 3], dtype='float32') grid = fluid.layers.affine_grid(theta=theta, out_shape=[3, 10, 32, 32]) out = fluid.layers.grid_sampler(x=x, grid=grid)
group_norm¶

paddle.fluid.layers.
group_norm
(input, groups, epsilon=1e05, param_attr=None, bias_attr=None, act=None, data_layout='NCHW', name=None)[source] Group Normalization Layer
Refer to Group Normalization .
 Parameters
input (Variable) – 4D Tensor, the data type is float32 or float64.
groups (int) – The number of groups that divided from channels, the data type is int32.
epsilon (float, optional) – The small value added to the variance to prevent division by zero, the data type is float32. Default: 1e05.
param_attr (ParamAttrbool, optional) – ParamAttr object that specifies weight parameter attribute. If a bool type, only False is supported, which means there is no weight parameter. Default: None, the default weight parameter attribute is used. For more information, please refer to api_guide_ParamAttr .
bias_attr (ParamAttrbool, optional) – ParamAttr object that specifies bias parameter attribute. If a bool type, only False is supported, which means there is no bias parameter. Default: None, the default bias parameter attribute is used. For more information, please refer to api_guide_ParamAttr .
act (str, optional) – Activation to be applied to the output of group normalizaiton.
data_layout (str, optional) – The data format of the input and output data. An optional string from: “NCHW”, “NHWC”. When it is “NCHW”, the data is stored in the order of: [batch_size, channels, height, width]. Default: “NCHW”.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
A 4D Tensor has same data type and data format with input.
 Return type
Variable
 Raises
ValueError
– If data_layout is neither ‘NCHW’ nor ‘NHWC’.
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 8, 32, 32], dtype='float32') x = fluid.layers.group_norm(input=data, groups=4)
gru_unit¶

paddle.fluid.layers.
gru_unit
(input, hidden, size, param_attr=None, bias_attr=None, activation='tanh', gate_activation='sigmoid', origin_mode=False)[source] Gated Recurrent Unit (GRU) RNN cell. This operator performs GRU calculations for one time step and it supports these two modes:
If
origin_mode
is True, then the formula used is from paper Learning Phrase Representations using RNN Encoder Decoder for Statistical Machine Translation .\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t1}) + b_c)\\h_t & = u_t \odot h_{t1} + (1u_t) \odot \tilde{h_t}\end{aligned}\end{align} \]if
origin_mode
is False, then the formula used is from paper Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling\[ \begin{align}\begin{aligned}u_t & = act_g(W_{ux}x_{t} + W_{uh}h_{t1} + b_u)\\r_t & = act_g(W_{rx}x_{t} + W_{rh}h_{t1} + b_r)\\\tilde{h_t} & = act_c(W_{cx}x_{t} + W_{ch}(r_t \odot h_{t1}) + b_c)\\h_t & = (1u_t) \odot h_{t1} + u_t \odot \tilde{h_t}\end{aligned}\end{align} \]\(x_t\) is the input of current time step, but it is not
input
. This operator does not include the calculations \(W_{ux}x_{t}, W_{rx}x_{t}, W_{cx}x_{t}\) , Note thus a fullyconnect layer whose size is 3 times of GRU hidden size should be used before this operator, and the output should be used asinput
here. \(h_{t1}\) is the hidden state from previous time step. \(u_t\) , \(r_t\) , \(\tilde{h_t}\) and \(h_t\) stand for update gate, reset gate, candidate hidden and hidden output separately. \(W_{uh}, b_u\) , \(W_{rh}, b_r\) and \(W_{ch}, b_c\) stand for the weight matrix and bias used in update gate, reset gate, candidate hidden calculations. For implementation, the three weight matrix are merged into a tensor shaped \([D, D \times 3]\) , the three bias are concatenated as a tensor shaped \([1, D \times 3]\) , where \(D\) stands for the hidden size; The data layout of weight tensor is: \(W_{uh}\) and \(W_{rh}\) are concatenated with shape \([D, D \times 2]\) lying on the first part, and \(W_{ch}\) lying on the latter part with shape \([D, D]\) . Parameters
input (Variable) – A 2D Tensor representing the input after linear projection after linear projection. Its shape should be \([N, D \times 3]\) , where \(N\) stands for batch size, \(D\) for the hidden size. The data type should be float32 or float64.
hidden (Variable) – A 2D Tensor representing the hidden state from previous step. Its shape should be \([N, D]\) , where \(N\) stands for batch size, \(D\) for the hidden size. The data type should be same as
input
.size (int) – Indicate the hidden size.
param_attr (ParamAttr, optional) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
bias_attr (ParamAttr, optional) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .
activation (str, optional) – The activation fuction corresponding to \(act_c\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “tanh”.
gate_activation (str, optional) – The activation fuction corresponding to \(act_g\) in the formula. “sigmoid”, “tanh”, “relu” and “identity” are supported. Default “sigmoid”.
 Returns
The tuple contains three Tensor variables with the same data type as
input
. They represent the hidden state for next time step ( \(h_t\) ), reseted previous hidden state ( \(r_t \odot h_{t1}\) ), and the concatenation of \(h_t, r_t, \tilde{h_t}\) . And they have shape \([N, D]\) , \([N, D]\) , \([N, D imes 3]\) separately. Usually only the hidden state for next time step ( \(h_t\) ) is used as output and state, the other two are intermediate results of calculations. Return type
tuple
Examples
import paddle.fluid as fluid dict_dim, emb_dim = 128, 64 data = fluid.data(name='step_data', shape=[None], dtype='int64') emb = fluid.embedding(input=data, size=[dict_dim, emb_dim]) hidden_dim = 512 x = fluid.layers.fc(input=emb, size=hidden_dim * 3) pre_hidden = fluid.data( name='pre_hidden', shape=[None, hidden_dim], dtype='float32') hidden = fluid.layers.gru_unit( input=x, hidden=pre_hidden, size=hidden_dim * 3)
hard_sigmoid¶

paddle.fluid.layers.
hard_sigmoid
(x, slope=0.2, offset=0.5, name=None)[source] HardSigmoid Activation Operator.
A 3part piecewise linear approximation of sigmoid(https://arxiv.org/abs/1603.00391), which is much faster than sigmoid.
\(out = \max(0, \min(1, slope * x + offset))\)
 Parameters
x (Variable) – An ND Tensor with data type float32, float64.
slope (float, optional) – The slope of the linear approximation of sigmoid. Its value MUST BE positive. Default is 0.2.
offset (float, optional) – The offset of the linear approximation of sigmoid. Default is 0.5.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
A Tensor with the same shape as input.
 Return type
Variable
Examples
import paddle.fluid as fluid data = fluid.layers.fill_constant(shape=[3, 2], value=0.5, dtype='float32') # [[0.5, 0.5], [0.5, 0.5], [0.5, 0.5]] result = fluid.layers.hard_sigmoid(data) # [[0.6, 0.6], [0.6, 0.6], [0.6, 0.6]]
hash¶

paddle.fluid.layers.
hash
(input, hash_size, num_hash=1, name=None)[source] This OP hash the input to an integer less than the hash_size. The hash algorithm we used was xxHash  Extremely fast hash algorithm (https://github.com/Cyan4973/xxHash/tree/v0.6.5)
 Parameters
input (Variable) – A TwoDimensional LoDTensor with type int32, int64. Only support LoDTensor.
num_hash (int, optional) – The times of hash, default is 1.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.
 Returns
A LoDTensor with the same data type as input.
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np place = fluid.core.CPUPlace() x = fluid.data(name="x", shape=[1], dtype="int32", lod_level=1) res = fluid.layers.hash(name="res",input=x, hash_size=1000, num_hash=4) exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) in1 = np.array([[1,2],[3,4]]).astype("int32") print(in1) x_i = fluid.core.LoDTensor() x_i.set(in1,place) x_i.set_recursive_sequence_lengths([[0,2]]) res = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res], return_numpy=False) print(np.array(res[0])) # [[[722] # [407] # [337] # [395]] # [[603] # [590] # [386] # [901]]]
hsigmoid¶

paddle.fluid.layers.
hsigmoid
(input, label, num_classes, param_attr=None, bias_attr=None, name=None, path_table=None, path_code=None, is_custom=False, is_sparse=False)[source] The hierarchical sigmoid organizes the classes into a complete binary tree to reduce the computational complexity and speed up the model training, especially the training of language model. Each leaf node of the complete binary tree represents a class(word) and each nonleaf node acts as a binary classifier. For each class(word), there’s a unique path from root to itself, hsigmoid calculate the cost for each nonleaf node on the path, and sum them to get a total cost. Comparing to softmax, the OP can reduce the computational complexity from \(O(N)\) to \(O(logN)\), where \(N\) represents the number of classes or the size of word dict.
The OP supports default tree and custom tree. For the default tree, you can refer to Hierarchical Probabilistic Neural Network Language Model <http://www.iro.umontreal.ca/~lisa/pointeurs/hierarchicalnnlmaistats05.pdf>. For the custom tree, you need to set
is_custom
to True, and do the following steps (take the language model as an example):Using a custom word dict to build a binary tree, each leaf node should be an word in the word dict.
Creating a dict map word_id > path that from the word to the root node, we call it path_table.
Creating a dict map word_id > code of path that from the word to the root node, we call it path_code. Code means the label of each binary classifier, 1 indicate true, 0 indicate false.
Now, each word should has its path and code along the path, you can pass a batch of path and code related to the same batch of inputs.
 Parameters
input (Variable) – A tensor with the shape [N, D], where N is the size of minibatch, and D is the feature size. Its data type supports float32 and float64.
label (Variable) – A tensor contains the labels of training data. Its shape is [N, 1] and data type is int64.
num_classes (int) – The number of classes or the size of word dict, must be greater than 2. If the default tree is used (
is_custom
is set to False),num_classes
should not be None. If the custom tree is used (is_custom
is set to True),num_classes
should be the number of nonleaf nodes, which indicates the num of classes using by the binary classifier.param_attr (ParamAttr, optional) – The parameter attribute for the learnable parameters/weights of hsigmoid. If it is set to None or one attribute of ParamAttr, hsigmoid will create a ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with Xavier. Default: None.
bias_attr (ParamAttrbool, optional) – The parameter attribute for the bias of hsigmoid. If it is set to False, no bias will be added. If it is set to None or one attribute of ParamAttr, hsigmoid will create a ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.
name (str, optional) – Normally there is no need for user to set this property. For more information, please refer to api_guide_Name. Default: None.
path_table (Variable, optional) – A tensor that stores each batch of samples’ path from leaf to root node, its shape is [N, L] and data type is int64, where L is the length of path. For each sample i, path_table[i] is a np.array like structure and each element in this array is the indexes in parent nodes’ weight matrix. Default: None.
path_code (Variable, optional) – A tensor that stores each batch of samples’ code of path from leaf to root node, its shape is [N, L] and data type is int64, which is the same as
path_table
. Each code of path is consisted with the code of nodes from leaf to root node. Default: None.is_custom (bool, optional) – Whether use custom binary tree. If it’s True,
path_table
,path_code
andnum_classes
should be set, otherwisenum_classes
should be set. Default: False.is_sparse (bool, optional) – Whether use sparse updating instead of dense updating, if it’s True, the gradient of W and input will be sparse. Default: False.
 Returns
A tensor with the cost of hierarchical sigmoid, its shape is [N, 1] and data type is the same as
input
. Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.layers.fill_constant(shape=[4, 3], value=0.9, dtype='float32') # x = [[0.9, 0.9, 0.9], [0.9, 0.9, 0.9], [0.9, 0.9, 0.9], [0.9, 0.9, 0.9]] y = fluid.layers.fill_constant( shape=[4, 1], value=1, dtype='int64') # y = [[1], [1], [1], [1]] out = fluid.layers.hsigmoid(input=x, label=y, num_classes=2, param_attr=fluid.initializer.Constant( value=0.05), bias_attr=fluid.initializer.Constant(value=.0)) # out = [[0.62792355], [0.62792355], [0.62792355], [0.62792355]]
huber_loss¶

paddle.fluid.layers.
huber_loss
(input, label, delta)[source] This operator computes the Huber loss between input and label. Huber loss is commonly used in regression tasks. Compared to square_error_cost, Huber loss is more robust and less sensitivity to outliers.
When the absolute difference between input and label is greater than delta, the linear error is calculated:
\[huber\_loss = delta * (label  input)  0.5 * delta * delta\]When the absolute difference between input and label is greater than delta, the square error is calculated:
\[huber\_loss = 0.5 * (label  input) * (label  input)\] Parameters
input (Variable) – Predicted data, 2DTensor with the shape of [batch_size, 1]. The data type should be float32 or float64.
label (Variable) – Ground truth label, 2DTensor with the shape of [batch_size, 1]. The data type should be float32 or float64.
delta (float) – The threshold for Huber loss, which is used to control the balance between the linear error and square error. The data type should be float32.
 Returns
The huber loss, a tensor with the same shape and data type as input.
 Return type
Variable
Examples:
import paddle.fluid as fluid import numpy as np DATATYPE='float32' input_data = np.array([[1.],[2.],[3.],[4.]]).astype(DATATYPE) label_data = np.array([[3.],[3.],[4.],[4.]]).astype(DATATYPE) x = fluid.data(name='input', shape=[None, 1], dtype=DATATYPE) y = fluid.data(name='label', shape=[None, 1], dtype=DATATYPE) loss = fluid.layers.huber_loss(input=x, label=y, delta=1.0) place = fluid.CPUPlace() #place = fluid.CUDAPlace(0) exe = fluid.Executor(place) HuberLoss, = exe.run(feed={'input':input_data ,'label':label_data}, fetch_list=[loss.name]) print(HuberLoss) #[[1.5], [0.5], [0.5], [0. ]], dtype=float32
im2sequence¶

paddle.fluid.layers.
im2sequence
(input, filter_size=1, stride=1, padding=0, input_image_size=None, out_stride=1, name=None)[source] Extracts image patches from the input tensor to form a tensor of shape {input.batch_size * output_height * output_width, filter_size_height * filter_size_width * input.channels}. This op use filter to scan images and convert these images to sequences. After expanding, the number of time step are output_height * output_width for an image, in which output_height and output_width are calculated by below equation:
\[\begin{split}output\_height = 1 + (padding\_up + padding\_down + input\_height  filter\_size\_height + stride\_height  1) / stride\_height \\ output\_width = 1 + (padding\_left + padding\_right + input\_width  filter\_size\_width + stride\_width  1) / stride\_width\end{split}\]And the dimension of each time step is filter_size_height * filter_size_width * input.channels.
 Parameters
input (Variable) – The input should be a 4D Tensor in \(NCHW\) format. The data type is float32.
filter_size (int32  List[int32]) – The filter size. If filter_size is a List, it must contain two integers, \([filter\_size\_height, filter\_size\_width]\) . Otherwise, the filter size will be a square \([filter\_size, filter\_size]\) . Default is 1.
stride (int32  List[int32]) – The stride size. If stride is a List, it must contain two integers, \([stride\_height, stride\_width]\) . Otherwise, the stride size will be a square \([stride\_size, stride\_size]\) . Default is 1.
padding (int32  List[int32]) – The padding size. If padding is a List, it can contain four integers like \([padding\_up, padding\_left, padding\_down, padding\_right]\) to indicate paddings of four direction. Or it can contain two integers \([padding\_height, padding\_width]\) which means padding_up = padding_down = padding_height and padding_left = padding_right = padding_width. Otherwise, a scalar padding means padding_up = padding_down = padding_left = padding_right = padding. Default is 0.
input_image_size (Variable, optional) – the input contains image real size.It’s dim is \([batchsize, 2]\) . It is just for batch inference when not None. Default is None.
out_stride (int32  List[int32]) – The scaling of image through CNN. It is valid only when input_image_size is not None. If out_stride is List, it must contain two intergers, \([out\_stride\_height, out\_stride\_W]\) . Otherwise, the out_stride_height = out_stride_width = out_stride. Default is 1.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
The output is a 2D LoDTensor with shape {input.batch_size * output_height * output_width, filter_size_height * filter_size_width * input.channels}. The data type is float32.
Return Type: Variable
Examples
Given: x = [[[[ 6. 2. 1.] [ 8. 3. 5.] [ 0. 2. 6.]] [[ 2. 4. 4.] [ 6. 3. 0.] [ 6. 4. 7.]]] [[[ 6. 7. 1.] [ 5. 7. 9.] [ 2. 4. 8.]] [[ 1. 2. 1.] [ 1. 3. 5.] [ 9. 0. 8.]]]] x.dims = {2, 2, 3, 3} And: filter = [2, 2] stride = [1, 1] padding = [0, 0] Then: output.data = [[ 6. 2. 8. 3. 2. 4. 6. 3.] [ 2. 1. 3. 5. 4. 4. 3. 0.] [ 8. 3. 0. 2. 6. 3. 6. 4.] [ 3. 5. 2. 6. 3. 0. 4. 7.] [ 6. 7. 5. 7. 1. 2. 1. 3.] [ 7. 1. 7. 9. 2. 1. 3. 5.] [ 5. 7. 2. 4. 1. 3. 9. 0.] [ 7. 9. 4. 8. 3. 5. 0. 8.]] output.dims = {8, 8} output.lod = [[4, 4]]
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32') output = fluid.layers.im2sequence( input=data, stride=[1, 1], filter_size=[2, 2])
image_resize¶

paddle.fluid.layers.
image_resize
(input, out_shape=None, scale=None, name=None, resample='BILINEAR', actual_shape=None, align_corners=True, align_mode=1, data_format='NCHW')[source] This op resizes a batch of images.
The input must be a 4D Tensor of the shape (num_batches, channels, in_h, in_w) or (num_batches, in_h, in_w, channels), or a 5D Tensor of the shape (num_batches, channels, in_d, in_h, in_w) or (num_batches, in_d, in_h, in_w, channels), and the resizing only applies on the three dimensions(depth, hight and width).
Warning: the parameter
actual_shape
will be deprecated in the future and only useout_shape
instead.Supporting resample methods:
‘BILINEAR’ : Bilinear interpolation
‘TRILINEAR’ : Trilinear interpolation
‘NEAREST’ : Nearest neighbor interpolation
Nearest neighbor interpolation is to perform nearest neighbor interpolation in both the 3rd dimention(in height direction) and the 4th dimention(in width direction) on input tensor.
Bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables (e.g. Hdirection and Wdirection in this op) on a rectilinear 2D grid. The key idea is to perform linear interpolation first in one direction, and then again in the other direction.
Trilinear interpolation is an extension of linear interpolation for interpolating functions of three variables (e.g. Ddirection, Hdirection and Wdirection in this op) on a rectilinear 3D grid. The linear interpolation is performed on three directions.
Align_corners and align_mode are optinal parameters,the calculation method of interpolation can be selected by them.
Example:
For scale: if align_corners = True && out_size > 1 : scale_factor = (in_size1.0)/(out_size1.0) else: scale_factor = float(in_size/out_size) Nearest neighbor interpolation: if: align_corners = False input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = floor (H_{in} * scale_{factor}) W_out = floor (W_{in} * scale_{factor}) else: align_corners = True input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = round(H_{in} * scale_{factor}) W_out = round(W_{in} * scale_{factor}) Bilinear interpolation: if: align_corners = False , align_mode = 0 input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = (H_{in}+0.5) * scale_{factor}  0.5 W_out = (W_{in}+0.5) * scale_{factor}  0.5 else: input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = H_{in} * scale_{factor} W_out = W_{in} * scale_{factor} Trilinear interpolation: if: align_corners = False , align_mode = 0 input : (N,C,D_in,H_in,W_in) output: (N,C,D_out,H_out,W_out) where: D_out = (D_{in}+0.5) * scale_{factor}  0.5 H_out = (H_{in}+0.5) * scale_{factor}  0.5 W_out = (W_{in}+0.5) * scale_{factor}  0.5 else: input : (N,C,D_in,H_in,W_in) output: (N,C,D_out,H_out,W_out) where: D_out = D_{in} * scale_{factor} H_out = H_{in} * scale_{factor} W_out = W_{in} * scale_{factor}
For details of nearest neighbor interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Nearestneighbor_interpolation.
For details of bilinear interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation.
For details of trilinear interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Trilinear_interpolation.
 Parameters
input (Variable) – 4D or 5D Tensor, its data type is float32, float64, or uint8, its data format is specified by
data_format
.out_shape (listtupleVariableNone) – Output shape of image resize layer, the shape is (out_h, out_w) when input is a 4D Tensor and is (out_d, out_h, out_w) when input is a 5D Tensor. Default: None. If a list, each element can be an integer or a Tensor Variable of shape: [1]. If a Tensor Variable, its dimensions size should be a 1.
scale (floatVariableNone) – The multiplier for the input height or width. At least one of
out_shape
orscale
must be set. Andout_shape
has a higher priority thanscale
. Default: None.name (strNone) – A name for this layer(optional). If set None, the layer will be named automatically.
resample (str) – The resample method. It supports ‘BILINEAR’, ‘TRILINEAR’ and ‘NEAREST’ currently. Default: ‘BILINEAR’
actual_shape (Variable) – An optional input to specify output shape dynamically. If provided, image resize according to this given shape rather than
out_shape
andscale
specifying shape. That is to say actual_shape has the highest priority. It is recommended to useout_shape
if you want to specify output shape dynamically, becauseactual_shape
will be deprecated. When using actual_shape to specify output shape, one ofout_shape
andscale
should also be set, otherwise errors would be occured in graph constructing stage. Default: Nonealign_corners (bool) – An optional bool, If True, the centers of the 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels. Default: True
align_mode (int) – An optional for bilinear interpolation. can be ‘0’ for src_idx = scale*(dst_indx+0.5)0.5 , can be ‘1’ for src_idx = scale*dst_index.
data_format (str, optional) – NCHW(num_batches, channels, height, width) or NHWC(num_batches, height, width, channels) for 4D Tensor, NCDHW(num_batches, channels, depth, height, width) or NDHWC(num_batches, depth, height, width, channels) for 5D Tensor. Default: ‘NCHW’.
 Returns
A 4D Tensor of the shape (num_batches, channels, out_h, out_w) or (num_batches, out_h, out_w, channels), or 5D Tensor of the shape (num_batches, channels, out_d, out_h, out_w) or (num_batches, out_d, out_h, out_w, channels).
 Raises
TypeError
– out_shape should be a list or tuple or Variable.TypeError
– actual_shape should either be Variable or None.ValueError
– The ‘resample’ of image_resize can only be ‘BILINEAR’, ‘TRILINEAR’ or ‘NEAREST’ currently.ValueError
– ‘BILINEAR’ and ‘NEAREST’ only support 4D tensor.ValueError
– ‘TRILINEAR’ only support 5D tensor.ValueError
– One of out_shape and scale must not be None.ValueError
– out_shape length should be 2 for input 4D tensor.ValueError
– out_shape length should be 3 for input 5D tensor.ValueError
– scale should be greater than zero.TypeError
– align_corners shoule be a bool valueValueError
– align_mode can only be ‘0’ or ‘1’ValueError
– data_format can only be ‘NCHW’, ‘NHWC’, ‘NCDHW’ or ‘NDHWC’.
Examples
#declarative mode import paddle.fluid as fluid import numpy as np input = fluid.data(name="input", shape=[None,3,6,10]) #1 output = fluid.layers.image_resize(input=input,out_shape=[12,12]) #2 #x = np.array([2]).astype("int32") #dim1 = fluid.data(name="dim1", shape=[1], dtype="int32") #fluid.layers.assign(input=x, output=dim1) #output = fluid.layers.image_resize(input=input,out_shape=[12,dim1]) #3 #x = np.array([3,12]).astype("int32") #shape_tensor = fluid.data(name="shape_tensor", shape=[2], dtype="int32") #fluid.layers.assign(input=x, output=shape_tensor) #output = fluid.layers.image_resize(input=input,out_shape=shape_tensor) #4 #x = np.array([0.5]).astype("float32") #scale_tensor = fluid.data(name="scale", shape=[1], dtype="float32") #fluid.layers.assign(x,scale_tensor) #output = fluid.layers.image_resize(input=input,scale=scale_tensor) place = fluid.CPUPlace() exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) input_data = np.random.rand(2,3,6,10).astype("float32") output_data = exe.run(fluid.default_main_program(), feed={"input":input_data}, fetch_list=[output], return_numpy=True) print(output_data[0].shape) #1 # (2, 3, 12, 12) #2 # (2, 3, 12, 2) #3 # (2, 3, 3, 12) #4 # (2, 3, 3, 5) #imperative mode import paddle.fluid.dygraph as dg with dg.guard(place) as g: input = dg.to_variable(input_data) output = fluid.layers.image_resize(input=input, out_shape=[12,12]) print(output.shape) # [2L, 3L, 12L, 12L]
image_resize_short¶

paddle.fluid.layers.
image_resize_short
(input, out_short_len, resample='BILINEAR')[source] This op resizes a batch of images. The short edge of input images will be resized to the given ‘out_short_len’. The long edge of input images will be resized proportionately to make images’ lengthwidth ratio constant.
 Parameters
input (Variable) – 4D tensor(NCHW), The input tensor of image resize layer.
out_short_len (int) – The length of output images’ short edge.
resample (str) – resample method, default: BILINEAR.
 Returns
4D tensor(NCHW).
 Return type
Variable
Examples
import paddle.fluid as fluid input = fluid.data(name="input", shape=[None,3,6,9], dtype="float32") out = fluid.layers.image_resize_short(input, out_short_len=3)
kldiv_loss¶

paddle.fluid.layers.
kldiv_loss
(x, target, reduction='mean', name=None)[source] This operator calculates the KullbackLeibler divergence loss between Input(X) and Input(Target). Notes that Input(X) is the logprobability and Input(Target) is the probability.
KL divergence loss is calculated as follows:
$$l(x, y) = y * (log(y)  x)$$
While \(x\) is Input(X) and \(y\) is Input(Target).
While
reduction
isnone
, output loss is in the same shape as Input(X), loss in each point is calculated seperately and no reduction is applied.While
reduction
ismean
, output loss is in shape of [1] and loss value is the mean value of all losses.While
reduction
issum
, output loss is in shape of [1] and loss value is the sum value of all losses.While
reduction
isbatchmean
, output loss is in shape of [1] and loss value is the sum value of all losses divided by batch size. Parameters
x (Variable) – The input tensor of KL divergence loss operator. This is a tensor with shape of [N, *], where N is the batch size, * means any number of additional dimensions. The data type is float32 or flaot64
target (Variable) – The tensor of KL divergence loss operator. This is a tensor with shape of Input(X). The data type is same as Input(X)
reduction (Variable) – The reduction type to apply to the output, available types are ‘none’  ‘batchmean’  ‘mean’  ‘sum’, ‘none’ for no reduction, ‘batchmean’ for the sum of output divided by batch size, ‘mean’ for the average value of all output, ‘sum’ for the sum of the output
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
The KL divergence loss. The data type is same as input tensor
 Return type
Variable(Tensor)
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[None,4,2,2], dtype='float32') target = fluid.layers.data(name='target', shape=[4,2,2], dtype='float32') loss = fluid.layers.kldiv_loss(x=x, target=target, reduction='batchmean')
l2_normalize¶

paddle.fluid.layers.
l2_normalize
(x, axis, epsilon=1e12, name=None)[source] This op normalizes x along dimension axis using an L2 norm. For a 1D tensor (dim is fixed to 0), this layer computes
\[y = \frac{x}{ \sqrt{\sum {x^2} + epsion }}\]For x with more dimensions, this layer independently normalizes each 1D slice along dimension axis.
 Parameters
x (Variablelist) – The input tensor could be ND tensor, and the input data type could be float32 or float64.
axis (int) – The axis on which to apply normalization. If axis < 0, the dimension to normalization is rank(X) + axis. 1 is the last dimension.
epsilon (float) – The epsilon value is used to avoid division by zero, the default value is 1e12.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
The output has the same shape and data type with x.
 Return type
Variable
Examples
# declarative mode import paddle.fluid as fluid import numpy as np input = fluid.data(name="input", shape=[2,3]) output = fluid.layers.l2_normalize(x=input,axis=0) place = fluid.CPUPlace() exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) input_data = np.random.rand(2,3).astype("float32") print(input_data) # [[0.5171216 0.12704141 0.56018186] # [0.93251234 0.5382788 0.81709313]] output_data = exe.run(fluid.default_main_program(), feed={"input":input_data}, fetch_list=[output], return_numpy=True) print(output_data) # [array([[0.48496857, 0.22970329, 0.56545246], # [0.8745316 , 0.9732607 , 0.82478094]], dtype=float32)] # imperative mode import paddle.fluid.dygraph as dg with dg.guard(place) as g: input = dg.to_variable(input_data) output = fluid.layers.l2_normalize(x=input, axis=1) print(output.numpy()) # [[0.66907585 0.16437206 0.7247892 ] # [0.6899054 0.3982376 0.6045142 ]]
label_smooth¶

paddle.fluid.layers.
label_smooth
(label, prior_dist=None, epsilon=0.1, dtype='float32', name=None)[source] Label smoothing is a mechanism to regularize the classifier layer and is called labelsmoothing regularization (LSR).
Label smoothing is proposed to encourage the model to be less confident, since optimizing the loglikelihood of the correct label directly may cause overfitting and reduce the ability of the model to adapt. Label smoothing replaces the groundtruth label \(y\) with the weighted sum of itself and some fixed distribution \(\mu\). For class \(k\), i.e.
\[\tilde{y_k} = (1  \epsilon) * y_k + \epsilon * \mu_k,\]where \(1  \epsilon\) and \(\epsilon\) are the weights respectively, and \(\tilde{y}_k\) is the smoothed label. Usually uniform distribution is used for \(\mu\).
See more details about label smoothing in https://arxiv.org/abs/1512.00567.
 Parameters
label (Variable) – The input variable containing the label data. The label data should use onehot representation. It’s a multidimensional tensor with a shape of \([N_1, ..., Depth]\), where Depth is class number.
prior_dist (Variable, optional) – The prior distribution to be used to smooth labels. If not provided, an uniform distribution is used. It’s a multidimensional tensor with a shape of \([1, class\_num]\) . The default value is None.
epsilon (float, optional) – The weight used to mix up the original groundtruth distribution and the fixed distribution. The default value is 0.1.
dtype (np.dtypecore.VarDesc.VarTypestr, optional) – The data type can be set as ‘float32’, ‘float64’. The default value is ‘float32’.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.
 Returns
The tensor variable containing the smoothed labels.
 Return type
Variable
Examples
import paddle.fluid as fluid import paddle.fluid.layers as layers label = layers.data(name="label", shape=[1], dtype="float32") one_hot_label = layers.one_hot(input=label, depth=10) smooth_label = layers.label_smooth( label=one_hot_label, epsilon=0.1, dtype="float32")
layer_norm¶

paddle.fluid.layers.
layer_norm
(input, scale=True, shift=True, begin_norm_axis=1, epsilon=1e05, param_attr=None, bias_attr=None, act=None, name=None)[source] Layer Normalization Layer
The API implements the function of the Layer Normalization Layer and can be applied to minibatch input data. Refer to Layer Normalization
The formula is as follows:
\[ \begin{align}\begin{aligned}\mu & = \frac{1}{H}\sum_{i=1}^{H} x_i\\\sigma & = \sqrt{\frac{1}{H}\sum_{i=1}^{H}{(x_i  \mu)^2} + \epsilon}\\y & = f(\frac{g}{\sigma}(x  \mu) + b)\end{aligned}\end{align} \]\(x\): the vector representation of the summed inputs to the neurons in that layer.
\(H\): the number of hidden units in a layers
\(\epsilon\): the small value added to the variance to prevent division by zero.
\(g\): the trainable scale parameter.
\(b\): the trainable bias parameter.
 Parameters
input (Variable) – A multidimension
Tensor
, and the data type is float32 or float64.scale (bool, optional) – Whether to learn the adaptive gain \(g\) after normalization. Default: True.
shift (bool, optional) – Whether to learn the adaptive bias \(b\) after normalization. Default: True.
begin_norm_axis (int, optional) – The normalization will be performed along dimensions from
begin_norm_axis
torank(input)
. Default: 1.epsilon (float, optional) – The small value added to the variance to prevent division by zero. Default: 1e05.
param_attr (ParamAttr, optional) – The parameter attribute for the learnable gain \(g\). If
scale
is False,param_attr
is omitted. Ifscale
is True andparam_attr
is None, a defaultParamAttr
would be added as scale. Theparam_attr
is initialized as 1 if it is added. Default: None.bias_attr (ParamAttr, optional) – The parameter attribute for the learnable bias \(b\). If
shift
is False,bias_attr
is omitted. Ifshift
is True andparam_attr
is None, a defaultParamAttr
would be added as bias. Thebias_attr
is initialized as 0 if it is added. Default: None.act (str, optional) – Activation to be applied to the output of layer normalizaiton. Default: None.
name (str) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
Tensor
indicating the normalized result, the data type is the same asinput
, and the return dimension is the same asinput
. Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np x = fluid.data(name='x', shape=[1, 32, 32], dtype='float32') hidden1 = fluid.layers.layer_norm(input=x, begin_norm_axis=1) place = fluid.CPUPlace() exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) np_x = np.random.random(size=(8, 3, 32, 32)).astype('float32') output = exe.run(feed={"x": np_x}, fetch_list = [hidden1]) print(output)
leaky_relu¶

paddle.fluid.layers.
leaky_relu
(x, alpha=0.02, name=None)[source] LeakyRelu Activation Operator.
$$out = max(x, alpha * x)$$
 Parameters
x (Variable) – A LoDTensor or Tensor representing preactivation values. Must be one of the following types: float32, float64
alpha (FLOAT0.02) – Slope of the activation function at x < 0
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
A LoDTensor or Tensor with the same type and size as that of x
 Return type
output(Variable)
Examples
import paddle.fluid as fluid import numpy as np # Graph Organizing x = fluid.layers.data(name="x", shape=[2], dtype="float32") res = fluid.layers.leaky_relu(x, alpha=0.1) # Create an executor using CPU as an example exe = fluid.Executor(fluid.CPUPlace()) # Execute x_i = np.array([[1, 2], [3, 4]]).astype(np.float32) res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res]) print(res_val) # [[0.1, 2], [3, 0.4]]
linear_chain_crf¶

paddle.fluid.layers.
linear_chain_crf
(input, label, param_attr=None, length=None)[source] Linear Chain CRF.
Conditional Random Field defines an undirected probabilistic graph with nodes denoting random variables and edges denoting dependencies between these variables. CRF learns the conditional probability \(P(YX)\), where \(X = (x_1, x_2, ... , x_n)\) are structured inputs and \(Y = (y_1, y_2, ... , y_n)\) are labels for the inputs.
Linear chain CRF is a special case of CRF that is useful for sequence labeling task. Sequence labeling tasks do not assume a lot of conditional independences among inputs. The only constraint they impose is that the input and output must be linear sequences. Thus, the graph of such a CRF is a simple chain or a line, which results in the linear chain CRF.
This operator implements the ForwardBackward algorithm for the linear chain CRF. Please refer to http://www.cs.columbia.edu/~mcollins/fb.pdf and http://cseweb.ucsd.edu/~elkan/250Bwinter2012/loglinearCRFs.pdf for details.
Equation:
Denote Input(Emission) to this operator as \(x\) here. 2. The first D values of Input(Transition) to this operator are for starting weights, denoted as \(a\) here. 3. The next D values of Input(Transition) of this operator are for ending weights, denoted as \(b\) here. 4. The remaning values of Input(Transition) are for transition weights, denoted as \(w\) here. 5. Denote Input(Label) as \(s\) here.
The probability of a sequence \(s\) of length \(L\) is defined as: $$P(s) = (1/Z) exp(a_{s_1} + b_{s_L} + sum_{l=1}^L x_{s_l} + sum_{l=2}^L w_{s_{l1},s_l})$$
where \(Z\) is a normalization value so that the sum of \(P(s)\) over all possible sequences is 1, and \(x\) is the emission feature weight to the linear chain CRF.
Finally, the linear chain CRF operator outputs the logarithm of the conditional likelihood of each training sample in a minibatch.
NOTE:
The feature function for a CRF is made up of the emission features and the transition features. The emission feature weights are NOT computed in this operator. They MUST be computed first before this operator is called.
Because this operator performs global normalization over all possible sequences internally, it expects UNSCALED emission feature weights. Please do not call this op with the emission feature being output of any nonlinear activation.
The 2nd dimension of Input(Emission) MUST be equal to the tag number.
 Parameters
input (Variable) – (LoDTensor/Tensor<float>). When a LoDTensor input,A 2D LoDTensor with shape [N x D], where N is the size of the minibatch and D is the total tag number. The unscaled emission weight matrix for the linear chain CRF. When a Tensor input,A Tensor with shape [N x S x D], where N is batch number,S is max length of sequences, D is the total tag number.A LoDTensor or Tensor with type float32, float64
label (Variable) – (LoDTensor/Tensor<int64_t>), when a LoDTensor input, [N x 1], where N is the total element number in a minibatch. when a Tensor input, [N x S], where N is batch number. S is max length of sequences. The ground truth.A LoDTensor or Tensor with int64
Length (Variable) – (Tensor, default Tensor<int64_t>) A Tensor with shape [M x 1], where M is the sequence number in a minibatch.A Tensor with type int64
param_attr (ParamAttr) – The attribute of the learnable parameter for transition parameter.
 Returns
(Tensor, default Tensor<float>), the same shape with Emission. The exponentials of Input(Emission). This is an intermediate computational result in forward computation, and will be reused in backward computation.A LoDTensor or Tensor with type float32, float64
output(Variable): (Tensor, default Tensor<float>) A 2D Tensor with shape [(D + 2) x D]. The exponentials of Input(Transition). This is an intermediate computational result in forward computation, and will be reused in backward computation.A LoDTensor or Tensor with type float32, float64
output(Variable): (Tensor, default Tensor<float>) The logarithm of the conditional likelihood of each training sample in a minibatch. This is a 2D tensor with shape [S x 1], where S is the sequence number in a minibatch. Note: S is equal to the sequence number in a minibatch. A Tensor with type float32, float64
 Return type
output(Variable)
Examples
import paddle.fluid as fluid import numpy as np #define net structure, using LodTensor train_program = fluid.Program() startup_program = fluid.Program() with fluid.program_guard(train_program, startup_program): input_data = fluid.data(name='input_data', shape=[1,10], dtype='float32') label = fluid.data(name='label', shape=[1,1], dtype='int') emission= fluid.layers.fc(input=input_data, size=10, act="tanh") crf_cost = fluid.layers.linear_chain_crf( input=emission, label=label, param_attr=fluid.ParamAttr( name='crfw', learning_rate=0.01)) use_cuda = False place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace() exe = fluid.Executor(place) exe.run(startup_program) #define data, using LoDTensor a = fluid.create_lod_tensor(np.random.rand(12,10).astype('float32'), [[3,3,4,2]], place) b = fluid.create_lod_tensor(np.array([[1],[1],[2],[3],[1],[1],[1],[3],[1],[1],[1],[1]]),[[3,3,4,2]] , place) feed1 = {'input_data':a,'label':b} loss= exe.run(train_program,feed=feed1, fetch_list=[crf_cost]) print(loss) #define net structure, using padding train_program = fluid.Program() startup_program = fluid.Program() with fluid.program_guard(train_program, startup_program): input_data2 = fluid.data(name='input_data2', shape=[1,10,10], dtype='float32') label2 = fluid.data(name='label2', shape=[1,10,1], dtype='int') label_length = fluid.data(name='length', shape=[1,1], dtype='int') emission2= fluid.layers.fc(input=input_data2, size=10, act="tanh", num_flatten_dims=2) crf_cost2 = fluid.layers.linear_chain_crf( input=emission2, label=label2, length=label_length, param_attr=fluid.ParamAttr( name='crfw', learning_rate=0.01)) use_cuda = False place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace() exe = fluid.Executor(place) exe.run(startup_program) #define data, using padding cc=np.random.rand(4,10,10).astype('float32') dd=np.random.rand(4,10,1).astype('int64') ll=np.array([[3],[3],[4],[2]]) feed2 = {'input_data2':cc,'label2':dd,'length':ll} loss2= exe.run(train_program,feed=feed2, fetch_list=[crf_cost2]) print(loss2) #[array([[ 7.8902354], # [ 7.3602567], # [ 10.004011], # [ 5.86721 ]], dtype=float32)] #you can use find_var to get transition parameter. transition=np.array(fluid.global_scope().find_var('crfw').get_tensor()) print(transition)
lod_reset¶

paddle.fluid.layers.
lod_reset
(x, y=None, target_lod=None)[source] Set LoD of
x
to a new one specified byy
ortarget_lod
. Wheny
provided,y.lod
would be considered as target LoD first, otherwisey.data
would be considered as target LoD. Ify
is not provided, target LoD should be specified bytarget_lod
. If target LoD is specified byy.data
ortarget_lod
, only one level LoD is supported.* Example 1: Given a 1level LoDTensor x: x.lod = [[ 2, 3, 1 ]] x.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]] x.dims = [6, 1] target_lod: [4, 2] then we get a 1level LoDTensor: out.lod = [[4, 2]] out.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]] out.dims = [6, 1] * Example 2: Given a 1level LoDTensor x: x.lod = [[2, 3, 1]] x.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]] x.dims = [6, 1] y is a Tensor: y.data = [[2, 4]] y.dims = [1, 3] target_lod: This parameter does not work when y is not none. then we get a 1level LoDTensor: out.lod = [[2, 4]] out.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]] out.dims = [6, 1] * Example 3: Given a 1level LoDTensor x: x.lod = [[2, 3, 1]] x.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]] x.dims = [6, 1] y is a 2level LoDTensor: y.lod = [[2, 2], [2, 2, 1, 1]] y.data = [[1.1], [2.1], [3.1], [4.1], [5.1], [6.1]] y.dims = [6, 1] target_lod: This parameter does not work when y is not none. then we get a 2level LoDTensor: out.lod = [[2, 2], [2, 2, 1, 1]] out.data = [[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]] out.dims = [6, 1]
 Parameters
x (Variable) – Input variable which could be a Tensor or LoDTensor.
y (Variableoptional) – If provided, output’s LoD would be derived from
y
.target_lod (listtupleoptional) – One level LoD which should be considered as target LoD when
y
not provided.
 Returns
Output variable with LoD specified by this layer.
 Return type
Variable
 Raises
ValueError
– Ify
andtarget_lod
are both None.
Examples
import paddle.fluid as fluid import numpy # Graph Organizing x = fluid.data(name='x', shape=[6]) y = fluid.data(name='y', shape=[6], lod_level=1) output = fluid.layers.lod_reset(x=x, y=y) # Create an executor using CPU as an example place = fluid.CPUPlace() exe = fluid.Executor(place) # Execute x_tensor = fluid.core.LoDTensor() x_tensor.set(numpy.ones([6]).astype(numpy.float32), place) y_ndarray = numpy.ones([6]).astype(numpy.float32) y_lod = [[2, 2], [2, 2, 1, 1]] y_tensor = fluid.create_lod_tensor(y_ndarray, y_lod, place) res, = exe.run(fluid.default_main_program(), feed={'x':x_tensor, 'y':y_tensor}, fetch_list=[output], return_numpy=False) print(res) # Output Value: # lod: [[0, 2, 4], [0, 2, 4, 5, 6]] # dim: 6 # layout: NCHW # dtype: float # data: [1 1 1 1 1 1]
log¶

paddle.fluid.layers.
log
(x, name=None)[source] Calculates the natural log of the given input tensor, elementwise.
\[Out = \ln(x)\] Parameters
x (Variable) – Input LoDTensor or Tensor. Must be one of the following types: float32, float64.
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
The natural log of the input LoDTensor or Tensor computed elementwise.
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np # Graph Organizing x = fluid.layers.data(name="x", shape=[1], dtype="float32") res = fluid.layers.log(x) # Create an executor using CPU as an example exe = fluid.Executor(fluid.CPUPlace()) # Execute x_i = np.array([[1], [2]]).astype(np.float32) res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res]) print(res_val) # [[0.], [0.6931472]]
log_loss¶

paddle.fluid.layers.
log_loss
(input, label, epsilon=0.0001, name=None)[source] Negative Log Loss Layer
This layer accepts input predictions and target label and returns the negative log loss.
\[Out = label * \log{(input + \epsilon)}  (1  label) * \log{(1  input + \epsilon)}\] Parameters
input (Variablelist) – A 2D tensor with shape [N x 1], where N is the batch size. This input is a probability computed by the previous operator. Data type float32.
label (Variablelist) – The ground truth which is a 2D tensor with shape [N x 1], where N is the batch size. Data type float32.
epsilon (float, optional) – A small number for numerical stability. Default 1e4.
name (strNone) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.
 Returns
A 2D tensor with shape [N x 1], the negative log loss.
 Return type
Variable
Examples
import paddle.fluid as fluid label = fluid.data(name='label', shape=[1, 1], dtype='int64') prob = fluid.data(name='prob', shape=[1, 10], dtype='float32') cost = fluid.layers.log_loss(input=prob, label=label)
logical_and¶

paddle.fluid.layers.
logical_and
(x, y, out=None, name=None)[source] logical_and Operator
It operates elementwise on X and Y, and returns the Out. X, Y and Out are Ndim boolean LoDTensor or Tensor. Each element of Out is calculated by
\[Out = X \land Y\] Parameters
x (Variable) – Left hand operand of logical_and operator. Must be a LoDTensor or Tensor of type bool
y (Variable) – Right hand operand of logical_and operator. Must be a LoDTensor or Tensor of type bool
out (LoDTensor or Tensor) – The LoDTensor or Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
ndim bool LoDTensor or Tensor
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np # Graph organizing x = fluid.layers.data(name='x', shape=[2], dtype='bool') y = fluid.layers.data(name='y', shape=[2], dtype='bool') res = fluid.layers.logical_and(x=x, y=y) # The comment lists another available method. # res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0) # fluid.layers.logical_and(x=x, y=y, out=res) # Create an executor using CPU as an example exe = fluid.Executor(fluid.CPUPlace()) # Execute x_i = np.array([[1, 0], [0, 1]]).astype(np.bool) y_i = np.array([[1, 1], [0, 0]]).astype(np.bool) res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i, 'y':y_i}, fetch_list=[res]) print(res_val) # [[True, False], [False, False]]
logical_not¶

paddle.fluid.layers.
logical_not
(x, out=None, name=None)[source] logical_not Operator
It operates elementwise on X, and returns the Out. X and Out are Ndim boolean LoDTensor or Tensor. Each element of Out is calculated by
\[Out = \lnot X\] Parameters
x (Variable) – Operand of logical_not operator. Must be a LoDTensor or Tensor of type bool
out (LoDTensor/Tensor) – The LoDTensor/Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
ndim bool LoDTensor or Tensor
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np # Graph organizing x = fluid.layers.data(name='x', shape=[2], dtype='bool') res = fluid.layers.logical_not(x) # The comment lists another availble method. # res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0) # fluid.layers.logical_not(x, out=res) # Create an executor using CPU as an example exe = fluid.Executor(fluid.CPUPlace()) # Execute x_i = np.array([[1, 0]]).astype(np.bool) res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i}, fetch_list=[res]) print(res_val) # [[False, True]]
logical_or¶

paddle.fluid.layers.
logical_or
(x, y, out=None, name=None)[source] logical_or Operator
It operates elementwise on X and Y, and returns the Out. X, Y and Out are Ndim boolean LoDTensor or Tensor. Each element of Out is calculated by
\[Out = X \lor Y\] Parameters
x (Variable) – Left hand operand of logical_or operator. Must be a LoDTensor or Tensor of type bool
y (Variable) – Right hand operand of logical_or operator. Must be a LoDTensor or Tensor of type bool
out (LoDTensor or Tensor) – The LoDTensor or Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
ndim bool LoDTensor or Tensor
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np # Graph organizing x = fluid.layers.data(name='x', shape=[2], dtype='bool') y = fluid.layers.data(name='y', shape=[2], dtype='bool') res = fluid.layers.logical_or(x=x, y=y) # The comment lists another available method. # res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0) # fluid.layers.logical_or(x=x, y=y, out=res) # Create an executor using CPU as an example exe = fluid.Executor(fluid.CPUPlace()) # Execute x_i = np.array([[1, 0], [0, 1]]).astype(np.bool) y_i = np.array([[1, 1], [0, 0]]).astype(np.bool) res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i, 'y':y_i}, fetch_list=[res]) print(res_val) # [[True, True], [False, True]]
logical_xor¶

paddle.fluid.layers.
logical_xor
(x, y, out=None, name=None)[source] logical_xor Operator
It operates elementwise on X and Y, and returns the Out. X, Y and Out are Ndim boolean LoDTensor or Tensor. Each element of Out is calculated by
\[Out = (X \lor Y) \land \lnot (X \land Y)\] Parameters
x (Variable) – Left hand operand of logical_xor operator. Must be a LoDTensor or Tensor of type bool
y (Variable) – Right hand operand of logical_xor operator. Must be a LoDTensor or Tensor of type bool
out (LoDTensor or Tensor) – The LoDTensor or Tensor that specifies the output of the operator, which can be any Variable that has been created in the program. The default value is None, and a new Variable will be created to save the output.
name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
ndim bool LoDTensor or Tensor
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np # Graph organizing x = fluid.layers.data(name='x', shape=[2], dtype='bool') y = fluid.layers.data(name='y', shape=[2], dtype='bool') res = fluid.layers.logical_xor(x=x, y=y) # The comment lists another available method. # res = fluid.layers.fill_constant(shape=[2], dtype='bool', value=0) # fluid.layers.logical_xor(x=x, y=y, out=res) # Create an executor using CPU as an example exe = fluid.Executor(fluid.CPUPlace()) # Execute x_i = np.array([[1, 0], [0, 1]]).astype(np.bool) y_i = np.array([[1, 1], [0, 0]]).astype(np.bool) res_val, = exe.run(fluid.default_main_program(), feed={'x':x_i, 'y':y_i}, fetch_list=[res]) print(res_val) # [[False, True], [False, True]]
lrn¶

paddle.fluid.layers.
lrn
(input, n=5, k=1.0, alpha=0.0001, beta=0.75, name=None)[source] This operator implements the Local Response Normalization Layer. This layer performs a type of “lateral inhibition” by normalizing over local input regions. For more information, please refer to ImageNet Classification with Deep Convolutional Neural Networks
The formula is as follows:
\[Output(i, x, y) = Input(i, x, y) / \left(k + \alpha \sum\limits^{\min(C1, i + n/2)}_{j = \max(0, i  n/2)}(Input(j, x, y))^2\right)^{\beta}\]In the above equation:
\(n\) : The number of channels to sum over.
\(k\) : The offset (avoid being divided by 0).
\(\alpha\) : The scaling parameter.
\(\beta\) : The exponent parameter.
 Parameters
input (Variable) – Input feature, 4DTensor with the shape of [N,C,H,W], where N is the batch size, C is the input channel, H is Height, W is weight. The data type is float32. The rank of this tensor must be 4, otherwise it will raise ValueError.
n (int, optional) – The number of channels to sum over. Default: 5
k (float, optional) – An offset, positive. Default: 1.0
alpha (float, optional) – The scaling parameter, positive. Default:1e4
beta (float, optional) – The exponent, positive. Default:0.75
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
A tensor variable storing the transformation result with the same shape and data type as input.
 Return type
Variable
Examples:
import paddle.fluid as fluid data = fluid.data( name="data", shape=[None, 3, 112, 112], dtype="float32") lrn = fluid.layers.lrn(input=data) print(lrn.shape) # [1, 3, 112, 112] print(lrn.dtype) # float32
lstm¶

paddle.fluid.layers.
lstm
(input, init_h, init_c, max_len, hidden_size, num_layers, dropout_prob=0.0, is_bidirec=False, is_test=False, name=None, default_initializer=None, seed=1)[source]  Note:
This OP only supports running on GPU devices.
This OP implements LSTM operation  Hochreiter, S., & Schmidhuber, J. (1997) .
The implementation of this OP does not include diagonal/peephole connections. Please refer to Gers, F. A., & Schmidhuber, J. (2000) . If you need peephole connections, please use dynamic_lstm .
This OP computes each timestep as follows:
\[i_t = \sigma(W_{ix}x_{t} + W_{ih}h_{t1} + b_{x_i} + b_{h_i})\]\[f_t = \sigma(W_{fx}x_{t} + W_{fh}h_{t1} + b_{x_f} + b_{h_f})\]\[o_t = \sigma(W_{ox}x_{t} + W_{oh}h_{t1} + b_{x_o} + b_{h_o})\]\[\widetilde{c_t} = tanh(W_{cx}x_t + W_{ch}h_{t1} + b{x_c} + b_{h_c})\]\[c_t = f_t \odot c_{t1} + i_t \odot \widetilde{c_t}\]\[h_t = o_t \odot tanh(c_t)\]The symbolic meanings in the formula are as follows:
\(x_{t}\) represents the input at timestep \(t\)
\(h_{t}\) represents the hidden state at timestep \(t\)
\(h_{t1}, c_{t1}\) represent the hidden state and cell state at timestep \(t1\) , respectively
\(\widetilde{c_t}\) represents the candidate cell state
\(i_t\) , \(f_t\) and \(o_t\) represent input gate, forget gate, output gate, respectively
\(W\) represents weight (e.g., \(W_{ix}\) is the weight of a linear transformation of input \(x_{t}\) when calculating input gate \(i_t\) )
\(b\) represents bias (e.g., \(b_{i}\) is the bias of input gate)
\(\sigma\) represents nonlinear activation function for gate, default sigmoid
\(\odot\) represents the Hadamard product of a matrix, i.e. multiplying the elements of the same position for two matrices with the same dimension to get another matrix with the same dimension
 Parameters
input (api_guide_Variable_en) – LSTM input tensor, 3D Tensor of shape \([batch\_size, seq\_len, input\_dim]\) . Data type is float32 or float64
init_h (api_guide_Variable_en) – The initial hidden state of the LSTM, 3D Tensor of shape \([num\_layers, batch\_size, hidden\_size]\) . If is_bidirec = True, shape should be \([num\_layers*2, batch\_size, hidden\_size]\) . Data type is float32 or float64.
init_c (api_guide_Variable_en) – The initial cell state of the LSTM, 3D Tensor of shape \([num\_layers, batch\_size, hidden\_size]\) . If is_bidirec = True, shape should be \([num\_layers*2, batch\_size, hidden\_size]\) . Data type is float32 or float64.
max_len (int) – max length of LSTM. the first dim of input tensor CAN NOT greater than max_len.
hidden_size (int) – hidden size of the LSTM.
num_layers (int) – total layers number of the LSTM.
dropout_prob (float, optional) – dropout prob, dropout ONLY work between rnn layers, NOT between time steps There is NO dropout work on rnn output of the last RNN layers. Default: 0.0.
is_bidirec (bool, optional) – If it is bidirectional. Default: False.
is_test (bool, optional) – If it is in test phrase. Default: False.
name (str, optional) – A name for this layer. If set None, the layer will be named automatically. Default: None.
default_initializer (Initializer, optional) – Where use initializer to initialize the Weight If set None, defaule initializer will be used. Default: None.
seed (int, optional) – Seed for dropout in LSTM, If it’s 1, dropout will use random seed. Default: 1.
 Returns
Three tensors, rnn_out, last_h, last_c:
rnn_out is result of LSTM hidden, shape is \([seq\_len, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([seq\_len, batch\_size, hidden\_size*2]\)
last_h is the hidden state of the last step of LSTM shape is \([num\_layers, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([num\_layers*2, batch\_size, hidden\_size]\)
last_c(Tensor): the cell state of the last step of LSTM shape is \([num\_layers, batch\_size, hidden\_size]\) if is_bidirec set to True, shape will be \([num\_layers*2, batch\_size, hidden\_size]\)
 Return type
tuple ( api_guide_Variable_en , api_guide_Variable_en , api_guide_Variable_en )
Examples
import paddle.fluid as fluid import paddle.fluid.layers as layers emb_dim = 256 vocab_size = 10000 data = fluid.data(name='x', shape=[None, 100], dtype='int64') emb = fluid.embedding(input=data, size=[vocab_size, emb_dim], is_sparse=True) batch_size = 20 max_len = 100 dropout_prob = 0.2 input_size = 100 hidden_size = 150 num_layers = 1 init_h = layers.fill_constant( [num_layers, batch_size, hidden_size], 'float32', 0.0 ) init_c = layers.fill_constant( [num_layers, batch_size, hidden_size], 'float32', 0.0 ) rnn_out, last_h, last_c = layers.lstm( emb, init_h, init_c, max_len, hidden_size, num_layers, dropout_prob=dropout_prob) rnn_out.shape # (1, 100, 150) last_h.shape # (1, 20, 150) last_c.shape # (1, 20, 150)
lstm_unit¶

paddle.fluid.layers.
lstm_unit
(x_t, hidden_t_prev, cell_t_prev, forget_bias=0.0, param_attr=None, bias_attr=None, name=None)[source] LongShort Term Memory (LSTM) RNN cell. This operator performs LSTM calculations for one time step, whose implementation is based on calculations described in RECURRENT NEURAL NETWORK REGULARIZATION .
We add forget_bias to the biases of the forget gate in order to reduce the scale of forgetting. The formula is as follows:
\[ \begin{align}\begin{aligned}i_{t} & = \sigma(W_{x_{i}}x_{t} + W_{h_{i}}h_{t1} + b_{i})\\f_{t} & = \sigma(W_{x_{f}}x_{t} + W_{h_{f}}h_{t1} + b_{f} + forget\_bias)\\c_{t} & = f_{t}c_{t1} + i_{t} tanh (W_{x_{c}}x_{t} + W_{h_{c}}h_{t1} + b_{c})\\o_{t} & = \sigma(W_{x_{o}}x_{t} + W_{h_{o}}h_{t1} + b_{o})\\h_{t} & = o_{t} tanh (c_{t})\end{aligned}\end{align} \]\(x_{t}\) stands for
x_t
, corresponding to the input of current time step; \(h_{t1}\) and \(c_{t1}\) correspond tohidden_t_prev
andcell_t_prev
, representing the output of from previous time step. \(i_{t}, f_{t}, c_{t}, o_{t}, h_{t}\) are input gate, forget gate, cell, output gate and hidden calculation. Parameters
x_t (Variable) – A 2D Tensor representing the input of current time step. Its shape should be \([N, M]\) , where \(N\) stands for batch size, \(M\) for the feature size of input. The data type should be float32 or float64.
hidden_t_prev (Variable) – A 2D Tensor representing the hidden value from previous step. Its shape should be \([N, D]\) , where \(N\) stands for batch size, \(D\) for the hidden size. The data type should be same as
x_t
.cell_t_prev (Variable) – A 2D Tensor representing the cell value from previous step. It has the same shape and data type with
hidden_t_prev
.forget_bias (float, optional) – \(forget\_bias\) added to the biases of the forget gate. Default 0.
param_attr (ParamAttr, optional) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
bias_attr (ParamAttr, optional) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
The tuple contains two Tensor variables with the same shape and data type with
hidden_t_prev
, representing the hidden value and cell value which correspond to \(h_{t}\) and \(c_{t}\) in the formula. Return type
tuple
 Raises
ValueError
– Rank of x_t must be 2.ValueError
– Rank of hidden_t_prev must be 2.ValueError
– Rank of cell_t_prev must be 2.ValueError
– The 1st dimensions of x_t, hidden_t_prev and cell_t_prev must be the same.ValueError
– The 2nd dimensions of hidden_t_prev and cell_t_prev must be the same.
Examples
import paddle.fluid as fluid dict_dim, emb_dim, hidden_dim = 128, 64, 512 data = fluid.data(name='step_data', shape=[None], dtype='int64') x = fluid.embedding(input=data, size=[dict_dim, emb_dim]) pre_hidden = fluid.data( name='pre_hidden', shape=[None, hidden_dim], dtype='float32') pre_cell = fluid.data( name='pre_cell', shape=[None, hidden_dim], dtype='float32') hidden = fluid.layers.lstm_unit( x_t=x, hidden_t_prev=pre_hidden, cell_t_prev=pre_cell)
margin_rank_loss¶

paddle.fluid.layers.
margin_rank_loss
(label, left, right, margin=0.1, name=None)[source] Margin Ranking Loss Layer for ranking problem, which compares left score and right score passed in. The ranking loss can be defined as following equation:
\[rank\_loss = max(0, label * (left  right) + margin)\] Parameters
label (Variable) – Indicates whether the left is ranked higher than the right or not. Data type is float32.
left (Variable) – Ranking score for left. Data type float32.
right (Variable) – Ranking score for right. Data type float32.
margin (float) – Indicates the given margin.
name (strNone) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.
 Returns
The ranking loss.
 Return type
Variable
 Raises
ValueError
– Any of label, left, and right is not a Variable.
Examples
import paddle.fluid as fluid label = fluid.data(name="label", shape=[1, 1], dtype="float32") left = fluid.data(name="left", shape=[1, 1], dtype="float32") right = fluid.data(name="right", shape=[1, 1], dtype="float32") out = fluid.layers.margin_rank_loss(label, left, right)
matmul¶

paddle.fluid.layers.
matmul
(x, y, transpose_x=False, transpose_y=False, alpha=1.0, name=None)[source] Applies matrix multiplication to two tensors.
Currently, the input tensors’ rank can be any, but when the rank of any inputs is bigger than 3, this two inputs’ rank should be equal.
The actual behavior depends on the shapes of \(x\), \(y\) and the flag values of
transpose_x
,transpose_y
. Specifically:If a transpose flag is specified, the last two dimensions of the tensor are transposed. If the tensor is rank1 of shape \([D]\), then for \(x\) it is treated as \([1, D]\) in nontransposed form and as \([D, 1]\) in transposed form, whereas for \(y\) it is the opposite: It is treated as \([D, 1]\) in nontransposed form and as \([1, D]\) in transposed form.
After transpose, the two tensors are 2D or nD and matrix multiplication performs in the following way.
If both are 2D, they are multiplied like conventional matrices.
If either is nD, it is treated as a stack of matrices residing in the last two dimensions and a batched matrix multiply supporting broadcast applies on the two tensors.
Also note that if the raw tensor \(x\) or \(y\) is rank1 and nontransposed, the prepended or appended dimension \(1\) will be removed after matrix multiplication.
 Parameters
x (Variable) – The input variable which is a Tensor or LoDTensor.
y (Variable) – The input variable which is a Tensor or LoDTensor.
transpose_x (bool) – Whether to transpose \(x\) before multiplication.
transpose_y (bool) – Whether to transpose \(y\) before multiplication.
alpha (float) – The scale of output. Default 1.0.
name (stroptional) – A name for this layer(optional). If set None, the layer will be named automatically.
 Returns
The product Tensor (or LoDTensor) variable.
 Return type
Variable
Examples
# Examples to clarify shapes of the inputs and output # x: [B, ..., M, K], y: [B, ..., K, N] # fluid.layers.matmul(x, y) # out: [B, ..., M, N] # x: [B, M, K], y: [B, K, N] # fluid.layers.matmul(x, y) # out: [B, M, N] # x: [B, M, K], y: [K, N] # fluid.layers.matmul(x, y) # out: [B, M, N] # x: [M, K], y: [K, N] # fluid.layers.matmul(x, y) # out: [M, N] # x: [B, M, K], y: [K] # fluid.layers.matmul(x, y) # out: [B, M] # x: [K], y: [K] # fluid.layers.matmul(x, y) # out: [1] # x: [M], y: [N] # fluid.layers.matmul(x, y, True, True) # out: [M, N] import paddle.fluid as fluid import numpy # Graph Organizing x = fluid.data(name='x', shape=[2, 3], dtype='float32') y = fluid.data(name='y', shape=[3, 2], dtype='float32') output = fluid.layers.matmul(x, y, True, True) # Create an executor using CPU as an example exe = fluid.Executor(fluid.CPUPlace()) # Execute input_x = numpy.ones([2, 3]).astype(numpy.float32) input_y = numpy.ones([3, 2]).astype(numpy.float32) res, = exe.run(fluid.default_main_program(), feed={'x':input_x, 'y':input_y}, fetch_list=[output]) print(res) ''' Output Value: [[2. 2. 2.] [2. 2. 2.] [2. 2. 2.]] '''
maxout¶

paddle.fluid.layers.
maxout
(x, groups, name=None)[source] MaxOut Operator.
Assumed the input shape is (N, Ci, H, W). The output shape is (N, Co, H, W). Then \(Co = Ci / groups\) and the operator formula is as follows:
$$ y_{si+j} = max_{k} x_{gsi + sk + j} $$ $$ g = groups $$ $$ s = \frac{input.size}{num\_channels} $$ $$ 0 \le i < \frac{num\_channels}{groups} $$ $$ 0 \le j < s $$ $$ 0 \le k < groups $$
Please refer to Paper:  Maxout Networks: http://www.jmlr.org/proceedings/papers/v28/goodfellow13.pdf  Multidigit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks: https://arxiv.org/pdf/1312.6082v4.pdf
 Parameters
x (Variable) – (Tensor) The input tensor of maxout operator with data type of float32. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of feature
groups (INT) – (int),Specifies how many groups the input tensor will be splitin the channel dimension. And the number of output channel is the number of channels divided by groups
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
out(Variable): (Tensor) The output tensor of maxout operator.The data type is float32.The format of output tensor is also NCHW.Where N is batch size, C is the number of channels, H and W is the height and width of feature
 Return type
Variable
Examples
import paddle.fluid as fluid input = fluid.data( name='data', shape=[None, 256, 32, 32], dtype='float32') out = fluid.layers.maxout(input, groups=2)
mean¶

paddle.fluid.layers.
mean
(x, name=None)[source] Mean Operator calculates the mean of all elements in X.
 Parameters
x (Variable) – (Tensor) The input of mean op
name (basestringNone) – Name of the output.
 Returns
(Tensor) The output of mean op
 Return type
out(Variable)
Examples
import paddle.fluid as fluid import numpy # Graph Organizing input = fluid.data( name='data', shape=[2, 3], dtype='float32') output = fluid.layers.mean(input) # Create an executor using CPU as an example place = fluid.CPUPlace() exe = fluid.Executor(place) # Execute x_ndarray = numpy.ones([2, 3]).astype(numpy.float32) res, = exe.run(fluid.default_main_program(), feed={'data':x_ndarray}, fetch_list=[output]) print(res) ''' Output Value: [1.] '''
mean_iou¶

paddle.fluid.layers.
mean_iou
(input, label, num_classes)[source] Mean IntersectionOverUnion is a common evaluation metric for semantic image segmentation, which first computes the IOU for each semantic class and then computes the average over classes. IOU is defined as follows:
\[IOU = \frac{true\_positive}{(true\_positive + false\_positive + false\_negative)}.\]The predictions are accumulated in a confusion matrix and meanIOU is then calculated from it.
 Parameters
input (Variable) – A nD Tensor of prediction results for semantic labels with type int32 or int64.
label (Variable) – A Tensor of ground truth labels with type int32 or int64. Its shape should be the same as input.
num_classes (int32) – The possible number of labels.
 Returns
Three Variables.
mean_iou(Variable) : A 1D Tensor representing the mean intersectionoverunion with shape [1]. Data type is float32.
out_wrong(Variable) : A 1D Tensor with shape [num_classes]. Data type is int32. The wrong numbers of each class.
out_correct(Variable): A 1D Tensor with shape [num_classes]. Data type is int32. The correct numbers of each class.
Examples
import paddle.fluid as fluid iou_shape = [None, 32, 32] num_classes = 5 predict = fluid.data(name='predict', shape=iou_shape, dtype='int64') label = fluid.data(name='label', shape=iou_shape, dtype='int64') mean_iou, out_wrong, out_correct = fluid.layers.mean_iou(predict, label, num_classes)
merge_selected_rows¶

paddle.fluid.layers.
merge_selected_rows
(x, name=None)[source] MergeSelectedRows Operator.
MergeSelectedRows is used to merge the duplicated rows of the input. The output’s row has no duplicated, and it’s order is incremental.
Example: Input: X.rows is [0, 5, 5, 4, 19] X.height is 20 X.value is: [[1, 1] [2, 2] [3, 3] [4, 4] [6, 6]]
Output: Out.row is [0, 4, 5, 19] Out.height is 20 Out.value is: [[1, 1] [4, 4] [5, 5] [6, 6]]
 Parameters
x (Variable) – The input type is SelectedRows, and the selected rows may be duplicated
name (basestringNone) – Name of the output.
 Returns
The output type is SelectedRows, and the selected rows are not duplicated
 Return type
out(Variable)
Examples
import paddle.fluid as fluid import numpy place = fluid.CPUPlace() block = fluid.default_main_program().global_block() var = block.create_var(name="X2", dtype="float32", persistable=True, type=fluid.core.VarDesc.VarType.SELECTED_ROWS) y = fluid.layers.merge_selected_rows(var) z = fluid.layers.get_tensor_from_selected_rows(y) x_rows = [0, 2, 2, 4, 19] row_numel = 2 np_array = numpy.ones((len(x_rows), row_numel)).astype("float32") x = fluid.global_scope().var("X2").get_selected_rows() x.set_rows(x_rows) x.set_height(20) x_tensor = x.get_tensor() x_tensor.set(np_array, place) exe = fluid.Executor(place=place) result = exe.run(fluid.default_main_program(), fetch_list=[z]) print("x_rows: ", x_rows) print("np_array: ", np_array) print("result: ", result) ''' Output Values: ('x_rows: ', [0, 2, 2, 4, 19]) ('np_array: ', array([[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.]], dtype=float32)) ('result: ', [array([[1., 1.], [2., 2.], [1., 1.], [1., 1.]], dtype=float32)]) '''
mul¶

paddle.fluid.layers.
mul
(x, y, x_num_col_dims=1, y_num_col_dims=1, name=None)[source] Mul Operator. This operator is used to perform matrix multiplication for input $x$ and $y$. The equation is:
\[Out = x * y\]Both the input $x$ and $y$ can carry the LoD (Level of Details) information, or not. But the output only shares the LoD information with input $x$.
 Parameters
x (Variable) – The first input Tensor/LoDTensor of mul_op.
y (Variable) – The second input Tensor/LoDTensor of mul_op.
x_num_col_dims (int, optional) – The mul_op can take tensors with more than two dimensions as its inputs. If the input $x$ is a tensor with more than two dimensions, $x$ will be flattened into a twodimensional matrix first. The flattening rule is: the first num_col_dims will be flattened to form the first dimension of the final matrix (the height of the matrix), and the rest rank(x)  num_col_dims dimensions are flattened to form the second dimension of the final matrix (the width of the matrix). As a result, height of the flattened matrix is equal to the product of $x$’s first x_num_col_dims dimensions’ sizes, and width of the flattened matrix is equal to the product of $x$’s last rank(x)  num_col_dims dimensions’ size. For example, suppose $x$ is a 6dimensional tensor with the shape [2, 3, 4, 5, 6], and x_num_col_dims = 3. Thus, the flattened matrix will have a shape [2 x 3 x 4, 5 x 6] = [24, 30]. Default is 1.
y_num_col_dims (int, optional) – The mul_op can take tensors with more than two dimensions as its inputs. If the input $y$ is a tensor with more than two dimensions, $y$ will be flattened into a twodimensional matrix first. The attribute y_num_col_dims determines how $y$ is flattened. See comments of x_num_col_dims for more details. Default is 1.
name (str, optional) – Name of the output. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name. Default is None.
 Returns
The output Tensor/LoDTensor of mul op.
 Return type
Variable(Tensor/LoDTensor)
Examples
import paddle.fluid as fluid dataX = fluid.layers.data(name="dataX", append_batch_size = False, shape=[2, 5], dtype="float32") dataY = fluid.layers.data(name="dataY", append_batch_size = False, shape=[5, 3], dtype="float32") output = fluid.layers.mul(dataX, dataY, x_num_col_dims = 1, y_num_col_dims = 1)
multiplex¶

paddle.fluid.layers.
multiplex
(inputs, index)[source] Based on the given index parameter, the OP selects a specific row from each input Tensor to construct the output Tensor.
If the input of this OP contains \(m\) Tensors, where \(I_{i}\) means the ith input Tensor, \(i\) between \([0,m)\) .
And \(O\) means the output, where \(O[i]\) means the ith row of the output, then the output satisfies that \(O[i] = I_{index[i]}[i]\) .
For Example:
Given: inputs = [[[0,0,3,4], [0,1,3,4], [0,2,4,4], [0,3,3,4]], [[1,0,3,4], [1,1,7,8], [1,2,4,2], [1,3,3,4]], [[2,0,3,4], [2,1,7,8], [2,2,4,2], [2,3,3,4]], [[3,0,3,4], [3,1,7,8], [3,2,4,2], [3,3,3,4]]] index = [[3],[0],[1],[2]] out = [[3,0,3,4], # out[0] = inputs[index[0]][0] = inputs[3][0] = [3,0,3,4] [0,1,3,4], # out[1] = inputs[index[1]][1] = inputs[0][1] = [0,1,3,4] [1,2,4,2], # out[2] = inputs[index[2]][2] = inputs[1][2] = [1,2,4,2] [2,3,3,4]] # out[3] = inputs[index[3]][3] = inputs[2][3] = [2,3,3,4]
 Parameters
inputs (list) – The input Tensor list. The list elements are ND Tensors of data types float32, float64, int32, int64. All input Tensor shapes should be the same and rank must be at least 2.
index (Variable) – Used to select some rows in the input Tensor to construct an index of the output Tensor. It is a 2D Tensor with data type int32 or int64 and shape [M, 1], where M is the number of input Tensors.
 Returns
Output of multiplex OP, with data type being float32, float64, int32, int64.
 Return type
Variable(Tensor)
Examples
import paddle.fluid as fluid import numpy as np x1 = fluid.data(name='x1', shape=[None, 2], dtype='float32') x2 = fluid.data(name='x2', shape=[None, 2], dtype='float32') index = fluid.data(name='index', shape=[None, 1], dtype='int32') out = fluid.layers.multiplex(inputs=[x1, x2], index=index) exe = fluid.Executor(fluid.CPUPlace()) exe.run(fluid.default_startup_program()) img1 = np.array([[1, 2], [3, 4]]).astype(np.float32) img2 = np.array([[5, 6], [7, 8]]).astype(np.float32) index = np.array([[1], [0]]).astype(np.int32) res = exe.run(fluid.default_main_program(), feed={'x1':img1, 'x2':img2, 'index':index}, fetch_list=[out]) print(res) # [array([[5., 6.], [3., 4.]], dtype=float32)]
nce¶

paddle.fluid.layers.
nce
(input, label, num_total_classes, sample_weight=None, param_attr=None, bias_attr=None, num_neg_samples=None, name=None, sampler='uniform', custom_dist=None, seed=0, is_sparse=False)[source] Compute and return the noisecontrastive estimation training loss. See Noisecontrastive estimation: A new estimation principle for unnormalized statistical models. By default this operator uses a uniform distribution for sampling.
 Parameters
input (Variable) – Input variable, 2D tensor with shape [batch_size, dim], and data type is float32 or float64.
label (Variable) – Input label, 2D tensor with shape [batch_size, num_true_class], and data type is int64.
num_total_classes (int) – Total number of classes in all samples.
sample_weight (VariableNone) – A Variable of shape [batch_size, 1] storing a weight for each sample. The default weight for each sample is 1.0.
param_attr (ParamAttrNone) – To specify the weight parameter attribute. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
bias_attr (ParamAttrNone) – To specify the bias parameter attribute. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .
num_neg_samples (int) – The number of negative classes. The default value is 10.
name (strNone) – For detailed information, please refer to api_guide_Name . Usually name is no need to set and None by default.
sampler (str, optional) – The sampler used to sample class from negtive classes. It can be ‘uniform’, ‘log_uniform’ or ‘custom_dist’. default: ‘uniform’.
custom_dist (nd.arrayNone) – A numpy ndarray with size=num_total_classes. It is used when sampler is set to ‘custom_dist’. custom_dist[i] is the probsbility of ith class to be sampled. default: None.
seed (int, optional) – The seed used in sampler. Default 0, means no random seed.
is_sparse (bool, optional) – The flag indicating whether to use sparse update, the weight@GRAD and bias@GRAD will be changed to SelectedRows. Default False.
 Returns
The output nce loss.
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np window_size = 5 words = [] for i in xrange(window_size): words.append(fluid.data( name='word_{0}'.format(i), shape=[1, 1], dtype='int64')) dict_size = 10000 label_word = int(window_size / 2) + 1 embs = [] for i in xrange(window_size): if i == label_word: continue emb = fluid.layers.embedding(input=words[i], size=[dict_size, 32], param_attr='embed', is_sparse=True) embs.append(emb) embs = fluid.layers.concat(input=embs, axis=1) loss = fluid.layers.nce(input=embs, label=words[label_word], num_total_classes=dict_size, param_attr='nce.w_0', bias_attr='nce.b_0') #or use custom distribution dist = np.array([0.05,0.5,0.1,0.3,0.05]) loss = fluid.layers.nce(input=embs, label=words[label_word], num_total_classes=5, param_attr='nce.w_1', bias_attr='nce.b_1', num_neg_samples=3, sampler="custom_dist", custom_dist=dist)
npair_loss¶

paddle.fluid.layers.
npair_loss
(anchor, positive, labels, l2_reg=0.002)[source] Npair Loss Layer
Read Improved Deep Metric Learning with Multi class N pair Loss Objective .
Npair loss requires paired data. Npair loss has two parts: the first part is L2 regularizer on the embedding vector; the second part is cross entropy loss which takes the similarity matrix of anchor and positive as logits.
 Parameters
anchor (Variable) – embedding vector for the anchor image. shape=[batch_size, embedding_dims], the data type is float32 or float64.
positive (Variable) – embedding vector for the positive image. shape=[batch_size, embedding_dims], the data type is float32 or float64.
labels (Variable) – 1D tensor. shape=[batch_size], the data type is float32 or float64 or int64.
l2_reg (float32) – L2 regularization term on embedding vector, default: 0.002.
 Returns
A Variable holding Tensor representing the npair loss, the data type is the same as anchor, the shape is [1].
Examples
import paddle.fluid as fluid anchor = fluid.data( name = 'anchor', shape = [18, 6], dtype = 'float32') positive = fluid.data( name = 'positive', shape = [18, 6], dtype = 'float32') labels = fluid.data( name = 'labels', shape = [18], dtype = 'float32') npair_loss = fluid.layers.npair_loss(anchor, positive, labels, l2_reg = 0.002)
one_hot¶

paddle.fluid.layers.
one_hot
(input, depth, allow_out_of_range=False)[source] WARING: This OP requires the last dimension of Tensor shape must be equal to 1. This OP will be deprecated in a future release. It is recommended to use fluid. api_fluid_one_hot .
The operator converts each id in the input to an onehot vector with a
depth
length. The value in the vector dimension corresponding to the id is 1, and the value in the remaining dimension is 0.The shape of output Tensor or LoDTensor is generated by adding
depth
dimension behind the last dimension of the input shape.Example 1 (allow_out_of_range=False): input: X.shape = [4, 1] X.data = [[1], [1], [3], [0]] depth = 4 output: Out.shape = [4, 4] Out.data = [[0., 1., 0., 0.], [0., 1., 0., 0.], [0., 0., 0., 1.], [1., 0., 0., 0.]] Example 2 (allow_out_of_range=True): input: X.shape = [4, 1] X.data = [[1], [1], [5], [0]] depth = 4 allow_out_of_range = True output: Out.shape = [4, 4] Out.data = [[0., 1., 0., 0.], [0., 1., 0., 0.], [0., 0., 0., 0.], # This id is 5, which goes beyond depth, so set it allzeros data. [1., 0., 0., 0.]] Example 3 (allow_out_of_range=False): input: X.shape = [4, 1] X.data = [[1], [1], [5], [0]] depth = 4 allow_out_of_range = False output: Throw an exception for Illegal value The second dimension in X is 5, which is greater than depth. Allow_out_of_range =False means that does not allow the word id to exceed depth, so it throws an exception.
 Parameters
input (Variable) – Tensor or LoDTensor with shape \([N_1, N_2, ..., N_k, 1]\) , which contains at least one dimension and the last dimension must be 1. The data type is int32 or int64.
depth (scalar) – An integer defining the
depth
of the one hot dimension. If input is word id, depth is generally the dictionary size.allow_out_of_range (bool) – A bool value indicating whether the input indices could be out of range \([0, depth)\) . When input indices are out of range, exceptions
Illegal value
is raised ifallow_out_of_range
is False, or zerofilling representations is created if it is set True. Default: False.
 Returns
The onehot representations of input. A Tensor or LoDTensor with type float32.
 Return type
Variable
Examples
import paddle.fluid as fluid # Correspond to the first example above, where label.shape is [4, 1] and one_hot_label.shape is [4, 4]. label = fluid.data(name="label", shape=[4, 1], dtype="int64") one_hot_label = fluid.layers.one_hot(input=label, depth=4)
pad¶

paddle.fluid.layers.
pad
(x, paddings, pad_value=0.0, name=None)[source] This op will pad a tensor with a constant value given by
pad_value
, and the padded shape is specified bypaddings
.Specifically, the number of values padded before the elements of
x
in dimensioni
is indicated bypaddings[2*i]
, and the number of values padded after the elements ofx
in dimensioni
is indicated bypaddings[2*i+1]
.See below for an example.
Given: x = [[1, 2], [3, 4]] paddings = [0, 1, 1, 2] pad_value = 0 Return: out = [[0, 1, 2, 0, 0] [0, 3, 4, 0, 0] [0, 0, 0, 0, 0]]
 Parameters
x (Variable) – Tensor, data type is float32.
paddings (list) – A list of integers. Its elements specify the padded width before and after each dimension in turn. The length of
paddings
must be equal to \(rank(x) \times 2\).pad_value (float) – The constant value used to pad.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
The padded tensor, with the same data type and rank as
x
 Return Type:
Variable
Examples
# x is a rank 2 tensor variable with shape [100, 224]. # out will be a tensor of shape [101, 227] import paddle.fluid as fluid x = fluid.data(name='data', shape=[100, 224], dtype='float32') out = fluid.layers.pad( x=x, paddings=[0, 1, 1, 2], pad_value=0.)
pad2d¶

paddle.fluid.layers.
pad2d
(input, paddings=[0, 0, 0, 0], mode='constant', pad_value=0.0, data_format='NCHW', name=None)[source] Pad 2d images accordding to ‘paddings’ and ‘mode’. If mode is ‘reflect’, paddings[0] and paddings[1] must be no greater than height1. And the width dimension has the same condition.
 Parameters
input (Variable) – The input image with [N, C, H, W] format or [N, H, W, C] format, which is a 4D Tensor with data type float32.
paddings (Variable  List[int32]) – The padding size. If padding is a List, it must contain four integers, (padding_top, padding_bottom, padding_left, padding_right). Otherwise, it is a 1D Tensor with shape [4]. Data type is int32. Default is [0, 0, 0, 0].
mode (str) – Three modes: ‘constant’ (default), ‘reflect’, ‘edge’ . When in ‘constant’ mode, this op uses a constant value to pad the input tensor. When in ‘reflect’ mode, uses reflection of the input boundaries to pad the input tensor. When in ‘edge’ mode, uses input boundaries to pad the input tensor. Default is ‘constant’
pad_value (float32) – The value to fill the padded areas in ‘constant’ mode . Default is 0.0
data_format (str) – An string from: “NHWC”, “NCHW”. Specify the data format of the input data. Default is “NCHW”
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
Returns: a 4D Tensor padded accordding to paddings and mode and data type is same as input.
Return Type: Variable
Examples
Given that X is a channel of image from input: X = [[1, 2, 3], [4, 5, 6]] Case 0: paddings = [0, 1, 2, 3], mode = 'constant' pad_value = 0 Out = [[0, 0, 1, 2, 3, 0, 0, 0] [0, 0, 4, 5, 6, 0, 0, 0] [0, 0, 0, 0, 0, 0, 0, 0]] Case 1: paddings = [0, 1, 2, 1], mode = 'reflect' Out = [[3, 2, 1, 2, 3, 2] [6, 5, 4, 5, 6, 5] [3, 2, 1, 2, 3, 2]] Case 2: paddings = [0, 1, 2, 1], mode = 'edge' Out = [[1, 1, 1, 2, 3, 3] [4, 4, 4, 5, 6, 6] [4, 4, 4, 5, 6, 6]]
 Code Examples:
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32') result = fluid.layers.pad2d(input=data, paddings=[1, 2, 3, 4], mode='reflect')
pad_constant_like¶

paddle.fluid.layers.
pad_constant_like
(x, y, pad_value=0.0, name=None)[source] Pad
y
withpad_value
, the number of values padded to the edges of each axis is specified by the difference of the shape ofx
andy
. ((0, shape_x_0  shape_y_0), … (0, shape_x_n  shape_y_n)) specify padding widths for each axis. The input should be a kD tensor(k > 0 and k < 7).See below for an example.
Given: X = [[[[ 0, 1, 2], [ 3, 4, 5]], [[ 6, 7, 8], [ 9, 10, 11]], [[12, 13, 14], [15, 16, 17]]], [[[18, 19, 20], [21, 22, 23]], [[24, 25, 26], [27, 28, 29]], [[30, 31, 32], [33, 34, 35]]]] X.shape = (2, 3, 2, 3) Y = [[[[35, 36, 37]], [[38, 39, 40]], [[41, 42, 43]]]] Y.shape = (1, 3, 1, 3) And pad_value = 1, Return: Out = [[[[35, 36, 37], [1, 1, 1]], [[38, 39, 40], [1, 1, 1]], [[41, 42, 43], [1, 1, 1]]], [[[1, 1, 1], [1, 1, 1]], [[1, 1, 1], [1, 1, 1]], [[1, 1, 1], [1, 1, 1]]]] Out.shape = (2, 3, 2, 3)
 Parameters
x (Variable) – Tensor, its shape spicifies the shape of output.
y (Variable) – Tensor, its rank is the same with
x
, and for each dimension \(i\) , \(y\_shape[i] <= x\_shape[i]\) . The data type can be float32 or float64.pad_value (float) – The constant value used to pad.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
The padded tensor, with the same shape as
x
and the same data type asy
 Return Type:
Variable
Examples
# x is a rank 4 tensor variable, x.shape = (2, 3, 2, 3) # y is a rank 4 tensor variable, y.shape = (1, 3, 1, 3) import paddle.fluid as fluid x = fluid.data(name='x', shape=[2,3,2,3], dtype='float32') y = fluid.data(name='y', shape=[1,3,1,3], dtype='float32') out = fluid.layers.pad_constant_like(x=x, y=y, pad_value=0.) # out is a rank 4 tensor variable, and out.shape = [2, 3 ,2 , 3]
pixel_shuffle¶

paddle.fluid.layers.
pixel_shuffle
(x, upscale_factor)[source] This op rearranges elements in a tensor of shape [N, C, H, W] to a tensor of shape [N, C/r**2, H*r, W*r]. This is useful for implementing efficient subpixel convolution with a stride of 1/r. Please refer to the paper: RealTime Single Image and Video SuperResolution Using an Efficient SubPixel Convolutional Neural Network . by Shi et. al (2016) for more details.
 Parameters
x (Variable) – 4D tensor, the data type should be float32 or float64.
upscale_factor (int) – factor to increase spatial resolution.
 Returns
Reshaped tensor according to the new dimension.
 Return type
Out(Variable)
 Raises
ValueError
– If the square of upscale_factor cannot divide the channels of input.
Examples
# declarative mode import paddle.fluid as fluid import numpy as np input = fluid.data(name="input", shape=[2,9,4,4]) output = fluid.layers.pixel_shuffle(x=input, upscale_factor=3) place = fluid.CPUPlace() exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) input_data = np.random.rand(2,9,4,4).astype("float32") output_data = exe.run(fluid.default_main_program(), feed={"input":input_data}, fetch_list=[output], return_numpy=True) # print(output.shape) # (2L, 1L, 12L, 12L)
pool2d¶

paddle.fluid.layers.
pool2d
(input, pool_size=1, pool_type='max', pool_stride=1, pool_padding=0, global_pooling=False, use_cudnn=True, ceil_mode=False, name=None, exclusive=True, data_format='NCHW')[source] This operation calculates the pooling output based on the input, pooling_type and pool_size, pool_stride, pool_padding parameters. Input(X) and Output(Out) are in NCHW or NHWC format, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. Parameters(pool_size, pool_stride, pool_padding) hold two integer elements. These two elements represent height and width, respectively. The input(X) size and output(Out) size may be different.
Example:
Input:
X shape: \((N, C, H_{in}, W_{in})\)
Output:
Out shape: \((N, C, H_{out}, W_{out})\)
For pool_padding = “SAME”: $$ H_{out} = \frac{(H_{in} + strides[0]  1)}{strides[0]} $$ $$ W_{out} = \frac{(W_{in} + strides[1]  1)}{strides[1]} $$
For pool_padding = “VALID”: $$ H_{out} = \frac{(H_{in}  ksize[0] + strides[0])}{strides[0]} $$ $$ W_{out} = \frac{(W_{in}  ksize[1] + strides[1])}{strides[1]} $$
For ceil_mode = false: $$ H_{out} = \frac{(H_{in}  ksize[0] + pad_height_top + pad_height_bottom}{strides[0]} + 1 $$ $$ W_{out} = \frac{(W_{in}  ksize[1] + pad_width_left + pad_width_right}{strides[1]} + 1 $$
For ceil_mode = true: $$ H_{out} = \frac{(H_{in}  ksize[0] + pad_height_top + pad_height_bottom + strides[0]  1)}{strides[0]} + 1 $$ $$ W_{out} = \frac{(W_{in}  ksize[1] + pad_width_left + pad_width_right + strides[1]  1)}{strides[1]} + 1 $$
For exclusive = false: $$ hstart = i * strides[0]  pad_height_top $$ $$ hend = hstart + ksize[0] $$ $$ wstart = j * strides[1]  pad_width_left $$ $$ wend = wstart + ksize[1] $$ $$ Output(i ,j) = \frac{sum(Input[hstart:hend, wstart:wend])}{ksize[0] * ksize[1]} $$
For exclusive = true: $$ hstart = max(0, i * strides[0]  pad_height_top) $$ $$ hend = min(H, hstart + ksize[0]) $$ $$ wstart = max(0, j * strides[1]  pad_width_left) $$ $$ wend = min(W, wstart + ksize[1]) $$ $$ Output(i ,j) = \frac{sum(Input[hstart:hend, wstart:wend])}{(hend  hstart) * (wend  wstart)} $$
 Parameters
input (Variable) – The input tensor of pooling operator which is a 4D tensor with shape [N, C, H, W]. The format of input tensor is “NCHW” or “NHWC”, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature. The data type if float32 or float64.
pool_size (intlisttuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain two integers, (pool_size_Height, pool_size_Width). Otherwise, the pool kernel size will be a square of an int.
pool_type – (string), pooling type, can be “max” for maxpooling and “avg” for averagepooling
pool_stride (intlisttuple) – The pool stride size. If pool stride size is a tuple or list, it must contain two integers, (pool_stride_Height, pool_stride_Width). Otherwise, the pool stride size will be a square of an int.
pool_padding (stringintlisttuple) – The pool padding. If pool_padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If pool padding size is a tuple or list, it could be in three forms: [pad_height, pad_width] or [pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCHW”, pool_padding can be in the form [[0,0], [0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NHWC”, pool_padding can be in the form [[0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]. Otherwise, the pool padding size will be a square of an int.
global_pooling (bool) – (bool) Whether to use the global pooling. If global_pooling = true, kernel size and paddings will be ignored. Default False
use_cudnn (bool) – (bool) Only used in cudnn kernel, need install cudnn. Default False
ceil_mode (bool) – (bool) Whether to use the ceil function to calculate output height and width. False is the default. If it is set to False, the floor function will be used. Default False
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
exclusive (bool) – Whether to exclude padding points in average pooling mode, default is true.
data_format (string) – The data format of the input and output data. An optional string from: “NCHW”, “NDHW”. The default is “NCHW”. When it is “NCHW”, the data is stored in the order of: [batch_size, input_channels, input_height, input_width].
 Returns
The output tensor of pooling result. The data type is same as input tensor.
 Return type
Variable
 Raises
ValueError
– If pool_type is not “max” nor “avg”ValueError
– If global_pooling is False and pool_size is 1ValueError
– If use_cudnn is not a bool value.
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32') # max pool2d pool2d = fluid.layers.pool2d( input = data, pool_size = 2, pool_type = "max", pool_stride = 1, global_pooling=False) # average pool2d pool2d = fluid.layers.pool2d( input = data, pool_size = 2, pool_type = "avg", pool_stride = 1, global_pooling=False) # global average pool2d pool2d = fluid.layers.pool2d( input = data, pool_size = 2, pool_type = "avg", pool_stride = 1, global_pooling=True) # Attr(pool_padding) is a list with 4 elements, Attr(data_format) is "NCHW". out_1 = fluid.layers.pool2d( input = data, pool_size = 3, pool_type = "avg", pool_stride = 1, pool_padding = [1, 2, 1, 0], data_format = "NCHW") # Attr(pool_padding) is a string, Attr(data_format) is "NCHW". out_2 = fluid.layers.pool2d( input = data, pool_size = 3, pool_type = "avg", pool_stride = 1, pool_padding = "VALID", data_format = "NCHW")
pool3d¶

paddle.fluid.layers.
pool3d
(input, pool_size=1, pool_type='max', pool_stride=1, pool_padding=0, global_pooling=False, use_cudnn=True, ceil_mode=False, name=None, exclusive=True, data_format='NCDHW')[source] This operation calculates the output based on the input, pooling_type, pool_size, pool_stride, and pool_padding parameters. Input(X) and output(Out) are in NCDHW or NDHWC format, where N is batch size, C is the number of channels, and D, H and W are the depth, height and width of the feature, respectively. Parameters(pool_size, pool_stride, pool_padding) hold three integer elements. These three elements represent depth, height and width, respectively. The input(X) size and output(Out) size may be different.
Example: Input: X shape: \((N, C, D_{in}, H_{in}, W_{in})\) Output: Out shape: \((N, C, D_{out}, H_{out}, W_{out})\)
For pool_padding = “SAME”: $$ D_{out} = \frac{(D_{in} + strides[0]  1)}{strides[0]} $$ $$ H_{out} = \frac{(H_{in} + strides[1]  1)}{strides[1]} $$ $$ W_{out} = \frac{(W_{in} + strides[2]  1)}{strides[2]} $$
For pool_padding = “VALID”: $$ D_{out} = \frac{(D_{in}  ksize[0] + strides[0])}{strides[0]} $$ $$ H_{out} = \frac{(H_{in}  ksize[1] + strides[1])}{strides[1]} $$ $$ W_{out} = \frac{(W_{in}  ksize[2] + strides[2])}{strides[2]} $$
For ceil_mode = false: $$ D_{out} = \frac{(D_{in}  ksize[0] + pad_depth_front + pad_depth_back)}{strides[0]} + 1 $$ $$ H_{out} = \frac{(H_{in}  ksize[1] + pad_height_top + pad_height_bottom)}{strides[1]} + 1 $$ $$ W_{out} = \frac{(W_{in}  ksize[2] + pad_width_left + pad_width_right)}{strides[2]} + 1 $$ For ceil_mode = true: $$ D_{out} = \frac{(D_{in}  ksize[0] + pad_depth_front + pad_depth_back + strides[0] 1)}{strides[0]} + 1 $$ $$ H_{out} = \frac{(H_{in}  ksize[1] + pad_height_top + pad_height_bottom + strides[1] 1)}{strides[1]} + 1 $$ $$ W_{out} = \frac{(W_{in}  ksize[2] + pad_width_left + pad_width_right + strides[2] 1)}{strides[2]} + 1 $$
For exclusive = false: $$ dstart = i * strides[0]  pad_depth_front $$ $$ dend = dstart + ksize[0] $$ $$ hstart = j * strides[1]  pad_height_top $$ $$ hend = hstart + ksize[1] $$ $$ wstart = k * strides[2]  pad_width_left $$ $$ wend = wstart + ksize[2] $$ $$ Output(i ,j, k) = \frac{sum(Input[dstart:dend, hstart:hend, wstart:wend])}{ksize[0] * ksize[1] * ksize[2]} $$
For exclusive = true: $$ dstart = max(0, i * strides[0]  pad_depth_front) $$ $$ dend = min(D, dstart + ksize[0]) $$ $$ hstart = max(0, j * strides[1]  pad_height_top) $$ $$ hend = min(H, hstart + ksize[1]) $$ $$ wstart = max(0, k * strides[2]  pad_width_left) $$ $$ wend = min(W, wstart + ksize[2]) $$ $$ Output(i ,j, k) = \frac{sum(Input[dstart:dend, hstart:hend, wstart:wend])}{(dend  dstart) * (hend  hstart) * (wend  wstart)} $$
 Parameters
input (Variable) – The input tensor of pooling operator, which is a 5D tensor with shape [N, C, D, H, W]. The format of input tensor is “NCDHW” or “NDHWC”, where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.
pool_size (intlisttuple) – The pool kernel size. If pool kernel size is a tuple or list, it must contain three integers, (pool_size_Depth, pool_size_Height, pool_size_Width). Otherwise, the pool kernel size will be the cube of an int.
pool_type (string) – (string) Pooling type, can be “max” for maxpooling and “avg” for averagepooling
pool_stride (stringintlisttuple)) – The pool padding. If pool_padding is a string, either ‘VALID’ or ‘SAME’ which is the padding algorithm. If pool stride size is a tuple or list, it must contain three integers, [stride_Depth, stride_Height, stride_Width]. Otherwise, the pool stride size will be a cube of an int.
pool_padding (intlisttuple) – The pool padding size. If pool padding size is a tuple or list, it could be in three forms: [pad_depth, pad_height, pad_width] or [pad_depth_front, pad_depth_back, pad_height_top, pad_height_bottom, pad_width_left, pad_width_right], and when data_format is “NCDHW”, pool_padding can be in the form [[0,0], [0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]. when data_format is “NDHWC”, pool_padding can be in the form [[0,0], [pad_depth_front, pad_depth_back], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]].
global_pooling (bool) – (bool) Whether to use the global pooling. If global_pooling = true, kernel size and paddings will be ignored. Default False
use_cudnn (bool) – (bool) Only used in cudnn kernel, need install cudnn. Default False
ceil_mode (bool) – (bool) Whether to use the ceil function to calculate output height and width. False is the default. If it is set to False, the floor function will be used. Default False
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
exclusive (bool) – Whether to exclude padding points in average pooling mode, default is true.
data_format (string) – The data format of the input and output data. An optional string from: “NCDHW”, “NDHWC”. The default is “NCDHW”. When it is “NCDHW”, the data is stored in the order of: [batch_size, input_channels, input_depth, input_height, input_width].
 Returns
The output tensor of pooling result. The data type is same as input tensor.
 Return type
Variable
Examples
import paddle.fluid as fluid data = fluid.data(name='data', shape=[None, 3, 32, 32, 32], dtype='float32') # max pool3d pool3d = fluid.layers.pool3d( input = data, pool_size = 2, pool_type = "max", pool_stride = 1, global_pooling=False) # average pool3d pool3d = fluid.layers.pool3d( input = data, pool_size = 2, pool_type = "avg", pool_stride = 1, global_pooling=False) # global average pool3d pool3d = fluid.layers.pool3d( input = data, pool_size = 2, pool_type = "avg", pool_stride = 1, global_pooling=True) # example 1: # Attr(pool_padding) is a list with 6 elements, Attr(data_format) is "NCDHW". out_1 = fluid.layers.pool3d( input = data, pool_size = 2, pool_type = "avg", pool_stride = 1, pool_padding = [1, 2, 1, 0, 1, 2], global_pooling = False, data_format = "NCDHW") # example 2: # Attr(pool_padding) is a string, Attr(data_format) is "NCDHW". out_2 = fluid.layers.pool3d( input = data, pool_size = 3, pool_type = "avg", pool_stride = 1, pool_padding = "VALID", global_pooling = False, data_format = "NCDHW")
pow¶

paddle.fluid.layers.
pow
(x, factor=1.0, name=None)[source] This is Pow Activation Operator.
\(out = x^{factor}\)
 Parameters
x (Variable) – A
Tensor
orLoDTensor
. The data type isfloat32
orfloat64
.factor (float32Variable, optional) – A scalar with type
float32
or aTensor
with shape [1] and typefloat32
. The exponential factor of Pow. Default 1.0.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
A
Tensor
orLoDTensor
. The data type is same asx
. Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.data(name="x", shape=[32,32], dtype="float32") # example 1: argument factor is float y_1 = fluid.layers.pow(x, factor=2.0) # y_1 is x^{2.0} # example 2: argument factor is Variable factor_tensor = fluid.layers.fill_constant([1], "float32", 3.0) y_2 = fluid.layers.pow(x, factor=factor_tensor) # y_2 is x^{3.0}
prelu¶

paddle.fluid.layers.
prelu
(x, mode, param_attr=None, name=None)[source] Equation:
\[y = \max(0, x) + \alpha * \min(0, x)\]There are three modes for the activation:
all: All elements share same alpha. channel: Elements in same channel share same alpha. element: All elements do not share alpha. Each element has its own alpha.
 Parameters
x (Variable) – The input Tensor or LoDTensor with data type float32.
mode (str) – The mode for weight sharing.
param_attr (ParamAttrNone) – The parameter attribute for the learnable weight (alpha), it can be create by ParamAttr. None by default. For detailed information, please refer to ParamAttr.
name (strNone) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
output(Variable): The tensor or LoDTensor with the same shape as input. The data type is float32.
 Return type
Variable
Examples
import paddle.fluid as fluid from paddle.fluid.param_attr import ParamAttr x = fluid.data(name="x", shape=[None,5,10,10], dtype="float32") mode = 'channel' output = fluid.layers.prelu( x,mode,param_attr=ParamAttr(name='alpha'))
psroi_pool¶

paddle.fluid.layers.
psroi_pool
(input, rois, output_channels, spatial_scale, pooled_height, pooled_width, name=None)[source] PSROIPool Operator, rois of this op should be a LoDTensor
Position sensitive region of interest pooling (also known as PSROIPooling) is to perform positionsensitive average pooling on regions of interest specified by input, takes as input N positionsensitive score maps and a list of num_rois regions of interest.
PSROIPooling for RFCN. Please refer to https://arxiv.org/abs/1605.06409 for more details.
 Parameters
input (Variable) – Tensor, the input of PSROIPoolOp. The format of input tensor is NCHW. Where N is the batch size, C is the number of input channels, H is the height of the input feature map, and W is the width. The data type can be float32 or float64
rois (Variable) – LoDTensor, ROIs (Regions of Interest) to pool over.It should be a 2D LoDTensor of shape (num_rois, 4), the lod level is 1. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates. The data type is the same as input
output_channels (int) – (int), the number of channels of the output feature map. For a task of C classes of objects, output_channels should be (C + 1) for classification only
spatial_scale (float) – (float, default 1.0), Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling Default: 1.0
pooled_height (int) – (int, default 1), the pooled output height Default: 1
pooled_width (int) – (int, default 1), the pooled output width Default: 1
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Tensor, the output of PSROIPoolOp is a 4D Tensor with shape (num_rois, output_channels, pooled_h, pooled_w). The data type is the same as x .
 Return Type:
Variable
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[100, 490, 28, 28], dtype='float32') rois = fluid.data(name='rois', shape=[None, 4], lod_level=1, dtype='float32') pool_out = fluid.layers.psroi_pool(x, rois, 10, 1.0, 7, 7)
py_func¶

paddle.fluid.layers.
py_func
(func, x, out, backward_func=None, skip_vars_in_backward_input=None)[source] This API is used to register customized OP to Fluid. The forward function of the registered OP is
func
and the backward function of that isbackward_func
. Paddle will callfunc
at forward runtime and callbackward_func
at backward runtime(ifbackward_func
is not None).x
is the input offunc
, whose type must be LoDTensor;out
is the output offunc
, whose type can be either LoDTensor or NumPy array.The input of the backward function
backward_func
isx
,out
and the gradient ofout
. If some variables ofout
have no gradient, the relevant input variable ofbackward_func
is None. If some variables ofx
do not have a gradient, the user should return None inbackward_func
.The data type and shape of
out
should also be set correctly before this API is called, and the data type and shape of the gradient ofout
andx
will be inferred automatically.This API can also be used to debug the neural network by setting the
func
as a function that only print variables. Parameters
func (callable) – The forward function of the registered OP. When the network is running, the forward output
out
will be calculated according to this function and the forward inputx
.x (Variable) – The input of the forward function
func
, its type can be Variable  tuple[Variable]  list[Variale], in which Variable is LoDTensor.out (Variable) – The output of the forward function
func
, its type can be Variable  tuple[Variable]  list[Variale], in which Variable can be either LoDTensor or NumPy array. Since Paddle cannot automatically infer the shape and data type ofout
,out
must be created in advance.backward_func (callable, optional) – The backward function of the registered OP. Its default value is None, which means there is no reverse calculation. If it is not None,
backward_func
is called to calculate the gradient ofx
when the network is at backward runtime.skip_vars_in_backward_input (Variable, optional) – It’s used to limit the input variable list of
backward_func
, and it can be single Variable, tuple[Variable] or list[Variable]. It must belong to eitherx
orout
. The default value is None, which means that no variables need to be removed fromx
andout
. If it is not None, these variables will not be the input ofbackward_func
. This parameter is only useful whenbackward_func
is not None.
 Returns
The output
out
of the forward functionfunc
. Return type
Variable
Examples
import paddle.fluid as fluid import six def create_tmp_var(name, dtype, shape): return fluid.default_main_program().current_block().create_var( name=name, dtype=dtype, shape=shape) # Tanh activation function provided by Paddle C++ op # Here, tanh is used as an example to show how to use py_func def tanh(x): return np.tanh(x) # Skip forward input x def tanh_grad(y, dy): return np.array(dy) * (1  np.square(np.array(y))) def debug_func(x): print(x) def simple_net(img, label): hidden = img for idx in six.moves.range(4): hidden = fluid.layers.fc(hidden, size=200) new_hidden = create_tmp_var(name='hidden_{}'.format(idx), dtype=hidden.dtype, shape=hidden.shape) # Userdefined forward and backward hidden = fluid.layers.py_func(func=tanh, x=hidden, out=new_hidden, backward_func=tanh_grad, skip_vars_in_backward_input=hidden) # Userdefined debugging layer, which can print out variable details fluid.layers.py_func(func=debug_func, x=hidden, out=None) prediction = fluid.layers.fc(hidden, size=10, act='softmax') loss = fluid.layers.cross_entropy(input=prediction, label=label) return fluid.layers.mean(loss)
random_crop¶

paddle.fluid.layers.
random_crop
(x, shape, seed=None)[source] This operator takes a batch of instance, and do random cropping on each instance. It means that cropping positions differs on each instance, which is determined by an uniform random generator. All cropped instances have the same shape, which is determined by the operator’s attribute ‘shape’.
 Parameters
x (Variable) – A batch of instances to random crop
shape (INTS) – The shape of a cropped instance
seed (intVariableNone) – The random seed By default, the seed will get from random.randint(65536, 65535).
 Returns
The cropped instance batch
Examples
import paddle.fluid as fluid img = fluid.data("img", [None, 3, 256, 256]) # cropped_img is [1, 3, 224, 224] cropped_img = fluid.layers.random_crop(img, shape=[3, 224, 224]) # cropped_img2 shape: [1, 2, 224, 224] # cropped_img2 = fluid.layers.random_crop(img, shape=[2, 224, 224]) # cropped_img3 shape: [1, 3, 128, 224] # cropped_img3 = fluid.layers.random_crop(img, shape=[128, 224])
rank¶

paddle.fluid.layers.
rank
(input)[source] The OP returns the number of dimensions for a tensor, which is a 0D int32 Tensor.
 Parameters
input (Variable) – The input ND tensor with shape of \([N_1, N_2, ..., N_k]\), the data type is arbitrary.
 Returns
The 0D tensor with the dimensions of the input variable.
 Return type
Variable, the output data type is int32.
Examples
import paddle.fluid as fluid input = fluid.data(name="input", shape=[3, 100, 100], dtype="float32") rank = fluid.layers.rank(input) # rank=(3,)
rank_loss¶

paddle.fluid.layers.
rank_loss
(label, left, right, name=None)[source] This operator implements the sort loss layer in the RankNet model. RankNet is a pairwise ranking model with a training sample consisting of a pair of documents (A and B), The label (P) indicates whether A is ranked higher than B or not. Please refer to more details: RankNet
Rank loss layer takes three inputs: left ( \(o_i\) ), right ( \(o_j\) ) and label ( \(P_{i,j}\) ). The inputs respectively represent RankNet’s output scores for documents A and B and the value of label P. Rank loss layer takes batch inputs with size batch_size (batch_size >= 1), P = {0, 1} or {0, 0.5, 1}, where 0.5 means that there is no information about the rank of the input pair. The following equation computes rank loss C_{i,j} from the inputs:
\[\begin{split}C_{i,j} &= \tilde{P_{ij}} * o_{i,j} + \log(1 + e^{o_{i,j}}) \\\end{split}\]\[\begin{split}o_{i,j} &= o_i  o_j \\\end{split}\]\[\tilde{P_{i,j}} &= \left \{0, 0.5, 1 \right \} \ or \ \left \{0, 1 \right \}\] Parameters
label (Variable) – 2D
Tensor
with the shape of \([batch,1]\), the data type is float32, batch indicates the size of the data. Indicats whether A ranked higher than B or not.left (Variable) – 2D
Tensor
with the shape of \([batch,1]\), the data type is float32. RankNet’s output score for doc A.right (Variable) – 2D
Tensor
with the shape of \([batch,1]\), the data type is float32. RankNet’s output score for doc B.name (strNone) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
Tensor
indicating the output value of the sort loss layer, the data type is float32, and the return value’s shape is \([batch,1]\) . Return type
Variable
 Raises
ValueError
– Any of label, left, and right is not aVariable
.
Examples
import paddle.fluid as fluid label = fluid.data(name="label", shape=[1, 1], dtype="float32") left = fluid.data(name="left", shape=[1, 1], dtype="float32") right = fluid.data(name="right", shape=[1, 1], dtype="float32") out = fluid.layers.rank_loss(label, left, right)
reduce_all¶

paddle.fluid.layers.
reduce_all
(input, dim=None, keep_dim=False, name=None)[source] This OP computes the
logical and
of tensor elements over the given dimension, and output the result. Parameters
input (Variable) – The input variable which is a Tensor or LoDTensor, the input data type should be bool.
dim (listintoptional) – The dimension along which the logical and is computed. If
None
, compute the logical and over all elements ofinput
and return a Tensor variable with a single element, otherwise must be in the range \([rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\). The default value is None.keep_dim (bool) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the
input
unlesskeep_dim
is true. The default value is False.name (strNone) – A name for this layer(optional). If set None, the layer will be named automatically. The default value is None.
 Returns
The reduced tensor variable with
logical and
in given dims. Return type
Variable, the output data type is bool.
Examples
import paddle.fluid as fluid import paddle.fluid.layers as layers import numpy as np # x is a bool Tensor variable with following elements: # [[True, False] # [True, True]] x = layers.assign(np.array([[1, 0], [1, 1]], dtype='int32')) x = layers.cast(x, 'bool') out = layers.reduce_all(x) # False out = layers.reduce_all(x, dim=0) # [True, False] out = layers.reduce_all(x, dim=1) # [False, True] # keep_dim=False, x.shape=(2,2), out.shape=(2,) out = layers.reduce_all(x, dim=1, keep_dim=True) # [[False], [True]] # keep_dim=True, x.shape=(2,2), out.shape=(2,1)
reduce_any¶

paddle.fluid.layers.
reduce_any
(input, dim=None, keep_dim=False, name=None)[source] This OP computes the
logical or
of tensor elements over the given dimension, and output the result. Parameters
input (Variable) – The input variable which is a Tensor or LoDTensor, the input data type should be bool.
dim (listintoptional) – The dimension along which the logical and is computed. If
None
, compute the logical and over all elements ofinput
and return a Tensor variable with a single element, otherwise must be in the range \([rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\). The default value is None.keep_dim (bool) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the
input
unlesskeep_dim
is true. The default value is False.name (strNone) – A name for this layer(optional). If set None, the layer
 Returns
The reduced tensor variable with
logical or
in given dims. Return type
Variable, the output data type is bool.
Examples
import paddle.fluid as fluid import paddle.fluid.layers as layers import numpy as np # x is a bool Tensor variable with following elements: # [[True, False] # [False, False]] x = layers.assign(np.array([[1, 0], [0, 0]], dtype='int32')) x = layers.cast(x, 'bool') out = layers.reduce_any(x) # True out = layers.reduce_any(x, dim=0) # [True, False] out = layers.reduce_any(x, dim=1) # [True, False] # keep_dim=False, x.shape=(2,2), out.shape=(2,) out = layers.reduce_any(x, dim=1, keep_dim=True) # [[True], [False]] # keep_dim=True, x.shape=(2,2), out.shape=(2,1)
reduce_max¶

paddle.fluid.layers.
reduce_max
(input, dim=None, keep_dim=False, name=None)[source] Computes the maximum of tensor elements over the given dimension.
 Parameters
input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.
dim (listint, optional) – The dimension along which the maximum is computed. If
None
, compute the maximum over all elements ofinput
and return a Tensor variable with a single element, otherwise must be in the range \([rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the
input
unlesskeep_dim
is true, default value is False.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Tensor, results of maximum on the specified dim of input tensor, it’s data type is the same as input’s Tensor.
 Return type
Variable
Examples
import paddle.fluid as fluid # x is a Tensor variable with following elements: # [[0.2, 0.3, 0.5, 0.9] # [0.1, 0.2, 0.6, 0.7]] # Each example is followed by the correspending output tensor. x = fluid.data(name='x', shape=[2, 4], dtype='float32') fluid.layers.reduce_max(x) # [0.9] fluid.layers.reduce_max(x, dim=0) # [0.2, 0.3, 0.6, 0.9] fluid.layers.reduce_max(x, dim=1) # [0.9, 0.7] fluid.layers.reduce_max(x, dim=1, keep_dim=True) # [[0.9], [0.7]] # y is a Tensor variable with shape [2, 2, 2] and elements as below: # [[[1.0, 2.0], [3.0, 4.0]], # [[5.0, 6.0], [7.0, 8.0]]] # Each example is followed by the correspending output tensor. y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32') fluid.layers.reduce_max(y, dim=[1, 2]) # [4.0, 8.0] fluid.layers.reduce_max(y, dim=[0, 1]) # [7.0, 8.0]
reduce_mean¶

paddle.fluid.layers.
reduce_mean
(input, dim=None, keep_dim=False, name=None)[source] Computes the mean of the input tensor’s elements along the given dimension.
 Parameters
input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.
dim (listint, optional) – The dimension along which the mean is computed. If None, compute the mean over all elements of
input
and return a variable with a single element, otherwise it must be in the range \([rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank(input) + dim[i]\).keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the
input
unlesskeep_dim
is true, default value is False.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Tensor, results of average on the specified dim of input tensor, it’s data type is the same as input’s Tensor.
 Return type
Variable
 Raises
TypeError, if out data type is different with the input data type.
Examples
import paddle.fluid as fluid # x is a Tensor variable with following elements: # [[0.2, 0.3, 0.5, 0.9] # [0.1, 0.2, 0.6, 0.7]] # Each example is followed by the correspending output tensor. x = fluid.data(name='x', shape=[2, 4], dtype='float32') fluid.layers.reduce_mean(x) # [0.4375] fluid.layers.reduce_mean(x, dim=0) # [0.15, 0.25, 0.55, 0.8] fluid.layers.reduce_mean(x, dim=1) # [0.475, 0.4] fluid.layers.reduce_mean(x, dim=1, keep_dim=True) # [[0.475], [0.4]] # y is a Tensor variable with shape [2, 2, 2] and elements as below: # [[[1.0, 2.0], [3.0, 4.0]], # [[5.0, 6.0], [7.0, 8.0]]] # Each example is followed by the correspending output tensor. y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32') fluid.layers.reduce_mean(y, dim=[1, 2]) # [2.5, 6.5] fluid.layers.reduce_mean(y, dim=[0, 1]) # [4.0, 5.0]
reduce_min¶

paddle.fluid.layers.
reduce_min
(input, dim=None, keep_dim=False, name=None)[source] Computes the minimum of tensor elements over the given dimension.
 Parameters
input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.
dim (listint, optional) – The dimensions along which the minimum is computed. If
None
, compute the minimum over all elements ofinput
and return a Tensor variable with a single element, otherwise must be in the range \([rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the
input
unlesskeep_dim
is true, default value is False.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Tensor, result of minimum on the specified dim of input tensor, it’s data type is the same as input’s Tensor.
 Return type
Variable
Examples
import paddle.fluid as fluid # x is a Tensor variable with following elements: # [[0.2, 0.3, 0.5, 0.9] # [0.1, 0.2, 0.6, 0.7]] # Each example is followed by the correspending output tensor. x = fluid.data(name='x', shape=[2, 4], dtype='float32') fluid.layers.reduce_min(x) # [0.1] fluid.layers.reduce_min(x, dim=0) # [0.1, 0.2, 0.5, 0.7] fluid.layers.reduce_min(x, dim=1) # [0.2, 0.1] fluid.layers.reduce_min(x, dim=1, keep_dim=True) # [[0.2], [0.1]] # y is a Tensor variable with shape [2, 2, 2] and elements as below: # [[[1.0, 2.0], [3.0, 4.0]], # [[5.0, 6.0], [7.0, 8.0]]] # Each example is followed by the correspending output tensor. y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32') fluid.layers.reduce_min(y, dim=[1, 2]) # [1.0, 5.0] fluid.layers.reduce_min(y, dim=[0, 1]) # [1.0, 2.0]
reduce_prod¶

paddle.fluid.layers.
reduce_prod
(input, dim=None, keep_dim=False, name=None)[source] Computes the product of tensor elements over the given dimension.
 Parameters
input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.
dim (listint, optional) – The dimensions along which the product is performed. If
None
, multipy all elements ofinput
and return a Tensor variable with a single element, otherwise must be in the range \([rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the
input
unlesskeep_dim
is true, default value is False.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Tensor, result of product on the specified dim of input tensor, it’s data type is the same as input’s Tensor.
 Return type
Variable
Examples
import paddle.fluid as fluid # x is a Tensor variable with following elements: # [[0.2, 0.3, 0.5, 0.9] # [0.1, 0.2, 0.6, 0.7]] # Each example is followed by the correspending output tensor. x = fluid.data(name='x', shape=[2, 4], dtype='float32') fluid.layers.reduce_prod(x) # [0.0002268] fluid.layers.reduce_prod(x, dim=0) # [0.02, 0.06, 0.3, 0.63] fluid.layers.reduce_prod(x, dim=1) # [0.027, 0.0084] fluid.layers.reduce_prod(x, dim=1, keep_dim=True) # [[0.027], [0.0084]] # y is a Tensor variable with shape [2, 2, 2] and elements as below: # [[[1.0, 2.0], [3.0, 4.0]], # [[5.0, 6.0], [7.0, 8.0]]] # Each example is followed by the correspending output tensor. y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32') fluid.layers.reduce_prod(y, dim=[1, 2]) # [24.0, 1680.0] fluid.layers.reduce_prod(y, dim=[0, 1]) # [105.0, 384.0]
reduce_sum¶

paddle.fluid.layers.
reduce_sum
(input, dim=None, keep_dim=False, name=None)[source] Computes the sum of tensor elements over the given dimension.
 Parameters
input (Variable) – The input variable which is a Tensor, the data type is float32, float64, int32, int64.
dim (listint, optional) – The dimensions along which the sum is performed. If
None
, sum all elements ofinput
and return a Tensor variable with a single element, otherwise must be in the range \([rank(input), rank(input))\). If \(dim[i] < 0\), the dimension to reduce is \(rank + dim[i]\).keep_dim (bool, optional) – Whether to reserve the reduced dimension in the output Tensor. The result tensor will have one fewer dimension than the
input
unlesskeep_dim
is true, default value is False.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Tensor, results of summation operation on the specified dim of input tensor, it’s data type is the same as input’s Tensor.
 Return type
Variable
 Raises
TypeError, if out data type is different with the input data type.
Examples
import paddle.fluid as fluid # x is a Tensor variable with following elements: # [[0.2, 0.3, 0.5, 0.9] # [0.1, 0.2, 0.6, 0.7]] # Each example is followed by the corresponding output tensor. x = fluid.data(name='x', shape=[2, 4], dtype='float32') fluid.layers.reduce_sum(x) # [3.5] fluid.layers.reduce_sum(x, dim=0) # [0.3, 0.5, 1.1, 1.6] fluid.layers.reduce_sum(x, dim=1) # [1.9, 1.6] fluid.layers.reduce_sum(x, dim=1, keep_dim=True) # [[1.9], [1.6]] # y is a Tensor variable with shape [2, 2, 2] and elements as below: # [[[1, 2], [3, 4]], # [[5, 6], [7, 8]]] # Each example is followed by the corresponding output tensor. y = fluid.data(name='y', shape=[2, 2, 2], dtype='float32') fluid.layers.reduce_sum(y, dim=[1, 2]) # [10, 26] fluid.layers.reduce_sum(y, dim=[0, 1]) # [16, 20]
relu¶

paddle.fluid.layers.
relu
(x, name=None)[source] Relu Activation Operator.
\(out = \max(x, 0)\)
 Parameters
x (Variable) – Input of Relu operator, an ND Tensor, with data type float32, float64 or float16
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.
 Returns
Output of Relu operator, a Tensor with shape same as input
 Return type
Variable
Examples
import paddle.fluid as fluid import numpy as np in1 = np.array([[1,0],[1,2.6]]) with fluid.dygraph.guard(): x1 = fluid.dygraph.to_variable(in1) out1 = fluid.layers.relu(x1) print(out1.numpy()) # [[0. 0. ] # [1. 2.6]]
relu6¶

paddle.fluid.layers.
relu6
(x, threshold=6.0, name=None)[source] Relu6 Activation Operator.
\(out = \min(\max(0, x), threshold)\)
 Parameters
x (Variable) – Input of relu6 operator, an ND Tensor, with data type float32, float64
threshold (float, optional) – The threshold value of Relu6. Default is 6.0.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name.
 Returns
Output of relu6 operator, a Tensor with the same shape as input
 Return type
output(Variable)
Examples
import paddle.fluid as fluid import numpy as np in1 = np.array([[1,0],[2.5,7.8]]) with fluid.dygraph.guard(): x1 = fluid.dygraph.to_variable(in1) out1 = fluid.layers.relu6(x=x1, threshold=6.0) print(out1.numpy()) # [[0. 0. ] # [2.5 6. ]]
reshape¶

paddle.fluid.layers.
reshape
(x, shape, actual_shape=None, act=None, inplace=False, name=None)[source] This operator changes the shape of
x
without changing its data.The target shape can be given by
shape
oractual_shape
. Whenshape
andactual_shape
are set at the same time,actual_shape
has a higher priority thanshape
but at this timeshape
can only be an integer list or tuple, andshape
still should be set correctly to gurantee shape inference in compiletime.Some tricks exist when specifying the target shape.
1. 1 means the value of this dimension is inferred from the total element number of x and remaining dimensions. Thus one and only one dimension can be set 1.
2. 0 means the actual dimension value is going to be copied from the corresponding dimension of x. The indice of 0s in shape can not exceed the dimension of x.
Here are some examples to explain it.
1. Given a 3D tensor x with a shape [2, 4, 6], and the target shape is [6, 8], the reshape operator will transform x into a 2D tensor with shape [6, 8] and leaving x’s data unchanged.
2. Given a 3D tensor x with a shape [2, 4, 6], and the target shape specified is [2, 3, 1, 2], the reshape operator will transform x into a 4D tensor with shape [2, 3, 4, 2] and leaving x’s data unchanged. In this case, one dimension of the target shape is set to 1, the value of this dimension is inferred from the total element number of x and remaining dimensions.
3. Given a 3D tensor x with a shape [2, 4, 6], and the target shape is [1, 0, 3, 2], the reshape operator will transform x into a 4D tensor with shape [2, 4, 3, 2] and leaving x’s data unchanged. In this case, besides 1, 0 means the actual dimension value is going to be copied from the corresponding dimension of x.
 Note:
The parameter
actual_shape
will be deprecated in the future and only useshape
instead to represent the target shape.
 Parameters
x (Variable) – A
Tensor
orLoDTensor
. The data type isfloat32
,float64
,int32
orint64
.shape (listtupleVariable) – Define the target shape. At most one dimension of the target shape can be 1. The data type is
int32
. Ifshape
is a list or tuple, the elements of it should be integers or Tensors with shape [1]. Ifshape
is an Variable, it should be an 1D Tensor .actual_shape (variable, optional) – An 1D
Tensor
orLoDTensor
. The data type isint32
. If provided, reshape according to this given shape rather thanshape
specifying shape. That is to sayactual_shape
has a higher priority thanshape(listtuple)
but notshape(Variable)
. This argumentactual_shape
will be removed in a future version. Instructions for updating:actual_shape
will be removed in future versions and replaced byshape
.act (str, optional) – The nonlinear activation to be applied to the reshaped input. Default None.
inplace (bool, optional) – If
inplace
is True, the input and output oflayers.reshape
are the same variable. Otherwise, the input and output oflayers.reshape
are different variable. Default False. Note that ifx
is more than one OPs’ input,inplace
must be False.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
A
Tensor
orLoDTensor
. The data type is same asx
. It is a new tensor variable ifinplace
isFalse
, otherwise it isx
. Ifact
is None, return the reshaped tensor variable, otherwise return the activated tensor variable. Return type
Variable
 Raises
TypeError
– If actual_shape is neither Variable nor None.ValueError
– If more than one elements ofshape
is 1.ValueError
– If the element ofshape
is 0, the corresponding dimension should be less than or equal to the dimension ofx
.ValueError
– If the elements inshape
is negative except 1.
Examples
import paddle.fluid as fluid # example 1: # attr shape is a list which doesn't contain tensor Variable. data_1 = fluid.data( name='data_1', shape=[2, 4, 6], dtype='float32') reshaped_1 = fluid.layers.reshape( x=data_1, shape=[1, 0, 3, 2], inplace=True) # the shape of reshaped_1 is [2,4,3,2]. # example 2: # attr shape is a list which contains tensor Variable. data_2 = fluid.layers.fill_constant([2,25], "int32", 3) dim = fluid.layers.fill_constant([1], "int32", 5) reshaped_2 = fluid.layers.reshape(data_2, shape=[dim, 10]) # the shape of reshaped_2 is [5,10].
resize_bilinear¶

paddle.fluid.layers.
resize_bilinear
(input, out_shape=None, scale=None, name=None, actual_shape=None, align_corners=True, align_mode=1, data_format='NCHW')[source] This op resizes the input by performing bilinear interpolation based on given output shape which specified by actual_shape, out_shape and scale in priority order.
Warning: the parameter
actual_shape
will be deprecated in the future and only useout_shape
instead.Bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables (e.g. Hdirection and Wdirection in this op) on a rectilinear 2D grid. The key idea is to perform linear interpolation first in one direction, and then again in the other direction.
For details of bilinear interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation
Align_corners and align_mode are optinal parameters,the calculation method of interpolation can be selected by them.
Example:
For scale: if align_corners = True && out_size > 1 : scale_factor = (in_size1.0)/(out_size1.0) else: scale_factor = float(in_size/out_size) Bilinear interpolation: if: align_corners = False , align_mode = 0 input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = (H_{in}+0.5) * scale_{factor}  0.5 W_out = (W_{in}+0.5) * scale_{factor}  0.5 else: input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = H_{in} * scale_{factor} W_out = W_{in} * scale_{factor}
 Parameters
input (Variable) – 4D Tensor(NCHW), its data type is float32, float64, or uint8, its data format is specified by
data_format
.out_shape (listtupleVariableNone) – Output shape of resize bilinear layer, the shape is (out_h, out_w).Default: None. If a list, each element can be an integer or a Tensor Variable with shape: [1]. If a Tensor Variable, its dimension size should be 1.
scale (floatVariableNone) – The multiplier for the input height or width. At least one of
out_shape
orscale
must be set. Andout_shape
has a higher priority thanscale
. Default: None.actual_shape (Variable) – An optional input to specify output shape dynamically. If provided, image resize according to this given shape rather than
out_shape
andscale
specifying shape. That is to say actual_shape has the highest priority. It is recommended to useout_shape
if you want to specify output shape dynamically, becauseactual_shape
will be deprecated. When using actual_shape to specify output shape, one ofout_shape
andscale
should also be set, otherwise errors would be occured in graph constructing stage. Default: Nonealign_corners (bool) – an optional bool. Defaults to True. If True, the centers of 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels, If False, are not aligned
align_mode (bool) – (int, default ‘1’), optional for bilinear interpolation, can be ‘0’ for src_idx = scale*(dst_indx+0.5)0.5 , can be ‘1’ for src_idx = scale*dst_index
data_format (str, optional) – NCHW(num_batches, channels, height, width) or NHWC(num_batches, height, width, channels). Default: ‘NCHW’.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
4D tensor(NCHW or NHWC).
 Return type
Variable
Examples
#declarative mode import paddle.fluid as fluid import numpy as np input = fluid.data(name="input", shape=[None,3,6,10]) #1 output = fluid.layers.resize_bilinear(input=input,out_shape=[12,12]) #2 #x = np.array([2]).astype("int32") #dim1 = fluid.data(name="dim1", shape=[1], dtype="int32") #fluid.layers.assign(input=x, output=dim1) #output = fluid.layers.resize_bilinear(input=input,out_shape=[12,dim1]) #3 #x = np.array([3,12]).astype("int32") #shape_tensor = fluid.data(name="shape_tensor", shape=[2], dtype="int32") #fluid.layers.assign(input=x, output=shape_tensor) #output = fluid.layers.resize_bilinear(input=input,out_shape=shape_tensor) #4 #x = np.array([0.5]).astype("float32") #scale_tensor = fluid.data(name="scale", shape=[1], dtype="float32") #fluid.layers.assign(x,scale_tensor) #output = fluid.layers.resize_bilinear(input=input,scale=scale_tensor) place = fluid.CPUPlace() exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) input_data = np.random.rand(2,3,6,10).astype("float32") output_data = exe.run(fluid.default_main_program(), feed={"input":input_data}, fetch_list=[output], return_numpy=True) print(output_data[0].shape) #1 # (2, 3, 12, 12) #2 # (2, 3, 12, 2) #3 # (2, 3, 3, 12) #4 # (2, 3, 3, 5) #imperative mode import paddle.fluid.dygraph as dg with dg.guard(place) as g: input = dg.to_variable(input_data) output = fluid.layers.resize_bilinear(input=input, out_shape=[12,12]) print(output.shape) # [2L, 3L, 12L, 12L]
resize_nearest¶

paddle.fluid.layers.
resize_nearest
(input, out_shape=None, scale=None, name=None, actual_shape=None, align_corners=True, data_format='NCHW')[source] This op resizes the input by performing nearest neighbor interpolation in both the height direction and the width direction based on given output shape which is specified by actual_shape, out_shape and scale in priority order.
Warning: the parameter
actual_shape
will be deprecated in the future and only useout_shape
instead.Example:
For scale: if align_corners = True && out_size > 1 : scale_factor = (in_size1.0)/(out_size1.0) else: scale_factor = float(in_size/out_size) Nearest neighbor interpolation: if: align_corners = False input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = floor(H_{in} * scale_{factor}) W_out = floor(W_{in} * scale_{factor}) else: align_corners = True input : (N,C,H_in,W_in) output: (N,C,H_out,W_out) where: H_out = round(H_{in} * scale_{factor}) W_out = round(W_{in} * scale_{factor})
For details of nearest neighbor interpolation, please refer to Wikipedia: https://en.wikipedia.org/wiki/Nearestneighbor_interpolation
 Parameters
input (Variable) – 4D Tensor, its data type is float32, float64, or uint8, its data format is specified by
data_format
.out_shape (listtupleVariableNone) – The output shape of resized tensor, the shape is (out_h, out_w). Default: None. Every element should be an integer or a tensor Variable with shape: [1] if it is a list. If it is a tensor Variable, its dimension size should be 1.
scale (floatVariableNone) – The multiplier for the input height or width. At least one of
out_shape
orscale
must be set. Andout_shape
has a higher priority thanscale
. Default: None.name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
actual_shape (Variable) – An optional input to specify output shape dynamically. If provided, image resize according to this given shape rather than
out_shape
andscale
specifying shape. That is to say actual_shape has the highest priority. It is recommended to useout_shape
if you want to specify output shape dynamically, becauseactual_shape
will be deprecated. When using actual_shape to specify output shape, one ofout_shape
andscale
should also be set, otherwise errors would be occured in graph constructing stage. Default: Nonealign_corners (bool) – an optional bool. Defaults to True. If True, the centers of 4 corner pixels of the input and output tensors are aligned, preserving the values at the corner pixels, If False, are not aligned
data_format (str, optional) – NCHW(num_batches, channels, height, width) or NHWC(num_batches, height, width, channels). Default: ‘NCHW’.
 Returns
4D tensor(NCHW or NHWC).
 Return type
Variable
Examples
#declarative mode import paddle.fluid as fluid import numpy as np input = fluid.data(name="input", shape=[None,3,6,10]) #1 output = fluid.layers.resize_nearest(input=input,out_shape=[12,12]) #2 #x = np.array([2]).astype("int32") #dim1 = fluid.data(name="dim1", shape=[1], dtype="int32") #fluid.layers.assign(input=x, output=dim1) #output = fluid.layers.resize_nearest(input=input,out_shape=[12,dim1]) #3 #x = np.array([3,12]).astype("int32") #shape_tensor = fluid.data(name="shape_tensor", shape=[2], dtype="int32") #fluid.layers.assign(input=x, output=shape_tensor) #output = fluid.layers.resize_nearest(input=input,out_shape=shape_tensor) #4 #x = np.array([0.5]).astype("float32") #scale_tensor = fluid.data(name="scale", shape=[1], dtype="float32") #fluid.layers.assign(x,scale_tensor) #output = fluid.layers.resize_nearest(input=input,scale=scale_tensor) place = fluid.CPUPlace() exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) input_data = np.random.rand(2,3,6,10).astype("float32") output_data = exe.run(fluid.default_main_program(), feed={"input":input_data}, fetch_list=[output], return_numpy=True) print(output_data[0].shape) #1 # (2, 3, 12, 12) #2 # (2, 3, 12, 2) #3 # (2, 3, 3, 12) #4 # (2, 3, 3, 5) #imperative mode import paddle.fluid.dygraph as dg with dg.guard(place) as g: input = dg.to_variable(input_data) output = fluid.layers.resize_nearest(input=input, out_shape=[12,12]) print(output.shape) # [2L, 3L, 12L, 12L]
roi_align¶

paddle.fluid.layers.
roi_align
(input, rois, pooled_height=1, pooled_width=1, spatial_scale=1.0, sampling_ratio=1, name=None)[source] RoIAlign Operator
Region of interest align (also known as RoI align) is to perform bilinear interpolation on inputs of nonuniform sizes to obtain fixedsize feature maps (e.g. 7*7)
Dividing each region proposal into equalsized sections with the pooled_width and pooled_height. Location remains the origin result.
In each ROI bin, the value of the four regularly sampled locations are computed directly through bilinear interpolation. The output is the mean of four locations. Thus avoid the misaligned problem.
 Parameters
input (Variable) – (Tensor), The input of ROIAlignOp. The data type is float32 or float64.The format of input tensor is NCHW. Where N is batch size, C is the number of input channels, H is the height of the feature, and W is the width of the feature
rois (Variable) – ROIs (Regions of Interest) to pool over.It should be a 2D LoDTensor of shape (num_rois, 4), the lod level is 1. The data type is float32 or float64. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates.
pooled_height (int32, optional) – (int, default 1), The pooled output height Default: 1
pooled_width (int32, optional) – (int, default 1), The pooled output width Default: 1
spatial_scale (float32, optional) – (float, default 1.0), Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling Default: 1.0
sampling_ratio (int32, optional) – (int,default 1),number of sampling points in the interpolation gridIf <=0, then grid points are adaptive to roi_width and pooled_w, likewise for height Default: 1
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
 Returns
Output: (Tensor), The output of ROIAlignOp is a 4D tensor with shape (num_rois, channels, pooled_h, pooled_w). The data type is float32 or float64.
 Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.data( name='data', shape=[None, 256, 32, 32], dtype='float32') rois = fluid.data( name='rois', shape=[None, 4], dtype='float32') align_out = fluid.layers.roi_align(input=x, rois=rois, pooled_height=7, pooled_width=7, spatial_scale=0.5, sampling_ratio=1)
roi_pool¶

paddle.fluid.layers.
roi_pool
(input, rois, pooled_height=1, pooled_width=1, spatial_scale=1.0)[source] This operator implements the roi_pooling layer. Region of interest pooling (also known as RoI pooling) is to perform max pooling on inputs of nonuniform sizes to obtain fixedsize feature maps (e.g. 7*7).
The operator has three steps:
Dividing each region proposal into equalsized sections with the pooled_width and pooled_height;
Finding the largest value in each section;
Copying these max values to the output buffer.
For more information, please refer to https://stackoverflow.com/questions/43430056/whatisroilayerinfastrcnn
 Parameters
input (Variable) – Input feature, 4DTensor with the shape of [N,C,H,W], where N is the batch size, C is the input channel, H is Height, W is weight. The data type is float32 or float64.
rois (Variable) – ROIs (Regions of Interest) to pool over. 2DLoDTensor with the shape of [num_rois,4], the lod level is 1. Given as [[x1, y1, x2, y2], …], (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates.
pooled_height (int, optional) – The pooled output height, data type is int32. Default: 1
pooled_width (int, optional) – The pooled output height, data type is int32. Default: 1
spatial_scale (float, optional) – Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling. Default: 1.0
 Returns
The pooled feature, 4DTensor with the shape of [num_rois, C, pooled_height, pooled_width].
 Return type
Variable
Examples:
import paddle.fluid as fluid import numpy as np DATATYPE='float32' place = fluid.CPUPlace() #place = fluid.CUDAPlace(0) input_data = np.array([i for i in range(1,17)]).reshape(1,1,4,4).astype(DATATYPE) roi_data =fluid.create_lod_tensor(np.array([[1., 1., 2., 2.], [1.5, 1.5, 3., 3.]]).astype(DATATYPE),[[2]], place) x = fluid.data(name='input', shape=[None,1,4,4], dtype=DATATYPE) rois = fluid.data(name='roi', shape=[None,4], dtype=DATATYPE) pool_out = fluid.layers.roi_pool( input=x, rois=rois, pooled_height=1, pooled_width=1, spatial_scale=1.0) exe = fluid.Executor(place) out, = exe.run(feed={'input':input_data ,'roi':roi_data}, fetch_list=[pool_out.name]) print(out) #array([[[[11.]]], [[[16.]]]], dtype=float32) print(np.array(out).shape) # (2, 1, 1, 1)
row_conv¶

paddle.fluid.layers.
row_conv
(input, future_context_size, param_attr=None, act=None)[source] Rowconvolution operator
The row convolution is called lookahead convolution. This operator was introduced in the following paper for DeepSpeech2: http://www.cs.cmu.edu/~dyogatam/papers/wang+etal.iclrworkshop2016.pdf
The main motivation is that a bidirectional RNN, useful in DeepSpeech like speech models, learns representation for a sequence by performing a forward and a backward pass through the entire sequence. However, unlike unidirectional RNNs, bidirectional RNNs are challenging to deploy in an online and lowlatency setting. The lookahead convolution incorporates information from future subsequences in a computationally efficient manner to improve unidirectional recurrent neural networks. The row convolution operator is different from the 1D sequence convolution, and is computed as follows:
Given an input sequence \(X\) of length \(t\) and input dimension \(D\), and a filter (\(W\)) of size \(context \times D\), the output sequence is convolved as:
$$ out_{i} = \sum_{j=i}^{i + context  1} X_{j} \cdot W_{ji} $$
In the above equation:
\(Out_{i}\): The ith row of output variable with shape [1, D].
\(context\): Future context size.
\(X_{j}\): The jth row of input variable with shape [1, D].
\(W_{ji}\): The (ji)th row of parameters with shape [1, D].
More details about row_conv please refer to the design document https://github.com/PaddlePaddle/Paddle/issues/2228#issuecomment303903645 .
 Parameters
input (Variable) – the input(X) is a LodTensor or tensor, LodTensor(X) supports variable timelength input sequences. The underlying tensor in this LoDTensor is a matrix with shape (T x N), where T is the total time steps in this minibatch and N is the input data dimension. the shape of Tensor input(X) has shape (B x T x N), B is batch size;.
future_context_size (int) – Future context size. Please note, the shape of convolution kernel is [future_context_size + 1, D].
param_attr (ParamAttr) – Attributes of parameters, including name, initializer etc.
act (str) – Nonlinear activation to be applied to output variable.
 Returns
the output(Out) is a LodTensor or Tensor, which has same type and same shape as X.
Examples
>>> # for LodTensor inputs >>> import paddle.fluid as fluid >>> x = fluid.data(name='x', shape=[9, 16], >>> dtype='float32', lod_level=1) >>> out = fluid.layers.row_conv(input=x, future_context_size=2) >>> # for Tensor inputs >>> x = fluid.data(name='x', shape=[9, 4, 16], dtype='float32') >>> out = fluid.layers.row_conv(input=x, future_context_size=2)
sampled_softmax_with_cross_entropy¶

paddle.fluid.layers.
sampled_softmax_with_cross_entropy
(logits, label, num_samples, num_true=1, remove_accidental_hits=True, use_customized_samples=False, customized_samples=None, customized_probabilities=None, seed=0)[source] Sampled Softmax With Cross Entropy Operator.
Cross entropy loss with sampled softmax is used as the output layer for larger output classes extensively. This operator samples a number of samples for all examples, and computes the softmax normalized values for each row of the sampled tensor, after which crossentropy loss is computed.
Because this operator performs a softmax on logits internally, it expects unscaled logits. This operator should not be used with the output of softmax operator since that would produce incorrect results.
For examples with T true labels (T >= 1), we assume that each true label has a probability of 1/T. For each sample, S samples are generated using a log uniform distribution. True labels are concatenated with these samples to form T + S samples for each example. So, assume the shape of logits is [N x K], the shape for samples is [N x (T+S)]. For each sampled label, a probability is calculated, which corresponds to the Q(yx) in [Jean et al., 2014](http://arxiv.org/abs/1412.2007).
Logits are sampled according to the sampled labels. Then if remove_accidental_hits is True, if a sample[i, j] accidentally hits true labels, then the corresponding sampled_logits[i, j] is minus by 1e20 to make its softmax result close to zero. Then sampled logits are subtracted by logQ(yx), these sampled logits and reindexed labels are used to compute a softmax with cross entropy.
 Parameters
logits (Variable) – The unscaled log probabilities, which is a 2D tensor with shape [N x K]. N is the batch_size, and K is the class number.
label (Variable) – The ground truth which is a 2D tensor. Label is a Tensor<int64> with shape [N x T], where T is the number of true labels per example.
num_samples (int) – The number for each example, num_samples should be less than the number of class.
num_true (int) – The number of target classes per training example.
remove_accidental_hits (bool) – A flag indicating whether to remove accidental hits when sampling. If True and if a sample[i, j] accidentally hits true labels, then the corresponding sampled_logits[i, j] is minus by 1e20 to make its softmax result close to zero. Default is True.
use_customized_samples (bool) – Whether to use custom samples and probabities to sample logits.
customized_samples (Variable) – User defined samples, which is a 2D tensor with shape [N, T + S]. S is the num_samples, and T is the number of true labels per example.
customized_probabilities (Variable) – User defined probabilities of samples, a 2D tensor which has the same shape with customized_samples.
seed (int) – The random seed for generating random number, which is used in the process of sampling. Default is 0.
 Returns
 Return the cross entropy loss which is a 2D tensor with shape
[N x 1].
 Return type
Variable
Examples
import paddle.fluid as fluid input = fluid.layers.data(name='data', shape=[256], dtype='float32') label = fluid.layers.data(name='label', shape=[1], dtype='int64') fc = fluid.layers.fc(input=input, size=100) out = fluid.layers.sampled_softmax_with_cross_entropy( logits=fc, label=label, num_samples=25)
sampling_id¶

paddle.fluid.layers.
sampling_id
(x, min=0.0, max=1.0, seed=0, dtype='float32')[source] This op is used for sampling id from multinomial distribution from the input, sampling one id for one sample.
 Parameters
x (Variable) – 2D tensor, [batch_size, input_feature_dimensions]
min (Float) – minimum , default 0.0.
max (Float) – maximum, default 1.0.
seed (Float) – Random seed, default 0. if seed is not 0, will generate same number every time.
dtype (np.dtypecore.VarDesc.VarTypestr) – The type of output data : float32, float_16, int etc
 Returns
sampling tensor.
 Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.data( name="X", shape=[13, 11], dtype='float32') out = fluid.layers.sampling_id(x)
scale¶

paddle.fluid.layers.
scale
(x, scale=1.0, bias=0.0, bias_after_scale=True, act=None, name=None)[source] Scale operator.
Putting scale and bias to the input Tensor as following:
bias_after_scale
is True:\[Out=scale*X+bias\]bias_after_scale
is False:\[Out=scale*(X+bias)\] Parameters
x (Variable) – Input ND Tensor of scale operator. Data type can be float32, float64, int8, int16, int32, int64, uint8.
scale (float) – The scale factor of the input.
bias (float) – The bias to be put on the input.
bias_after_scale (bool) – Apply bias addition after or before scaling. It is useful for numeric stability in some circumstances.
act (str, optional) – Activation applied to the output such as tanh, softmax, sigmoid, relu.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name
 Returns
Output tensor of scale operator, with shape and data type same as input.
 Return type
Variable(TensorLoDTensor)
Examples
import paddle.fluid as fluid import numpy as np inputs = fluid.layers.data(name="x", shape=[2, 3], dtype='float32') output = fluid.layers.scale(inputs, scale = 2.0, bias = 1.0) exe = fluid.Executor(fluid.CPUPlace()) exe.run(fluid.default_startup_program()) img = np.array([[1, 2, 3], [4, 5, 6]]).astype(np.float32) res = exe.run(fluid.default_main_program(), feed={'x':img}, fetch_list=[output]) print(res) # [array([[ 3., 5., 7.], [ 9., 11., 13.]], dtype=float32)]
scatter¶

paddle.fluid.layers.
scatter
(input, index, updates, name=None, overwrite=True)[source] Scatter Layer
Output is obtained by updating the input on selected indices based on updates.
 Parameters
input (Variable) – The input ND Tensor with rank>=1. Data type can be float32.
index (Variable) – The index 1D Tensor. Data type can be int32, int64. The length of index cannot exceed updates’s length, and the value in index cannot exceed input’s length.
updates (Variable) – update input with updates parameter based on index. shape should be the same as input, and dim value with dim > 1 shoule be the same as input.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
overwrite (bool) – The mode that updating the output when there are same indices. If True, use the overwrite mode to update the output of the same index, if False, use the accumulate mode to update the output of the same index. Default value is True.
 Returns
The output is a Tensor with the same shape as input.
 Return type
Variable(TensorLoDTensor)
Examples
import numpy as np import paddle.fluid as fluid input = fluid.layers.data(name='data', shape=[3, 2], dtype='float32', append_batch_size=False) index = fluid.layers.data(name='index', shape=[4], dtype='int64', append_batch_size=False) updates = fluid.layers.data(name='update', shape=[4, 2], dtype='float32', append_batch_size=False) output = fluid.layers.scatter(input, index, updates, overwrite=False) exe = fluid.Executor(fluid.CPUPlace()) exe.run(fluid.default_startup_program()) in_data = np.array([[1, 1], [2, 2], [3, 3]]).astype(np.float32) index_data = np.array([2, 1, 0, 1]).astype(np.int64) update_data = np.array([[1, 1], [2, 2], [3, 3], [4, 4]]).astype(np.float32) res = exe.run(fluid.default_main_program(), feed={'data':in_data, "index":index_data, "update":update_data}, fetch_list=[output]) print(res) # [array([[3., 3.], # [6., 6.], # [1., 1.]], dtype=float32)]
selu¶

paddle.fluid.layers.
selu
(x, scale=None, alpha=None, name=None)[source] Selu Operator.
The equation is:
\[ \begin{align}\begin{aligned}selu= \lambda* \begin{cases} x &\quad \text{ if } x>0\\ \alpha * e^x  \alpha &\quad \text{ if } x<=0 \end{cases}\end{aligned}\end{align} \]The input X can carry the LoD (Level of Details) information, or not. And the output shares the LoD information with input X.
 Parameters
x (Variable) – The input ND Tensor.
scale (float, optional) – lambda in selu activation function, the default value is 1.0507009873554804934193349852946. For more information about this value, please refer to: https://arxiv.org/abs/1706.02515.
alpha (float, optional) – alpha in selu activation function, the default value is 1.6732632423543772848170429916717. For more information about this value, please refer to: https://arxiv.org/abs/1706.02515.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
The output Tensor or LoDTensor with the same shape and LoD information as input.
 Return type
Variable(TensorLoDTensor)
Examples
import paddle.fluid as fluid import numpy as np inputs = fluid.layers.data(name="x", shape=[2, 2], dtype="float32") output = fluid.layers.selu(inputs) exe = fluid.Executor(fluid.CPUPlace()) exe.run(fluid.default_startup_program()) img = np.array([[0, 1],[2, 3]]).astype(np.float32) res = exe.run(fluid.default_main_program(), feed={'x':img}, fetch_list=[output]) print(res) # [array([[0. , 1.050701],[2.101402, 3.152103]], dtype=float32)]
sequence_concat¶

paddle.fluid.layers.
sequence_concat
(input, name=None)[source] Notes: The Op only receives LoDTensor as input. If your input is Tensor, please use concat Op.(fluid.layers. concat ).
This operator only supports LoDTensor as input. It concatenates the multiple LoDTensor from input by the LoD information, and outputs the concatenated LoDTensor.
input is a list of LoDTensor: input = [x1, x2] where: x1.lod = [[0, 3, 5]] x1.data = [[1], [2], [3], [4], [5]] x1.shape = [5, 1] x2.lod = [[0, 2, 4]] x2.data = [[6], [7], [8], [9]] x2.shape = [4, 1] and should satisfy: len(x1.lod[0]) == len(x2.lod[0]) output is LoDTensor: out.lod = [[0, 3+2, 5+4]] out.data = [[1], [2], [3], [6], [7], [4], [5], [8], [9]] out.shape = [9, 1]
 Parameters
input (list of Variable) – List of LoDTensor to be concatenated. The length of each LoDTensor should be same. The data type can be float32, float64 or int64.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
Output the concatenated LoDTensor. The data type is same as input.
 Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[1, 10], dtype='float32', lod_level=1) y = fluid.data(name='y', shape=[1, 10], dtype='float32', lod_level=1) out = fluid.layers.sequence_concat(input=[x, y])
sequence_conv¶

paddle.fluid.layers.
sequence_conv
(input, num_filters, filter_size=3, filter_stride=1, padding=True, padding_start=None, bias_attr=None, param_attr=None, act=None, name=None)[source] Notes: The Op only receives LoDTensor as input. If your input is Tensor, please use conv2d Op.(fluid.layers. conv2d ).
This operator receives input sequences with variable length and other convolutional configuration parameters(num_filters, filter_size) to apply the convolution operation. It fills allzero padding data on both sides of the sequence by default to ensure that the output is the same length as the input. You can customize the padding behavior by configuring the parameter
padding_start
.Warning: the parameter
padding
take no effect and will be deprecated in the future.Here we will illustrate the details of the padding operation: For a minibatch of 2 variable lengths sentences, containing 3, and 1 timesteps: Assumed input (X) is a [4, N] float LoDTensor, and for the sake of simplicity, we assume N=2. input.data = [[1, 1], [2, 2], [3, 3], [4, 4]] This is to say that input (X) has 4 words and the dimension of each word representation is 2. * Case1: If padding_start is 1 and filter_size is 3. The length of padding data is calculated as follows: up_pad_len = max(0, padding_start) = 1 down_pad_len = max(0, filter_size + padding_start  1) = 1 The output of the input sequence after padding is: data_aftet_padding = [[0, 0, 1, 1, 2, 2], [1, 1, 2, 2, 3, 3], [2, 2, 3, 3, 0, 0], [0, 0, 4, 4, 0, 0]] It will be multiplied by the filter weight to get the final output. Assume num_filters = 3 output.data = [[ 0.3234, 0.2334, 0.7433], [ 0.5646, 0.9464, 0.1223], [0.1343, 0.5653, 0.4555], [ 0.9954, 0.1234, 0.1234]] output.shape = [4, 3] # 3 = num_filters output.lod = [[0, 3, 4]] # Remain the same
 Parameters
input (Variable) – LoDTensor with shape \((M, K)\), where M is the total timestep of minibatch and K is hidden_size of input. Only lod_level of 1 is supported. The data type should be float32 or float64.
num_filters (int) – the number of filters.
filter_size (int) – the height of filter. Specified filter width is not supported, the width is hidden_size by default. Default: 3.
filter_stride (int) – stride of the filter. Currently only supports
stride
= 1.padding (bool) – the parameter
padding
take no effect and will be discarded in the future. Currently, it will always pad input to make sure the length of the output is the same as input whetherpadding
is set true or false. Because the length of input sequence may be shorter thanfilter_size
, which will cause the convolution result to not be computed correctly. These padding data will not be trainable or updated while trainnig. Default: True.padding_start (int) – It is used to indicate the start index for padding the input sequence, which can be negative. The negative number means to pad
padding_start
timesteps of allzero data at the beginning of each instance. The positive number means to skippadding_start
timesteps of each instance, and it will pad \(filter\_size + padding\_start  1\) timesteps of allzero data at the end of the sequence to ensure that the output is the same length as the input. If set None, the same length \(\frac{filter\_size}{2}\) of data will be filled on both sides of the sequence. If set 0, the length of \(filter\_size  1\) data is padded at the end of each input sequence. Default: None.bias_attr (ParamAttr) – To specify the bias parameter property. Default: None, which means the default bias parameter property is used. See usage for details in ParamAttr .
param_attr (ParamAttr) – To specify the weight parameter property. Default: None, which means the default weight parameter property is used. See usage for details in ParamAttr .
act (str) – Activation to be applied to the output of this layer, such as tanh, softmax, sigmoid, relu. For more information, please refer to Activation Function . Default: None.
name (str, optional) – The default value is None. Normally there is no need for user to set this property. For more information, please refer to api_guide_Name .
 Returns
LoDTensor with the same length as input. The data type is float32 or float64, which is same as input.
 Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[1, 10], dtype='float32', lod_level=1) x_conved = fluid.layers.sequence_conv(input=x, num_filters=2, filter_size=3, padding_start=1)
sequence_enumerate¶

paddle.fluid.layers.
sequence_enumerate
(input, win_size, pad_value=0, name=None)[source] Generate a new sequence for the input index sequence with shape
[d_1, win_size]
, which enumerates all the subsequences with lengthwin_size
of the input with shape[d_1, 1]
, and padded bypad_value
if necessary in generation.Please note that the input must be LodTensor.
Input x: x.lod = [[0, 3, 5]] x.data = [[1], [2], [3], [4], [5]] x.dims = [5, 1] Attrs: win_size = 2 pad_value = 0 Output: out.lod = [[0, 3, 5]] out.data = [[1, 2], [2, 3], [3, 0], [4, 5], [5, 0]] out.dims = [5, 2]
 Parameters
input (Variable) – The input variable which is a index sequence, which should be a LodTensor with shape
[d_1, 1]
and 1level lod info. The data type should be float32, float64, int8, int32 or int64.win_size (int) – The window size for enumerating all subsequences.
pad_value (int, optional) – The padding value, default 0.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
Returns: The enumerate sequence variable which is a LoDTensor with shape
[d_1, win_size]
and 1level lod info. The data type is same asinput
.Return Type: Variable
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[1, 1], dtype='int32', lod_level=1) out = fluid.layers.sequence_enumerate(input=x, win_size=3, pad_value=0)
sequence_expand¶

paddle.fluid.layers.
sequence_expand
(x, y, ref_level=1, name=None)[source] Sequence Expand Layer. This layer will expand the input variable
x
according to specified levelref_level
lod ofy
. Please note that the lod level ofx
is at most 1. If the lod level ofx
is 1, than the size of lod ofx
must be equal to the length ofref_level
lod ofy
. If the lod level ofx
is 0, then the first dim ofx
should be equal to the size ofref_level
ofy
. The rank of x is at least 2. When rank ofx
is greater than 2, then it would be viewed as a 2D tensor.Please note that the input
x
should be LodTensor or Tensor, and inputy
must be LodTensor.Following examples will explain how sequence_expand works:
Case 1 Consider 2 sequences [a][b] and [c][d], now we want to expand them to [a][b], [a][b], [c][d] and [c][d]. Sequence [a][b] expand twice and [c][d] expands twice, so the lod which according to is [2, 2]. Input x is a 1level LoDTensor: x.lod = [[2, 2]] #lod based on length may be easier to understand x.data = [[a], [b], [c], [d]] x.dims = [4, 1] input y is a LoDTensor: y.lod = [[2, 2], #the 0th level lod, according to this level [3, 3, 1, 1]] #the 1st level lod, it has nothing to do with this level ref_level: 0 then output is a 1level LoDTensor out: out.lod = [[2, 2, 2, 2]] #lod based on offfset out.data = [[a], [b], [a], [b], [c], [d], [c], [d]] out.dims = [8, 1] Case 2 Consider 3 sequences [a], [b], [c], now we want to expand them to [a][a], [c][c][c]. It's obvious that the lod info of expanded sequences is [2, 0, 3]. x is a Tensor: x.data = [[a], [b], [c]] x.dims = [3, 1] y is a LoDTensor: y.lod = [[2, 0, 3]] ref_level: 1 then output is a 1level LodTensor: out.data = [[a], [a], [c], [c], [c]] out.dims = [5, 1]
 Parameters
x (Variable) – The input variable which is a Tensor or LoDTensor, with the dims
[M, K]
. The lod level is at most 1. The data type should be float32, float64, int8, int32 or int64.y (Variable) – The input variable which is a LoDTensor, the lod level is at least 1.
ref_level (int) – Lod level of
y
to be referred byx
. If set to 1, refer the last level of lod.name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
Returns: The expanded variable which is a LoDTensor, with dims
[N, K]
.N
depends on the lod info ofx
andy
. The data type is same as input.Return Type: Variable
Examples
import paddle.fluid as fluid import paddle.fluid.layers as layers import numpy as np x = fluid.data(name='x', shape=[4, 1], dtype='float32') y = fluid.data(name='y', shape=[8, 1], dtype='float32', lod_level=1) out = layers.sequence_expand(x=x, y=y, ref_level=0) exe = fluid.Executor(fluid.CPUPlace()) place = fluid.CPUPlace() np_data = np.array([[1], [2], [3], [4]]).astype('float32') x_lod_tensor = fluid.create_lod_tensor(np_data, [[2, 2]], place) print(x_lod_tensor) #lod: [[0, 2, 4]] # dim: 4, 1 # layout: NCHW # dtype: float # data: [1 2 3 4] np_data = np.array([[1], [2], [3], [4], [5], [6], [7], [8]]).astype('float32') y_lod_tensor = fluid.create_lod_tensor(np_data, [[2, 2], [3,3,1,1]], place) print(y_lod_tensor) #lod: [[0, 2, 4][0, 3, 6, 7, 8]] # dim: 8, 1 # layout: NCHW # dtype: int64_t # data: [0 0 1 1 1 1 1 0] out_main = exe.run(fluid.default_main_program(), feed={'x': x_lod_tensor, 'y': y_lod_tensor}, fetch_list=[out], return_numpy=False) print(out_main[0]) #lod: [[0, 2, 4, 6, 8]] # dim: 8, 1 # layout: NCHW # dtype: float # data: [1 2 1 2 3 4 3 4]
sequence_expand_as¶

paddle.fluid.layers.
sequence_expand_as
(x, y, name=None)[source] Sequence Expand As Layer. This OP will expand the input variable
x
according to the zeroth level lod ofy
. Current implementation requires the level number ofy
’s lod must be 1, and the first dimension ofx
should be equal to the size ofy
’s zeroth level lod, thus the expanded LodTensor has the same lod info asy
. The expanded result has nothing to do withx
’s lod, so the lod of Input(X) is not considered.Please note that the input
x
should be LodTensor or Tensor, and inputy
must be LodTensor.Following examples will explain how sequence_expand_as works:
Case 1: Consider 4 sequences [a], [b], [c], [d], now we want to expand them to [a][a][a], [b][b][b], [c] and [d]. It's obvious that the lod info of expanded sequences is [0, 3, 6, 7, 8]. Given a 1level LodTensor ``x``: x.data = [[a], [b], [c], [d]] x.dims = [4, 1] and input ``y`` y.lod = [[3, 3, 1, 1]] #lod based on length may be easier to understand then we get 1level LoDTensor out: Out.lod = [[0, 3, 6, 7, 8]] #based on offset Out.data = [[a], [a], [a], [b], [b], [b], [c], [d]] Out.dims = [8, 1] Case 2: Given a common Tensor ``x``: x.data = [[a, b], [c, d], [e, f]] x.dims = [3, 2] and input ``y``: y.lod = [[0, 2, 3, 6]] then we get a 1level LoDTensor: out.lod = [[0, 2, 3, 6]] out.data = [[a, b], [a, b] [c, d], [e, f], [e, f], [e, f]] out.dims = [6, 2]
 Parameters
x (Variable) – The input variable which is a Tensor or LoDTensor, with the dims
[M, K]
. The data type should be float32, float64, int8, int32 or int64.y (Variable) – The input variable which is a LoDTensor with 1level lod.
name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
Returns: The expanded variable which is a LoDTensor with the dims
[N, K]
.N
depends on the lod ofy
, and the lod level must be 1. The data type is same as input.Return Type: Variable
Examples
import paddle.fluid as fluid import paddle.fluid.layers as layers import numpy as np x = fluid.data(name='x', shape=[4, 1], dtype='float32') y = fluid.data(name='y', shape=[8, 1], dtype='float32', lod_level=1) out = layers.sequence_expand_as(x=x, y=y) exe = fluid.Executor(fluid.CPUPlace()) place = fluid.CPUPlace() np_data = np.array([[1], [2], [3], [4]]).astype('float32') x_lod_tensor = fluid.create_lod_tensor(np_data, [[2, 2]], place) print(x_lod_tensor) #lod: [[0, 2, 4]] # dim: 4, 1 # layout: NCHW # dtype: float # data: [1 2 3 4] np_data = np.array([[1], [2], [3], [4], [5], [6], [7], [8]]).astype('float32') y_lod_tensor = fluid.create_lod_tensor(np_data, [[3,3,1,1]], place) print(y_lod_tensor) #lod: [[0, 3, 6, 7, 8]] # dim: 8, 1 # layout: NCHW # dtype: int64_t # data: [0 0 1 0 1 1 1 0] out_main = exe.run(fluid.default_main_program(), feed={'x': x_lod_tensor, 'y': y_lod_tensor}, fetch_list=[out], return_numpy=False) print(out_main[0]) #lod: [[0, 3, 6, 7, 8]] # dim: 8, 1 # layout: NCHW # dtype: float # data: [1 1 1 2 2 2 3 4]
sequence_first_step¶

paddle.fluid.layers.
sequence_first_step
(input)[source] This operator only supports LoDTensor as input. Given the input LoDTensor, it will select first timestep feature of each sequence as output.
Case 1: input is 1level LoDTensor: input.lod = [[0, 2, 5, 7]] input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]] input.shape = [7, 1] output is a LoDTensor: out.shape = [3, 1] out.shape[0] == len(x.lod[1]) == 3 out.data = [[1.], [2.], [5.]], where 1.=first(1., 3.), 2.=first(2., 4., 6.), 5.=first(5., 1.) Case 2: input is a 2level LoDTensor containing 3 sequences with length info [2, 0, 3], where 0 means empty sequence. The first sequence contains 2 subsequence with length info [1, 2]; The last sequence contains 3 subsequence with length info [1, 0, 3]. input.lod = [[0, 2, 2, 5], [0, 1, 3, 4, 4, 7]] input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]] input.shape = [7, 1] It will apply pooling on last lod_level [0, 1, 3, 4, 4, 7]. pad_value = 0.0 output is a LoDTensor: out.shape= [5, 1] out.lod = [[0, 2, 2, 5]] out.shape[0] == len(x.lod[1]) == 5 out.data = [[1.], [3.], [4.], [0.0], [6.]] where 1.=first(1.), 3.=first(3., 2.), 4.=first(4.), 0.0 = pad_value, 6.=first(6., 5., 1.)
 Parameters
input (Variable) – LoDTensor with lod_level no more than 2. The data type should be float32.
 Returns
LoDTensor consist of the sequence’s first step vector. The data type is float32.
 Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[None, 10], dtype='float32', lod_level=1) x_first_step = fluid.layers.sequence_first_step(input=x)
sequence_last_step¶

paddle.fluid.layers.
sequence_last_step
(input)[source] This operator only supports LoDTensor as input. Given the input LoDTensor, it will select last timestep feature of each sequence as output.
Case 1: input is 1level LoDTensor: input.lod = [[0, 2, 5, 7]] input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]] input.shape = [7, 1] output is a LoDTensor: out.shape = [3, 1] out.shape[0] == len(x.lod[1]) == 3 out.data = [[3.], [6.], [1.]], where 3.=last(1., 3.), 6.=last(2., 4., 6.), 1.=last(5., 1.) Case 2: input is a 2level LoDTensor containing 3 sequences with length info [2, 0, 3], where 0 means empty sequence. The first sequence contains 2 subsequence with length info [1, 2]; The last sequence contains 3 subsequence with length info [1, 0, 3]. input.lod = [[0, 2, 2, 5], [0, 1, 3, 4, 4, 7]] input.data = [[1.], [3.], [2.], [4.], [6.], [5.], [1.]] input.shape = [7, 1] It will apply pooling on last lod_level [0, 1, 3, 4, 4, 7]. pad_value = 0.0 output is a LoDTensor: out.shape= [5, 1] out.lod = [[0, 2, 2, 5]] out.shape[0] == len(x.lod[1]) == 5 out.data = [[1.], [2.], [4.], [0.0], [1.]] where 1.=last(1.), 2.=last(3., 2.), 4.=last(4.), 0.0 = pad_value, 1=last(6., 5., 1.)
 Parameters
input (Variable) – LoDTensor with lod_level no more than 2. The data type should be float32.
 Returns
LoDTensor consist of the sequence’s last step vector. The data type is float32.
 Return type
Variable
Examples
import paddle.fluid as fluid x = fluid.data(name='x', shape=[None, 10], dtype='float32', lod_level=1) x_last_step = fluid.layers.sequence_last_step(input=x)
sequence_mask¶

paddle.fluid.layers.
sequence_mask
(x, maxlen=None, dtype='int64', name=None)[source] SequenceMask Layer
This layer outputs a mask according to the input
x
andmaxlen
with data type ofdtype
.Supposing
x
is a Tensor with shape [d_1, d_2, …, d_n], they
is a mask with shape [d_1, d_2, …, d_n, maxlen], where:\[y(i_1, i_2,..., i_n, j) = (j < x(i_1, i_2,..., i_n))\]Case: Consider input: x = [3, 1, 1, 0] max_len = 4 then we get out: mask = [[1, 1, 1, 0], [1, 0, 0, 0], [1, 0, 0, 0], [0, 0, 0, 0]]
 Parameters
x (Variable) – Input tensor of sequence_mask layer, whose elements are integers less than
maxlen
. Tensor or LodTensor with shape [d_1, d_2, …, d_n].maxlen (int, optional) – Maximum length of the sequence. If
maxlen
is None, it would be replace with \(max(x)\).dtype (np.dtypecore.VarDesc.VarTypestr, optional) – Data type of the output,
int64
by default.name (str, optional) – For detailed information, please refer to api_guide_Name. Usually name is no need to set and None by default.
Returns: The output sequence mask. Tensor or LodTensor with shape [d_1, d_2, …, d_n, maxlen] and data type of
dtype
. The data type should be float32, float64, int8, int32 or int64.Return Type: Variable
Examples