shard_index

paddle. shard_index ( input, index_num, nshards, shard_id, ignore_value=- 1 ) [source]

Warning: API “paddle.fluid.layers.nn.shard_index” is deprecated since 2.0.0, and will be removed in future versions. Please use “paddle.shard_index” instead.

Recompute the input indices according to the offset of the shard. The length of the indices is evenly divided into N shards, and if the shard_id matches the shard with the input index inside, the index is recomputed on the basis of the shard offset, elsewise it is set to ignore_value. The detail is as follows:

shard_size = (index_num + nshards - 1) // nshards
y = x % shard_size if x // shard_size == shard_id else ignore_value

NOTE: If the length of indices cannot be evely divided by the shard number, the size of the last shard will be less than the calculated shard_size

Args:

input (Tensor): Input indices with data type int64. It’s last dimension must be 1. index_num (int): An integer defining the range of the index. nshards (int): The number of shards. shard_id (int): The index of the current shard. ignore_value (int): An integer value out of sharded index range.

Returns:

Tensor: The sharded index of input.

Examples:
import paddle
label = paddle.to_tensor([[16], [1]], "int64")
shard_label = paddle.shard_index(input=label,
                                 index_num=20,
                                 nshards=2,
                                 shard_id=0)
print(shard_label)
# [[-1], [1]]