- paddle. shard_index ( input, index_num, nshards, shard_id, ignore_value=- 1 )
Warning: API “paddle.fluid.layers.nn.shard_index” is deprecated since 2.0.0, and will be removed in future versions. Please use “paddle.shard_index” instead.
Recompute the input indices according to the offset of the shard. The length of the indices is evenly divided into N shards, and if the shard_id matches the shard with the input index inside, the index is recomputed on the basis of the shard offset, elsewise it is set to ignore_value. The detail is as follows:
shard_size = (index_num + nshards - 1) // nshards y = x % shard_size if x // shard_size == shard_id else ignore_value
NOTE: If the length of indices cannot be evely divided by the shard number, the size of the last shard will be less than the calculated shard_size
input (Tensor): Input indices with data type int64. It’s last dimension must be 1. index_num (int): An integer defining the range of the index. nshards (int): The number of shards. shard_id (int): The index of the current shard. ignore_value (int): An integer value out of sharded index range.
Tensor: The sharded index of input.
import paddle label = paddle.to_tensor([, ], "int64") shard_label = paddle.shard_index(input=label, index_num=20, nshards=2, shard_id=0) print(shard_label) # [[-1], ]