QueueDataset¶

class paddle.distributed.fleet.dataset. QueueDataset [source]

Api_attr

Static Graph

QueueDataset, it will process data streamly.

Examples

import paddle
dataset = paddle.distributed.QueueDataset()

init ( **kwargs )

Api_attr

Static Graph

should be called only once in user’s python scripts to initialize setings of dataset instance

set_filelist ( filelist )

Set file list in current worker. The filelist is indicated by a list of file names (string).

Examples

import paddle
dataset = paddle.distributed.fleet.DatasetBase()
dataset.set_filelist(['a.txt', 'b.txt'])

Parameters

filelist (list[str]) – list of file names of inputs.