QueueDataset

class paddle.distributed. QueueDataset [source]
Api_attr

Static Graph

QueueDataset, it will process data streamly.

Examples

>>> import paddle
>>> dataset = paddle.distributed.QueueDataset()
init ( **kwargs )

init

Api_attr

Static Graph

should be called only once in user’s python scripts to initialize settings of dataset instance

set_filelist ( filelist )

set_filelist

Set file list in current worker. The filelist is indicated by a list of file names (string).

Examples

>>> import paddle
>>> dataset = paddle.distributed.fleet.DatasetBase()
>>> dataset.set_filelist(['a.txt', 'b.txt'])
Parameters

filelist (list[str]) – list of file names of inputs.

Used in the guide/tutorials