create_py_reader_by_data¶
- paddle.fluid.layers.io. create_py_reader_by_data ( capacity, feed_list, name=None, use_double_buffer=True ) [source]
-
- api_attr
-
Static Graph
The OP creates a Python reader for data feeding in Python, it is similar to api_fluid_layers_py_reader except that it can read data from the list of feed variables.
- Parameters
-
capacity (int) – The buffer capacity maintained by
py_reader
. Its unit is batch number. Set largercapacity
if the reader is fast.feed_list (list(Variable)) – The feed variables, are usually created by
fluid.data()
.name (str, optional) – Normally there is no need for user to set this property. For more information, please refer to Name. Default: None.
use_double_buffer (bool, optional) – Whether use double buffer. If it’s True, the OP would prefetch next batch data asynchronously. Default: True.
- Returns
-
A Reader for data feeding. The data types of read data are the same as the data types of variables of
feed_list
. - Return type
-
Reader
Examples
import paddle import paddle.fluid as fluid import paddle.dataset.mnist as mnist def network(img, label): # User defined network. Here a simple regression as example predict = fluid.layers.fc(input=img, size=10, act='softmax') loss = fluid.layers.cross_entropy(input=predict, label=label) return fluid.layers.mean(loss) MEMORY_OPT = False USE_CUDA = False image = fluid.data(name='image', shape=[None, 1, 28, 28], dtype='float32') label = fluid.data(name='label', shape=[None, 1], dtype='int64') reader = fluid.layers.create_py_reader_by_data(capacity=64, feed_list=[image, label]) reader.decorate_paddle_reader( paddle.reader.shuffle(paddle.batch(mnist.train(), batch_size=5), buf_size=500)) img, label = fluid.layers.read_file(reader) loss = network(img, label) # The definition of custom network and the loss function place = fluid.CUDAPlace(0) if USE_CUDA else fluid.CPUPlace() exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) build_strategy = fluid.BuildStrategy() build_strategy.memory_optimize = True if MEMORY_OPT else False exec_strategy = fluid.ExecutionStrategy() compiled_prog = fluid.compiler.CompiledProgram( fluid.default_main_program()).with_data_parallel( loss_name=loss.name, build_strategy=build_strategy, exec_strategy=exec_strategy) for epoch_id in range(2): reader.start() try: while True: exe.run(compiled_prog, fetch_list=[loss.name]) except fluid.core.EOFException: reader.reset()