TracedLayer¶
- api_attr
imperative programming (dynamic graph)
-
class
paddle.fluid.dygraph.
TracedLayer
(program, parameters, feed_names, fetch_names)[source] TracedLayer is used to convert a forward dygraph model to a static graph model. This is mainly used to save the dygraph model for online inference using C++. Besides, users can also do inference in Python using the converted static graph model, which usually has better performance than the original dygraph model.
TracedLayer would run the static graph model using
Executor
andCompiledProgram
. The static graph model would share parameters with the dygraph model.All TracedLayer objects should not be created by constructor and should be created by static method
TracedLayer.trace(layer, inputs)
.The TracedLayer can only be used to convert the data-independent dygraph model into the static graph model, which means the dygraph model should be independent with the tensor data and shape.
-
static
trace
(layer, inputs) This method is the only allowed method to create TracedLayer object. It would call the
layer(*inputs)
method to run the dygraph model and convert it into a static graph model.- Parameters
layer (dygraph.Layer) – the layer object to be traced.
inputs (list(Variable)) – the input variables of the layer object.
- Returns
A tuple of 2 items, whose the first item is the output of
layer(*inputs)
, and the second item is the created TracedLayer object.- Return type
tuple
Examples
import paddle.fluid as fluid from paddle.fluid.dygraph import Linear, to_variable, TracedLayer import numpy as np class ExampleLayer(fluid.dygraph.Layer): def __init__(self): super(ExampleLayer, self).__init__() self._fc = Linear(3, 10) def forward(self, input): return self._fc(input) with fluid.dygraph.guard(): layer = ExampleLayer() in_np = np.random.random([2, 3]).astype('float32') in_var = to_variable(in_np) out_dygraph, static_layer = TracedLayer.trace(layer, inputs=[in_var]) # run the static graph model using Executor inside out_static_graph = static_layer([in_var]) print(len(out_static_graph)) # 1 print(out_static_graph[0].shape) # (2, 10) # save the static graph model for inference static_layer.save_inference_model(dirname='./saved_infer_model')
-
set_strategy
(build_strategy=None, exec_strategy=None) Set the strategies when running static graph model.
- Parameters
build_strategy (BuildStrategy, optional) – build strategy of
CompiledProgram
inside TracedLayer. Default None.exec_strategy (ExecutionStrategy, optional) – execution strategy of
CompiledProgram
inside TracedLayer. Default None.
- Returns
None
Examples
import paddle.fluid as fluid from paddle.fluid.dygraph import Linear, to_variable, TracedLayer import numpy as np class ExampleLayer(fluid.dygraph.Layer): def __init__(self): super(ExampleLayer, self).__init__() self._fc = Linear(3, 10) def forward(self, input): return self._fc(input) with fluid.dygraph.guard(): layer = ExampleLayer() in_np = np.random.random([2, 3]).astype('float32') in_var = to_variable(in_np) out_dygraph, static_layer = TracedLayer.trace(layer, inputs=[in_var]) build_strategy = fluid.BuildStrategy() build_strategy.enable_inplace = True exec_strategy = fluid.ExecutionStrategy() exec_strategy.num_threads = 2 static_layer.set_strategy(build_strategy=build_strategy, exec_strategy=exec_strategy) out_static_graph = static_layer([in_var])
-
save_inference_model
(dirname, feed=None, fetch=None) Save the TracedLayer to a model for inference. The saved inference model can be loaded by C++ inference APIs.
- Parameters
dirname (str) – the directory to save the inference model.
feed (list[int], optional) – the input variable indices of the saved inference model. If None, all input variables of the TracedLayer object would be the inputs of the saved inference model. Default None.
fetch (list[int], optional) – the output variable indices of the saved inference model. If None, all output variables of the TracedLayer object would be the outputs of the saved inference model. Default None.
- Returns
None
Examples
import paddle.fluid as fluid from paddle.fluid.dygraph import Linear, to_variable, TracedLayer import numpy as np class ExampleLayer(fluid.dygraph.Layer): def __init__(self): super(ExampleLayer, self).__init__() self._fc = Linear(3, 10) def forward(self, input): return self._fc(input) save_dirname = './saved_infer_model' in_np = np.random.random([2, 3]).astype('float32') with fluid.dygraph.guard(): layer = ExampleLayer() in_var = to_variable(in_np) out_dygraph, static_layer = TracedLayer.trace(layer, inputs=[in_var]) static_layer.save_inference_model(save_dirname, feed=[0], fetch=[0]) place = fluid.CPUPlace() exe = fluid.Executor(place) program, feed_vars, fetch_vars = fluid.io.load_inference_model(save_dirname, exe) fetch, = exe.run(program, feed={feed_vars[0]: in_np}, fetch_list=fetch_vars) print(fetch.shape) # (2, 10)
-
static