BackwardStrategy¶

api_attr

imperative programming (dynamic graph)

class paddle.fluid.dygraph.BackwardStrategy

BackwardStrategy is a descriptor of how to run the backward process.

Note:

This API is only available in Dygraph Mode

Attribute:

sort_sum_gradient:

If framework will sum the gradient by the reverse order of trace. eg. x_var ( Variable ) will be the input of multiple OP such as scale , this attr will decide if framework will sum gradient of x_var by the reverse order.

By Default: False

Examples:

import numpy as np
import paddle.fluid as fluid

x = np.ones([2, 2], np.float32)
with fluid.dygraph.guard():
    x_var = fluid.dygraph.to_variable(x)
    sums_inputs = []
    # x_var will be multi-scales' input here
    for _ in range(10):
        sums_inputs.append(fluid.layers.scale(x_var))
    ret2 = fluid.layers.sums(sums_inputs)
    loss2 = fluid.layers.reduce_sum(ret2)
    backward_strategy = fluid.dygraph.BackwardStrategy()
    backward_strategy.sort_sum_gradient = True
    loss2.backward(backward_strategy)