Applies piecewise decay to the initial learning rate.
The algorithm can be described as the code below.
boundaries = [10000, 20000] values = [1.0, 0.5, 0.1] if step < 10000: learning_rate = 1.0 elif 10000 <= step < 20000: learning_rate = 0.5 else: learning_rate = 0.1
boundaries – A list of steps numbers.
values – A list of learning rate values that will be picked during different step boundaries.
The decayed learning rate.
import paddle.fluid as fluid boundaries = [10000, 20000] values = [1.0, 0.5, 0.1] optimizer = fluid.optimizer.Momentum( momentum=0.9, learning_rate=fluid.layers.piecewise_decay(boundaries=boundaries, values=values), regularization=fluid.regularizer.L2Decay(1e-4))