PiecewiseDecay( boundaries, values, begin, step=1, dtype='float32' )
Piecewise decay scheduler.
The algorithm can be described as the code below.
boundaries = [10000, 20000] values = [1.0, 0.5, 0.1] if global_step < 10000: learning_rate = 1.0 elif 10000 <= global_step < 20000: learning_rate = 0.5 else: learning_rate = 0.1
boundaries (list) – A list of steps numbers. The type of element in the list is python int.
values (list) – A list of learning rate values that will be picked during different step boundaries. The type of element in the list is python float.
begin (int) – The begin step to initialize the global_step in the description above.
step (int, optional) – The step size used to calculate the new global_step in the description above. The default value is 1.
dtype (str, optional) – The data type used to create the learning rate variable. The data type can be set as ‘float32’, ‘float64’. The default value is ‘float32’.
import paddle.fluid as fluid boundaries = [10000, 20000] values = [1.0, 0.5, 0.1] with fluid.dygraph.guard(): emb = fluid.dygraph.Embedding( [10, 10] ) optimizer = fluid.optimizer.SGD( learning_rate=fluid.dygraph.PiecewiseDecay(boundaries, values, 0), parameter_list = emb.parameters() )