LambdaDecay¶
-
class
paddle.fluid.dygraph.learning_rate_scheduler.
LambdaDecay
( learning_rate, lr_lambda ) [source] -
- Api_attr
-
imperative
Sets the learning rate of
optimizer
to the initial lr times a multiplicative factor, and this multiplicative factor is computed by functionlr_lambda
.lr_lambda
is funciton which receivesepoch
.The algorithm can be described as the code below.
learning_rate = 0.5 # init learning_rate lr_lambda = lambda epoch: 0.95 ** epoch learning_rate = 0.5 # epoch 0 learning_rate = 0.475 # epoch 1 learning_rate = 0.45125 # epoch 2
- Parameters
-
learning_rate (float|int) – The initial learning rate. It can be set to python float or int number.
lr_lambda (function) – A function which computes a multiplicative factor given an integer parameter
epoch
, and then multiply the initial learning rate by this multiplicative factor.
- Returns
-
None.
Examples
import paddle.fluid as fluid import numpy as np with fluid.dygraph.guard(): x = np.random.uniform(-1, 1, [10, 10]).astype("float32") linear = fluid.dygraph.Linear(10, 10) input = fluid.dygraph.to_variable(x) scheduler = fluid.dygraph.LambdaDecay(0.5, lr_lambda=lambda x: 0.95**x) adam = fluid.optimizer.Adam(learning_rate = scheduler, parameter_list = linear.parameters()) for epoch in range(6): for batch_id in range(5): out = linear(input) loss = fluid.layers.reduce_mean(out) adam.minimize(loss) scheduler.epoch() print("epoch:%d, current lr is %f" .format(epoch, adam.current_step_lr())) # epoch:0, current lr is 0.5 # epoch:1, current lr is 0.475 # epoch:2, current lr is 0.45125