cosine_decay¶
- paddle.fluid.layers.learning_rate_scheduler. cosine_decay ( learning_rate, step_each_epoch, epochs ) [source]
-
Applies cosine decay to the learning rate.
when training a model, it is often recommended to lower the learning rate as the training progresses. By using this function, the learning rate will be decayed by following cosine decay strategy.
decayed_lr=learning_rate∗0.5∗(math.cos∗(epoch∗fracmath.piepochs)+1)- Parameters
-
learning_rate (Variable|float) – The initial learning rate.
step_each_epoch (int) – the number of steps in an epoch.
epochs (int) – the number of epochs.
- Returns
-
The decayed learning rate.
- Return type
-
Variable
Examples
import paddle.fluid as fluid base_lr = 0.1 lr = fluid.layers.cosine_decay( learning_rate = base_lr, step_each_epoch=10000, epochs=120)