cosine_decay

paddle.fluid.layers.learning_rate_scheduler. cosine_decay ( learning_rate, step_each_epoch, epochs ) [source]

Applies cosine decay to the learning rate.

when training a model, it is often recommended to lower the learning rate as the training progresses. By using this function, the learning rate will be decayed by following cosine decay strategy.

decayed_lr=learning_rate0.5(math.cos(epochfracmath.piepochs)+1)
Parameters
  • learning_rate (Variable|float) – The initial learning rate.

  • step_each_epoch (int) – the number of steps in an epoch.

  • epochs (int) – the number of epochs.

Returns

The decayed learning rate.

Return type

Variable

Examples

import paddle.fluid as fluid
base_lr = 0.1
lr = fluid.layers.cosine_decay(
learning_rate = base_lr, step_each_epoch=10000, epochs=120)