CosineDecay¶

class paddle.fluid.dygraph.learning_rate_scheduler. CosineDecay ( learning_rate, step_each_epoch, epochs, begin=0, step=1, dtype='float32' ) [source]

Api_attr

imperative

Applies cosine decay to the learning rate.

The algorithm can be described as following.

\[\begin{split}decayed\_learning\_rate = learning\_rate * 0.5 * (math.cos(global\_step * \\frac{math.pi}{step\_each\_epoch} ) + 1)\end{split}\]

Parameters

• learning_rate (Variable|float) – The initial learning rate. If the type is Variable, it’s a tensor with shape [1], the data type can be float32 or float64. It also can be set to python int number.

• step_each_epoch (int) – The number of steps in an epoch.

• epochs (int) – The number of epochs.

• begin (int, optional) – The begin step. The initial value of global_step described above. The default value is 0.

• step (int, optional) – The step size used to calculate the new global_step in the description above. The default value is 1.

• dtype (str, optional) – The data type used to create the learning rate variable. The data type can be set as ‘float32’, ‘float64’. The default value is ‘float32’.

Returns

None.

Examples

base_lr = 0.1
with fluid.dygraph.guard():
    optimizer  = fluid.optimizer.SGD(
        learning_rate = fluid.dygraph.CosineDecay(
                base_lr, 10000, 120) )

create_lr_var ( lr )

create_lr_var¶

convert lr from float to variable

Parameters

lr – learning rate

Returns

learning rate variable

set_dict ( state_dict )

set_dict¶

Loads the schedulers state.

set_state_dict ( state_dict )

set_state_dict¶

Loads the schedulers state.

state_dict ( )

state_dict¶

Returns the state of the scheduler as a dict.

It is a subset of self.__dict__ .