extend_with_decoupled_weight_decay¶

paddle.fluid.contrib.extend_optimizer.extend_optimizer_with_weight_decay. extend_with_decoupled_weight_decay ( base_optimizer ) [source]

extend_with_decoupled_weight_decay is a decorator function, it returns an optimizer class with decoupled weight decay. The returned optimizer will apply weight decay on the optimized parameters with the parameters before optimization, i.e: new_parameter = optimized_parameter - parameter * coeff. The details of decoupled weight decay yplease refer to this DECOUPLED WEIGHT DECAY REGULARIZATION.

Parameters

base_optimizer (Optimizer) – The base_optimizer should be a derived class of Optimizer.

Returns

the optimizer with decouple weight decay.

Return type

OptimizerWithDecoupledWeightDecay

Examples

AdamW = fluid.contrib.extend_with_decoupled_weight_decay(
    fluid.optimizer.Adam)
optimizer = AdamW(learning_rate=0.1,
                  weight_decay=0.01)

optimizer.minimize(cost)