extend_with_decoupled_weight_decay¶
- paddle.fluid.contrib.extend_optimizer.extend_optimizer_with_weight_decay. extend_with_decoupled_weight_decay ( base_optimizer ) [source]
-
extend_with_decoupled_weight_decay is a decorator function, it returns an optimizer class with decoupled weight decay. The returned optimizer will apply weight decay on the optimized parameters with the parameters before optimization, i.e: new_parameter = optimized_parameter - parameter * coeff. The details of decoupled weight decay yplease refer to this DECOUPLED WEIGHT DECAY REGULARIZATION.
- Parameters
-
base_optimizer (Optimizer) – The base_optimizer should be a derived class of Optimizer.
- Returns
-
the optimizer with decouple weight decay.
- Return type
-
OptimizerWithDecoupledWeightDecay
Examples
AdamW = fluid.contrib.extend_with_decoupled_weight_decay( fluid.optimizer.Adam) optimizer = AdamW(learning_rate=0.1, weight_decay=0.01) optimizer.minimize(cost)