ExponentialMovingAverage¶

API属性：声明式编程(静态图)专用API

class paddle.fluid.optimizer. ExponentialMovingAverage ( decay=0.999, thres_steps=None, name=None ) [源代码]

\begin{split}\begin{align}\begin{aligned}\text{EMA}_0 & = 0\\\text{EMA}_t & = \text{decay} * \text{EMA}_{t-1} + (1 - \text{decay}) * \theta_t\end{aligned}\end{align}\end{split}

update() 方法计算出的平均结果将保存在由实例化对象创建和维护的临时变量中，并且可以通过调用 apply() 方法把结果应用于当前模型的参数。同时，可用 restore() 方法恢复原始参数。

$\widehat{\text{EMA}}_t = \frac{\text{EMA}_t}{1 - \text{decay}^t}$

$\min(\text{decay}, \frac{1 + \text{thres_steps}}{10 + \text{thres_steps}})$

参数¶

• decay (float) – 指数衰减率，通常接近1，如0.999，0.9999，……
• thres_steps (Variable, 可选) – 调节衰减率的阈值步数，默认值为 None。
• name (str，可选) – 具体用法请参见 cn_api_guide_Name ，一般无需设置，默认值为None。

代码示例¶

import numpy

data = fluid.layers.data(name='x', shape=[5], dtype='float32')
hidden = fluid.layers.fc(input=data, size=10)
cost = fluid.layers.mean(hidden)

test_program = fluid.default_main_program().clone(for_test=True)

optimizer.minimize(cost)

global_steps = fluid.layers.learning_rate_scheduler._decay_step_counter()
ema = fluid.optimizer.ExponentialMovingAverage(0.999, thres_steps=global_steps)
ema.update()

place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

for pass_id in range(3):
for batch_id in range(6):
data = numpy.random.random(size=(10, 5)).astype('float32')
exe.run(program=fluid.default_main_program(),
feed={'x': data},
fetch_list=[cost.name])

# usage 1
with ema.apply(exe):
data = numpy.random.random(size=(10, 5)).astype('float32')
exe.run(program=test_program,
feed={'x': data},
fetch_list=[hidden.name])

# usage 2
with ema.apply(exe, need_restore=False):
data = numpy.random.random(size=(10, 5)).astype('float32')
exe.run(program=test_program,
feed={'x': data},
fetch_list=[hidden.name])
ema.restore(exe)

方法¶

apply(executor, need_restore=True)¶

• executor (Executor) – 将滑动平均值作用在参数上的执行器。
• need_restore (bool) –是否在结束后恢复原始参数，默认值为 True

restore(executor)¶

• executor (Executor) – 执行恢复动作的执行器。