PyLayerContext

class paddle.autograd. PyLayerContext [source]

PyLayerContext can assist the PyLayer in implementing certain functionalities.

Examples

>>> import paddle
>>> from paddle.autograd import PyLayer

>>> class cus_tanh(PyLayer):
...     @staticmethod
...     def forward(ctx, x):
...         # ctx is a object of PyLayerContext.
...         y = paddle.tanh(x)
...         ctx.save_for_backward(y)
...         return y
...
...     @staticmethod
...     def backward(ctx, dy):
...         # ctx is a object of PyLayerContext.
...         y, = ctx.saved_tensor()
...         grad = dy * (1 - paddle.square(y))
...         return grad
save_for_backward ( *tensors )

save_for_backward

Saves given tensors that backward need. Use saved_tensor in the backward to get the saved tensors.

Note

This API should be called at most once, and only inside forward.

Parameters

tensors (list of Tensors) – Tensors to be stored.

Returns

None

Examples

>>> import paddle
>>> from paddle.autograd import PyLayer

>>> class cus_tanh(PyLayer):
...     @staticmethod
...     def forward(ctx, x):
...         # ctx is a context object that store some objects for backward.
...         y = paddle.tanh(x)
...         # Pass tensors to backward.
...         ctx.save_for_backward(y)
...         return y
...
...     @staticmethod
...     def backward(ctx, dy):
...         # Get the tensors passed by forward.
...         y, = ctx.saved_tensor()
...         grad = dy * (1 - paddle.square(y))
...         return grad
saved_tensor ( )

saved_tensor

Get the tensors stored by save_for_backward.

Returns

If context contains tensors stored by save_for_backward, then return these tensors, otherwise return None.

Return type

list of Tensors or None

Examples

>>> import paddle
>>> from paddle.autograd import PyLayer

>>> class cus_tanh(PyLayer):
...     @staticmethod
...     def forward(ctx, x):
...         # ctx is a context object that store some objects for backward.
...         y = paddle.tanh(x)
...         # Pass tensors to backward.
...         ctx.save_for_backward(y)
...         return y
...
...     @staticmethod
...     def backward(ctx, dy):
...         # Get the tensors passed by forward.
...         y, = ctx.saved_tensor()
...         grad = dy * (1 - paddle.square(y))
...         return grad
mark_not_inplace ( *args )

mark_not_inplace

Marks inputs as not inplace. This should be called at most once, only from inside the forward method, and all arguments should be Tensor inputs.

If the Tensor returned by forward method is the same as the Tensor input of forward, and this Tensor is marked as not_inplace, then Paddle will help the user create a new Tensor as output. Thereby preventing the auto grad information of the input Tensor from being overwritten.

Examples

>>> import paddle

>>> class Exp(paddle.autograd.PyLayer):
...     @staticmethod
...     def forward(ctx, x):
...         ctx.mark_not_inplace(x)
...         return x
...
...     @staticmethod
...     def backward(ctx, grad_output):
...         out = grad_output.exp()
...         return out

>>> paddle.seed(2023)
>>> x = paddle.randn((1, 1))
>>> x.stop_gradient = False
>>> attn_layers = []
>>> for idx in range(0, 2):
...     attn_layers.append(Exp())

>>> for step in range(0, 2):
...     a = x
...     for j in range(0,2):
...         a = attn_layers[j].apply(x)
...     a.backward()
mark_non_differentiable ( *args )

mark_non_differentiable

Marks outputs as non-differentiable. This should be called at most once, only from inside the forward method, and all arguments should be tensor outputs.

This will mark outputs as not requiring gradients, increasing the efficiency of backward computation. You still need to accept a gradient for each output in backward, but it’s always going to be a zero tensor with the same shape as the shape of a corresponding output.

Examples

>>> import paddle
>>> from paddle.autograd import PyLayer
>>> import numpy as np

>>> class Tanh(PyLayer):
...     @staticmethod
...     def forward(ctx, x):
...         a = x + x
...         b = x + x + x
...         ctx.mark_non_differentiable(a)
...         return a, b
...
...     @staticmethod
...     def backward(ctx, grad_a, grad_b):
...         assert np.equal(grad_a.numpy(), paddle.zeros([1]).numpy())
...         assert np.equal(grad_b.numpy(), paddle.ones([1], dtype="float64").numpy())
...         return grad_b

>>> x = paddle.ones([1], dtype="float64")
>>> x.stop_gradient = False
>>> a, b = Tanh.apply(x)
>>> b.sum().backward()
set_materialize_grads ( value: bool )

set_materialize_grads

Sets whether to materialize output grad tensors. Default is True.

This should be called only from inside the forward method.

If True, undefined output grad tensors will be expanded to tensors full of zeros prior to calling the backward method.

If False, undefined output grad tensors will be None.

Examples

>>> import paddle
>>> from paddle.autograd import PyLayer
>>> import numpy as np

>>> class Tanh(PyLayer):
...     @staticmethod
...     def forward(ctx, x):
...         return x+x+x, x+x
...
...     @staticmethod
...     def backward(ctx, grad, grad2):
...         assert np.equal(grad2.numpy(), paddle.zeros([1]).numpy())
...         return grad

>>> class Tanh2(PyLayer):
...     @staticmethod
...     def forward(ctx, x):
...         ctx.set_materialize_grads(False)
...         return x+x+x, x+x
...
...     @staticmethod
...     def backward(ctx, grad, grad2):
...         assert grad2==None
...         return grad

>>> x = paddle.ones([1], dtype="float64")
>>> x.stop_gradient = False
>>> Tanh.apply(x)[0].backward()

>>> x2 = paddle.ones([1], dtype="float64")
>>> x2.stop_gradient = False
>>> Tanh2.apply(x2)[0].backward()