torch.Tensor.is_leaf#

Tensor.is_leaf#

All Tensors that haverequires_grad which isFalse will be leaf Tensors by convention.

For Tensors that haverequires_grad which isTrue, they will be leaf Tensors if they werecreated by the user. This means that they are not the result of an operation and sograd_fn is None.

Only leaf Tensors will have theirgrad populated during a call tobackward().To getgrad populated for non-leaf Tensors, you can useretain_grad().

Example:

>>>a=torch.rand(10,requires_grad=True)>>>a.is_leafTrue>>>b=torch.rand(10,requires_grad=True).cuda()>>>b.is_leafFalse# b was created by the operation that cast a cpu Tensor into a cuda Tensor>>>c=torch.rand(10,requires_grad=True)+2>>>c.is_leafFalse# c was created by the addition operation>>>d=torch.rand(10).cuda()>>>d.is_leafTrue# d does not require gradients and so has no operation creating it (that is tracked by the autograd engine)>>>e=torch.rand(10).cuda().requires_grad_()>>>e.is_leafTrue# e requires gradients and has no operations creating it>>>f=torch.rand(10,requires_grad=True,device="cuda")>>>f.is_leafTrue# f requires grad, has no operation creating it