Gradient auto-computing in Pytorch

Pytorch is a powerful Python package designed for Machine Learning purposes. One of the useful techniques is that we can calculate the gradient automatically using the backward method of the target value. The following example may be the simplest example explaining the syntaxes.

x = torch.tensor(3., requires_grad=True)
for i in  range(5):
	x.grad = torch.tensor(0.)
	y = x**2
	y.backward()
	print(x.grad)

The result will be

tensor(6.)
tensor(6.)
tensor(6.)
tensor(6.)
tensor(6.)

Remark: You may wonder why we need to set x.grad = torch.tensor(0.). Simply because if we remove this line, the gradient will be calculated cumulatively and the result is as follows

tensor(6.)
tensor(12.)
tensor(18.)
tensor(24.)
tensor(30.)