It is essential to understand all the basic concepts which are required to work with PyTorch. PyTorch is completely based on Tensors. Tensor has operations to perform. Apart from these, there are lots of other concepts which are required to perform the task.
Now, understand all the concepts one by one to gain deep knowledge of PyTorch.
Matrices or Tensors
Tensors are the key components of Pytorch. We can say PyTorch is completely based on the Tensors. In mathematical term, a rectangular array of number is called a metrics. In the Numpy library, these metrics called ndaaray. In PyTorch, it is known as Tensor. A Tensor is an n-dimensional data container. For example, In PyTorch, 1d-Tensor is a vector, 2d-Tensor is a metrics, 3d- Tensor is a cube, and 4d-Tensor is a cube vector.
Above matrics represent 2D-Tensor with three rows and two columns.
There are three ways to create Tensor. Each one has a different way to create Tensor. Tensors are created as:
Let see how Tensors are created
Create a PyTorch Tensor as an array
In this, you have first to define the array and then pass that array in your Tensor method of the torch as an argument.
tensor ([[3., 4.],[8., 5.]])
Create a Tensor with the random number and all one
To create a random number Tensor, you have to use rand() method and to create a Tensor with all ones you have to use ones() of the torch. To generate random number one more method of the torch will be used with the rand, i.e., manual_seed with 0 parameters.
Tensor ([[1., 1.],[1., 1.]]) tensor ([[0.4963, 0.7682],[0.0885, 0.1320]])
Create a Tensor from numpy array
To create a Tensor from the numpy array, we have to create a numpy array. Once your numpy array is created, we have to pass it in from_numpy() as an argument. This method converts the numpy array to Tensor.
[[1. 1.] [1. 1.]]
Tensors are similar to an array, so all the operation which we are performing on an array can also apply for Tensor.
1) Resizing a Tensor
We can resize the Tensor using the size property of Tensor. We use Tensor.view() for resizing a Tensor. Resizing a Tensor means the conversion of 2*2 dimensional Tensor to 4*1 or 4*4 dimensional Tensor to 16*1 and so on. To print the Tensor size, we use Tensor.size() method.
Let see an example of resizing a Tensor.
torch.Size ([2, 2]) tensor ([1., 1., 1., 1.])
2) Mathematical Operations
All the mathematical operation such as addition, subtraction, division, and multiplication can be performed on Tensor. The torch can do the mathematical operation. We use a torch.add(), torch.sub(), torch.mul() and torch.div() to perform operations on Tensors.
Let see an example how mathematical operations are performed:
tensor ([[2., 2.], [2., 2.]]) tensor ([[2., 2.], [2., 2.]])
3) Mean and Standard deviation
We can calculate the standard deviation of Tensor either for one dimensional or multi-dimensional. In our mathematical calculation, we have first to calculate mean, and then we apply the following formula on the given data with mean.
But in Tensor, we can use Tensor.mean() and Tensor.std() to find the deviation and mean of the given Tensor.
Let see an example of how it performed.
tensor (3.) tensor (1.5811)
Variables and Gradient
The central class of the package is autograd.variable. Its main task is to wrap a Tensor. It supports nearly all of the operations defined on it. You can call .backword() and have all the gradient computed only when you finish your computation.
Through .data attribute, you can access the row Tensor, while the gradient for this variable is accumulated into .grad.
In Deep learning, gradient calculation is the key point. Variables are used to calculate the gradient in PyTorch. In simple words, variables are just a wrapper around Tensors with gradient calculation functionality.
Below is the python code which is used to manage variables.
Above code behaves the same as Tensors, so that we can apply all operations in the same way.
Let see how we can calculate the gradient in PyTorch.