Training of Linear Model

We plotted our linear model given the random parameters that were assigned to it. We found that it does not fit our data well. What we have to do. We need to train this model so that the model has the optimal weight and bias parameters and fit this data.

There are the following steps to train a model:

Step 1

Our first step is to specify the loss function, which we intend to minimize. PyTorch provides a very efficient way to specify the lost function. PyTorch provides MSELoss() function, known as mean squared loss, to calculate loss as

Step 2

Now, our next step is to update our parameters. For this purpose, we specify the optimizer that uses the gradient descent algorithm. We use SGD() function known as stochastic gradient descent for optimization. SGD minimizes the total loss one sample at a time and typically reaches convergence much faster as it will frequently update the weight of our model within the same sample size.

Here, lr stands for learning rate, which is initially set to 0.01.

Step 3

We will train our model for a specified number of epochs (We calculated the error function and backpropagated the gradient descent of this error function to update the weight).

And now, for every epoch, we have to minimize the error of our model system. The error is simply a comparison between the prediction made by the model and the actual values.

Losses=[]
For i in range (epochs):
	ypred=model.forward(x)	#Prediction of y
	loss=criterion(ypred,y)	#Find loss
	losses.append()		# Add loss in list 
	optimizer.zero_grad() # Set the gradient to zero
	loss.backward()	#To compute derivatives 
	optimizer.step()	# Update the parameters 

Step 4

Now, at last, we plot our new linear model by simply calling plotfit () method.

Complete Code

Program

import torch
import torch.nn as nn
import matplotlib.pyplot as plt
import numpy as np
X=torch.randn(100,1)*10
y=X+3*torch.randn(100,1)
plt.plot(X.numpy(),y.numpy(),'o')
plt.ylabel('y')
plt.xlabel('x')
class LR(nn.Module):
	def __init__(self,input_size,output_size):
		super().__init__()
		self.linear=nn.Linear(input_size,output_size)
	def forward(self,x):
		pred=self.linear(X)
		return pred	
torch.manual_seed(1)	#For consistency of random result 
model=LR(1,1)
criterion=nn.MSELoss()	#Using Loss Function
optimizer=torch.optim.SGD(model.parameters(),lr=0.01)  #Using optimizer which uses GD algorithm
print(model)
[a,b]=model.parameters()	#Unpacking of parameters
epochs=100
losses=[]
for i in range(epochs):
	ypred=model.forward(X)
	loss=criterion(ypred,y)
	print("epoch:",i,"loss:",loss.item())
	losses.append(loss)
	optimizer.zero_grad()
	loss.backward()
	optimizer.step()
defgrtparameters():
       return(a[0][0].item(),b[0].item())
defplotfit(title):
	plt.title=title
	a1,b1=grtparameters()
	x1=np.array([-30,30])
	y1=a1*x1+b1
	plt.plot(x1,y1,'r')
	plt.scatter(X,y)
	plt.show()
plotfit('Trained Model')

Output: