Single Layer Perceptron in TensorFlow

The perceptron is a single processing unit of any neural network. Frank Rosenblatt first proposed in 1958 is a simple neuron which is used to classify its input into one or two categories. Perceptron is a linear classifier, and is used in supervised learning. It helps to organize the given input data.

A perceptron is a neural network unit that does a precise computation to detect features in the input data. Perceptron is mainly used to classify the data into two parts. Therefore, it is also known as Linear Binary Classifier.

Perceptron uses the step function that returns +1 if the weighted sum of its input 0 and -1.

The activation function is used to map the input between the required value like (0, 1) or (-1, 1).

A regular neural network looks like this:

The perceptron consists of 4 parts.

Input value or One input layer: The input layer of the perceptron is made of artificial input neurons and takes the initial data into the system for further processing.
Weights and Bias:
Weight: It represents the dimension or strength of the connection between units. If the weight to node 1 to node 2 has a higher quantity, then neuron 1 has a more considerable influence on the neuron.
Bias: It is the same as the intercept added in a linear equation. It is an additional parameter which task is to modify the output along with the weighted sum of the input to the other neuron.
Net sum: It calculates the total sum.
Activation Function: A neuron can be activated or not, is determined by an activation function. The activation function calculates a weighted sum and further adding bias with it to give the result.

A standard neural network looks like the below diagram.

How does it work?

The perceptron works on these simple steps which are given below:

a. In the first step, all the inputs x are multiplied with their weights w.

b. In this step, add all the increased values and call them the Weighted sum.

c. In our last step, apply the weighted sum to a correct Activation Function.

For Example:

A Unit Step Activation Function

There are two types of architecture. These types focus on the functionality of artificial neural networks as follows-

Single Layer Perceptron
Multi-Layer Perceptron

Single Layer Perceptron

The single-layer perceptron was the first neural network model, proposed in 1958 by Frank Rosenbluth. It is one of the earliest models for learning. Our goal is to find a linear decision function measured by the weight vector w and the bias parameter b.

To understand the perceptron layer, it is necessary to comprehend artificial neural networks (ANNs).

The artificial neural network (ANN) is an information processing system, whose mechanism is inspired by the functionality of biological neural circuits. An artificial neural network consists of several processing units that are interconnected.

This is the first proposal when the neural model is built. The content of the neuron's local memory contains a vector of weight.

The single vector perceptron is calculated by calculating the sum of the input vector multiplied by the corresponding element of the vector, with each increasing the amount of the corresponding component of the vector by weight. The value that is displayed in the output is the input of an activation function.

Let us focus on the implementation of a single-layer perceptron for an image classification problem using TensorFlow. The best example of drawing a single-layer perceptron is through the representation of "logistic regression."

Now, We have to do the following necessary steps of training logistic regression-

The weights are initialized with the random values at the origination of each training.
For each element of the training set, the error is calculated with the difference between the desired output and the actual output. The calculated error is used to adjust the weight.
The process is repeated until the fault made on the entire training set is less than the specified limit until the maximum number of iterations has been reached.

Complete code of Single layer perceptron

# Import the MINST dataset
from tensorflow.examples.tutorials.mnist import input_data 
mnist = input_data.read_data_ ("/tmp/data/", one_hot=True)

import tensorflow as tf 
import matplotlib.pyplot as plt 
# Parameters 
learning_rate = 0.01 
training_epochs = 25 
batch_size = 100 
display_step = 1 

# tf Graph Input 
x = tf.placeholder("float", [none, 784]) # MNIST data image of shape 28*28 = 784 
y = tf.placeholder("float", [none, 10]) # 0-9 digits recognition => 10 classes 
# Create model 
# Set model weights 
W = tf.Variable(tf.zeros([784, 10])) 
b = tf.Variable(tf.zeros([10])) 
# Constructing the model
activation=tf.nn.softmaxx(tf.matmul (x, W)+b) # Softmax
 of function 
# Minimizing error using cross entropy
cross_entropy = y*tf.log(activation) 
cost = tf.reduce_mean\ (-tf.reduce_sum\ (cross_entropy, reduction_indice = 1))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
#Plot settings 
avg_set = [] 
epoch_set = [] 
# Initializing the variables where init = tf.initialize_all_variables()
# Launching the graph
with tf.Session() as sess:
   sess.run(init)
   
# Training of the cycle in  the dataset
   for epoch in range(training_epochs):
      avg_cost = 0.
      total_batch = int(mnist.train.num_example/batch_size)
      
# Creating loops at all the batches in the code
      for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
         # Fitting the training by the batch data sess.run(optimizr,  feed_dict = {
x: batch_xs, y: batch_ys})
 # Compute all the average of loss avg_cost += sess.run(cost, \ feed_dict = {
x: batch_xs, \ y: batch_ys}) //total batch
      # Display the logs at each epoch steps 
      if epoch % display_step==0: 
      print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format (avg_cost))
            avg_set.append(avg_cost) epoch_set.append(epoch+1)
   print ("Training phase finished")
    
   plt.plot(epoch_set,avg_set, 'o', label = 'Logistics Regression Training')   
   plt.ylabel('cost') 
   plt.xlabel('epoch') 
   plt.legend() 
   plt.show() 
    
# Test the model
   correct_prediction = tf.equal (tf.argmax (activation, 1),tf.argmax(y,1)) 
  
# Calculating the accuracy of dataset
accuracy = tf.reduce_mean(tf.cast (correct_prediction, "float")) print 
("Model accuracy:", accuracy.eval({x:mnist.test.images, y: mnist.test.labels}))

The output of the Code:

The logistic regression is considered as predictive analysis. Logistic regression is mainly used to describe data and use to explain the relationship between the dependent binary variable and one or many nominal or independent variables.