Hidden Layer Perceptron in TensorFlow

A hidden layer is an artificial neural network that is a layer in between input layers and output layers. Where the artificial neurons take in a set of weighted inputs and produce an output through an activation function. It is a part of nearly and neural in which engineers simulate the types of activity that go on in the human brain.

The hidden neural network is set up in some techniques. In many cases, weighted inputs are randomly assigned. On the other hand, they are fine-tuned and calibrated through a process called backpropagation.

The artificial neuron in the hidden layer of perceptron works as a biological neuron in the brain- it takes in its probabilistic input signals, and works on them. And it converts them into an output corresponding to the biological neuron's axon.

Layers after the input layer are called hidden because they are directly resolved to the input. The simplest network structure is to have a single neuron in the hidden layer that directly outputs the value.

Deep learning can refer to having many hidden layers in our neural network. They are deep because they will have been unimaginably slow to train historically, but may take seconds or minutes to prepare using modern techniques and hardware.

A single hidden layer will build a simple network.

The code for the hidden layers of the perceptron is shown below:

#Importing the essential modules in the hidden layer
import tensorflow as tf 
import numpy as np 
import mat
plotlib.pyplot as plt 
import math, random 



np.random.seed(1000) 
function_to_learn = lambda x: np.cos(x) + 0.1*np.random.randn(*x.shape) 
layer_1_neurons = 10 
NUM_points = 1000 

#Train the parameters of hidden layer
batch_size = 100 
NUM_EPOCHS = 1500 

all_x = np.float32(np.random.uniform(-2*math.pi, 2*math.pi, (1, NUM_points))).T 
   np.random.shuffle(all_x) 

train_size = int(900) 
#Train the first 700 points in the set x_training = all_x[:train_size]
y_training = function_to_learn(x_training)

#Training the last 300 points in the given set x_validation = all_x[train_size:] 
y_validation = function_to_learn(x_validation) 

plt.figure(1) 
plt.scatter(x_training, y_training, c = 'blue', label = 'train') 
plt.scatter(x_validation, y_validation, c = 'pink', label = 'validation') 
plt.legend() 
plt.show()

X = tf.placeholder(tf.float32, [None, 1], name = "X")
Y = tf.placeholder(tf.float32, [None, 1], name = "Y")

#first layer 
#Number of neurons = 10 
w_h = tf.Variable(
   tf.random_uniform([1, layer_1_neurons],\ minval = -1, maxval = 1, dtype = tf.float32)) 
b_h = tf.Variable(tf.zeros([1, layer_1_neurons], dtype = tf.float32)) 
h = tf.nn.sigmoid(tf.matmul(X, w_h) + b_h)

#output layer 
#Number of neurons = 10 
w_o = tf.Variable(
   tf.random_uniform([layer_1_neurons, 1],\ minval = -1, maxval = 1, dtype = tf.float32)) 
b_o = tf.Variable(tf.zeros([1, 1], dtype = tf.float32)) 

#building the model
model = tf.matmul(h, w_o) + b_o 

#minimize the cost function (model - Y) 
train_op = tf.train.AdamOptimizer().minimize(tf.nn.l2_loss(model - Y)) 

#Starting the Learning phase
sess = tf.Session() sess.run(tf.initialize_all_variables()) 

errors = [] 
for i in range(NUM_EPOCHS): 
   for start, end in zip(range(0, len(x_training), batch_size),\ 
      range(batch_size, len(x_training), batch_size)): 
      sess.run(train_op, feed_dict = {X: x_training[start:end],\ Y: y_training[start:end]})
   cost = sess.run(tf.nn.l2_loss(model - y_validation),\ feed_dict = {X:x_validation}) 
   errors.append(cost) 
   
   if i%100 == 0: 
      print("epoch %d, cost = %g" % (i, cost)) 
      
plt.plot(errors,label='MLP Function Approximation') plt.xlabel('epochs') 
plt.ylabel('cost') 
plt.legend() 
plt.show()

Output

Following is the illustration of function layer approximation-