How do we build Deep Neural Network using Perceptron?

Single layer neural network

In this post, you will understand about how a deep neural network is built using a perceptron. This is a very important concept in relation to getting a good understanding of deep learning. You will also learn related Tensorflow / Keras code snippet.

Here are the key concepts related to how deep neural network is built using one or more perceptrons:

  • First and foremost, it is key to understand what is a Perceptron?
    • A perceptron is the most fundamental unit which is used to build a neural network. A perceptron resembles a neuron in the human brain. In case of a neuron, multiple input signals are fed into a neuron via dendrite and an output signal is fired appropriately based on the strength of the input signal and some other mechanism. In case of a perceptron, the input signals are combined with different weights and fed into the perceptron along with a bias element. Representationally, within perceptron, the net sum is calculated as sum of weights and input signal and a bias element, then, the net sum is fed into a non-linear activation function. Based on the activation function, the output signal is sent out. To understand greater details around perceptron, here is my post – Perceptron explained with Python example

      Here is a picture of a perceptron. Note the summation of inputs with weights (w1, w2, w3, wm) is passed through activation function and final output is obtained. 

      Perceptron

      Fig. Perceptron


      Here is an example of a multi-output perceptron. Note that perceptron is stacked and there are two outputs. Also, note that the perceptrons are fully connected, e.g., each output is connected with every input through perceptrons. The layers consisting of perceptrons which are connected to all inputs are called as dense layers or fully connected layers. Dense layer is also called as a stack of perceptrons.

      Multi output perceptron

      Fig. Multi-output perceptron


      The above dense layer consisting of two units can be represented using the following tensorflow code:

      import tensorflow as tf
      layer = tf.keras.layer.Dense(units = 2)
       


  • What is a deep neural network (DNN) and how it is made up of perceptrons?
    • Deep neural network (DNN) can be said to represent mathematical model for human brain. As human brain is made up large number of neurons, a deep neural network is also made up of large number of neuron-like unit called as perceptron. The perceptron in deep neural network is laid out in form of four or more layers including one input layer, one output layer and two or more hidden layers. Thus, a deep neural network comprises of four or more layers consisting of one or more perceptrons. A neural network consisting of one input layer, one hidden layer and one output layer is commonly known as multi-layer perceptron (MLP) neural network.

      This is how the input flows through the network. The input comprising of one or more inputs combined with weights and along with a bias element, is fed into each perceptron of the first layer. For each of the perceptrons, the weights can be different. The output from the perceptrons in first layer is combined with new weights and fed into each perceptron of the second layer along with a bias element and so on and so forth until final layer is reached. Note that weights combined with input to different perceptrons can be different. Such layers where all perceptrons are connected with all inputs combined with different weights are also termed as dense layers. And, the neural network with dense layers is called as fully connected neural network. Refer to the picture given below showing a neural network comprising of one hidden layer and one output layer.Note than each layer will have its own set of weights. Also, z1, z2, z3 and z4 represents net input which is sum of weights and inputs. In the neural network shown below, there are 4 perceptrons in the hidden layer and 2 perceptrons in the output layer. Each of the perceptron will have an associated non-linear activation functions represented using letter g.  When there are two or more hidden layer, such neural networks are called as deep neural network. And, the learning is called as deep learning. In the picture below, a multi-layer perceptron (MLP) neural network is shown.

      Single layer neural network

      Fig 1. Multi-layer perceptron (MLP) neural network 


      In the picture below, note how net input is calculated as a sum of input signals and weights. 

      Single layer neural network - Net input explained

      Fig 2. Neural network – Net input explained


      The above can be represented as sequential neural network with two layers (hidden and output layer) in tensorflow using the following code:
import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(units=n),
    tf.keras.layers.Dense(units=2)
])

# Above code can also be written as the following:
model = tf.keras.Sequential([
    tf.keras.layers.Dense(n),
    tf.keras.layers.Dense(2)
])

In the above code, the first layer is a dense layer having n units, and the second layer, which is also the output layer, is a dense layer having two units.

In case, there are 5 dense layers with 4 hidden and 1 output layer with n1 unit in the first layer, n2 in the second layer, n3 in the third layer, n4 in the 4th layer, and 3 units in last layer, the model code in TensorFlow would look like the following:

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(n1),
    tf.keras.layers.Dense(n2),
    tf.keras.layers.Dense(n3),
    tf.keras.layers.Dense(n4),
    tf.keras.layers.Dense(3)
])
Ajitesh Kumar
Follow me
Latest posts by Ajitesh Kumar (see all)

Ajitesh Kumar

I have been recently working in the area of Data Science and Machine Learning / Deep Learning. In addition, I am also passionate about various different technologies including programming languages such as Java/JEE, Javascript, Python, R, Julia etc and technologies such as Blockchain, mobile computing, cloud-native technologies, application security, cloud computing platforms, big data etc. I would love to connect with you on Linkedin.
Posted in Data Science, Deep Learning, Machine Learning. Tagged with , .

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.