Machine Learning – 7 Steps to Train a Neural Network

7 Steps to Train a Neural Network
This article represents some of the key steps required to train a neural network. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos.
Key Steps for Training a Neural Network

Following are 7 key steps for training a neural network.

  1. Pick a neural network architecture. This implies that you shall be pondering primarily upon the connectivity patterns of the neural network including some of the following aspects:
    • Number of input nodes: The way to identify number of input nodes is identify the number of features.
    • Number of hidden layers: The default is to use the single or one hidden layer. This is the most common practice.
    • Number of nodes in each of the hidden layers: In case of using multiple hidden layers, the best practice is to use same number of nodes in each hidden layer. In general practice, the number of hidden units is taken as comparable number to that of number of input nodes. That means one could take either the same number of hidden nodes as input nodes or maybe twice or thrice the number of input nodes.
    • Number of output nodes: The way to identify number of output nodes is to identify the number of output classes you want the neural network to process.
  2. Random Initialization of Weights: The weights are randomly intialized to value in between 0 and 1, or rather, very close to zero.
  3. Implementation of forward propagation algorithm to calculate hypothesis function for a set on input vector for any of the hidden layer.
  4. Implementation of cost function for optimizing parameter values. One may recall that cost function would help determine how well the neural network fits the training data.
  5. Implementation of back propagation algorithm to compute the error vector related with each of the nodes.
  6. Use gradient checking method to compare the gradient calculated using partial derivatives of cost function using back propagation and using numerical estimate of cost function gradient. The gradient checking method is used to validate if the implementation of backpropagation method is correct.
  7. Use gradient descent or advanced optimization technique with back propagation to try and minimize the cost function as a function of parameters or weights.


Ajitesh Kumar

I have been recently working in the area of Data analytics including Data Science and Machine Learning / Deep Learning. I am also passionate about different technologies including programming languages such as Java/JEE, Javascript, Python, R, Julia, etc, and technologies such as Blockchain, mobile computing, cloud-native technologies, application security, cloud computing platforms, big data, etc. For latest updates and blogs, follow us on Twitter. I would love to connect with you on Linkedin. Check out my latest book titled as First Principles Thinking: Building winning products using first principles thinking. Check out my other blog,
Posted in Big Data. Tagged with , , .

Leave a Reply

Your email address will not be published. Required fields are marked *