Machine Learning – 7 Steps to Train a Neural Network

This article represents some of the key steps required to train a neural network. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos.
Key Steps for Training a Neural Network

Following are 7 key steps for training a neural network.

  1. Pick a neural network architecture. This implies that you shall be pondering primarily upon the connectivity patterns of the neural network including some of the following aspects:
    • Number of input nodes: The way to identify number of input nodes is identify the number of features.
    • Number of hidden layers: The default is to use the single or one hidden layer. This is the most common practice.
    • Number of nodes in each of the hidden layers: In case of using multiple hidden layers, the best practice is to use same number of nodes in each hidden layer. In general practice, the number of hidden units is taken as comparable number to that of number of input nodes. That means one could take either the same number of hidden nodes as input nodes or maybe twice or thrice the number of input nodes.
    • Number of output nodes: The way to identify number of output nodes is to identify the number of output classes you want the neural network to process.
  2. Random Initialization of Weights: The weights are randomly intialized to value in between 0 and 1, or rather, very close to zero.
  3. Implementation of forward propagation algorithm to calculate hypothesis function for a set on input vector for any of the hidden layer.
  4. Implementation of cost function for optimizing parameter values. One may recall that cost function would help determine how well the neural network fits the training data.
  5. Implementation of back propagation algorithm to compute the error vector related with each of the nodes.
  6. Use gradient checking method to compare the gradient calculated using partial derivatives of cost function using back propagation and using numerical estimate of cost function gradient. The gradient checking method is used to validate if the implementation of backpropagation method is correct.
  7. Use gradient descent or advanced optimization technique with back propagation to try and minimize the cost function as a function of parameters or weights.


Ajitesh Kumar

Ajitesh Kumar

Ajitesh has been recently working in the area of AI and machine learning. Currently, his research area includes Safe & Quality AI. In addition, he is also passionate about various different technologies including programming languages such as Java/JEE, Javascript and technologies such as Blockchain, mobile computing, cloud-native technologies, application security, cloud computing platforms, big data etc.

He has also authored the book, Building Web Apps with Spring 5 and Angular.
Ajitesh Kumar

Leave A Reply

Time limit is exhausted. Please reload the CAPTCHA.