NLTK – How to Read & Process Text File

In this post, you will learn about the how to read one or more text files using NLTK and process words contained in the text file. As data scientists starting to work on NLP, the Python code sample for reading multiple text files from local storage will be very helpful.  Python Code Sample for Reading Text File using NLTK Here is the Python code sample for reading one or more text files. Pay attention to some of the following aspects: Class nltk.corpus.PlaintextCorpusReader reader is used for reading the text file. The constructor takes input parameter such as corpus root and the regular expression representing the files. List of files that are read could be found using method such as fileids List …

Continue reading

Posted in AI, NLP. Tagged with , .

Geometric Distribution Explained with Python Examples

In this post, you will learn about the concepts of Geometric probability distribution with the help of real-world examples and Python code examples. It is of utmost importance for data scientists to understand and get an intuition of different kinds of probability distribution including geometric distribution. You may want to check out some of my following posts on other probability distribution. Normal distribution explained with Python examples Binomial distribution explained with 10+ examples Hypergeometric distribution explained with 10+ examples In this post, the following topics have been covered: Geometric probability distribution concepts Geometric distribution python examples Geometric distribution real-world examples Geometric Probability Distribution Concepts Geometric probability distribution is a discrete …

Continue reading

Posted in Data Science, statistics. Tagged with , .

Top 10 Analytics Strategies for Great Data Products

In this post, you will learn about the top 10 data analytics strategies which will help you create successful data products. These strategies will be helpful in case you are setting up a data analytics practice or center of excellence (COE). As an AI / Machine Learning / Data Science stakeholders, it will be important to understand these strategies in order to deliver analytics solution which creates business value having positive business impact.  Here are the top 10 data analytics strategies: Identify top 2-3 business problems Identify related business / engineering organizations Create measurement plan by identifying right KPIs Identify analytics deliverables such as analytics reports, predictions etc Gather data …

Continue reading

Posted in Analytics, Data Science, Machine Learning. Tagged with , , .

Keras CNN Image Classification Example

In this post, you will learn about how to train a Keras Convolution Neural Network (CNN) for image classification. Before going ahead and looking at the Python / Keras code examples and related concepts, you may want to check my post on Convolution Neural Network – Simply Explained in order to get a good understanding of CNN concepts. Keras CNN Image Classification Code Example First and foremost, we will need to get the image data for training the model. In this post, Keras CNN used for image classification uses the Kaggle Fashion MNIST dataset. Fashion-MNIST is a dataset of Zalando’s article images—consisting of a training set of 60,000 examples and a …

Continue reading

Posted in Data Science, Deep Learning, Machine Learning. Tagged with , , , , .

Data Quality Challenges for Machine Learning Models

In this post, you will learn about some of the key data quality challenges which need to be dealt with in a consistent and sustained manner to ensure high quality machine learning models. Note that high quality models can be termed as models which generalizes better (lower true error with predictions) with unseen data or data derived from larger population. As a data science architect or quality assurance (QA) professional dealing with quality of machine learning models, you must learn some of these challenges and plan appropriate development processes to deal with these challenges.   Here are some of the key data quality challenges which need to be tackled appropriately in …

Continue reading

Posted in Data Science, Machine Learning, QA. Tagged with , , .

Convolutional Neural Network (CNN) – Simply Explained

In this post, you will learn about the basic concepts of convolutional neural network (CNN) explained with examples. As data scientists / machine learning / deep learning enthusiasts, you must get a good understanding of convolution neural network as there are many applications of CNN. Before getting into the details on CNN, let’s understand the meaning of Convolution in convolutional neural network. What’s Convolution? What’s intuition behind Convolution? Convolution represents a mathematical operation on two functions. As there can be applied different mathematical operations such as addition or multiplication on two different functions, in the similar manner, convolution operation can be applied on two different functions. Mathematically, the convolution of two different …

Continue reading

Posted in Deep Learning. Tagged with .

Data Quality Assessment Frameworks – Machine Learning

data quality assessment framework for machine learning

In this post, you will learn about data quality assessment frameworks / techniques in relation to machine learning and why one needs to assess data quality for building high-performance machine learning models? As a data science architect or development manager, you must get a sense of the importance of data quality in relation to building high-performance machine learning models. The idea is to understand what is the value of data set. The goal is to determine whether the value of data can be quantised. This is because it is important to understand whether the data contains rich information which could be valuable for building models and inform stakeholders on data …

Continue reading

Posted in Data Science, Machine Learning.

Keras Neural Network for Regression Problem

Keras Neural network for regression problem

In this post, you will learn about how to train neural network for regression machine learning problems using Python Keras. Regression problems are those which are related to predicting numerical continuous value based on input parameters / features. You may want to check out some of the following posts in relation to how to use Keras to train neural network for classification problems: Keras – How to train neural network to solve multi-class classification Keras – How to use learning curve to select most optimal neural network configuration for training classification model In this post, the following topics are covered: Design Keras neural network architecture for regression Keras neural network …

Continue reading

Posted in Data Science, Deep Learning. Tagged with , , .

Keras – Categorical Cross Entropy Loss Function

Cross Entropy Loss Function

In this post, you will learn about when to use categorical cross entropy loss function when training neural network using Python Keras. Generally speaking, the loss function is used to compute the quantity that the the model should seek to minimize during training. For regression models, the commonly used loss function used is mean squared error function while for classification models predicting the probability, the loss function most commonly used is cross entropy. In this post, you will learn about different types of cross entropy loss function which is used to train the Keras neural network model. Cross entropy loss function is an optimization function which is used in case …

Continue reading

Posted in Data Science, Deep Learning. Tagged with , .

Python Keras – Learning Curve for Classification Model

Training & Validation Accuracy & Loss of Keras Neural Network Model

In this post, you will learn about how to train an optimal neural network using Learning Curves and Python Keras. As a data scientist, it is good to understand the concepts of learning curve vis-a-vis neural network classification model to select the most optimal configuration of neural network for training high-performance neural network. In this post, the following topics have been covered: Concepts related to training a classification model using a neural network Python Keras code for creating the most optimal neural network using a learning curve  Training a Classification Neural Network Model using Keras Here are some of the key aspects of training a neural network classification model using Keras: …

Continue reading

Posted in Data Science, Deep Learning, Machine Learning. Tagged with , , .

Free MIT Course on Machine Learning for Healthcare

machine learning and healthcare MIT free course

In this post, you will get a quick overview on free MIT course on machine learning for healthcare. This is going to be really helpful for machine learning / data science enthusiasts as building machine learning solutions to serve healthcare requirements comes with its own set of risks. It will be good to learn about different machine learning techniques, applications related disease progression modeling, cardiac imaging, pathology etc, risks and risk mitigation techniques. Here is the link to the course – Machine Learning for Healthcare Here are the links to some of the important course content: Video lectures Lecture notes (PDF) The entire course material can be downloaded from this page – …

Continue reading

Posted in Machine Learning, Online Courses. Tagged with , .

Keras Multi-class Classification using IRIS Dataset

Python keras for multi-class classification model using IRIS dataset

In this post, you will learn about how to train a neural network for multi-class classification using Python Keras libraries and Sklearn IRIS dataset. As a deep learning enthusiasts, it will be good to learn about how to use Keras for training a multi-class classification neural network. The following topics are covered in this post: Keras neural network concepts for training multi-class classification model Python Keras code for fitting neural network using IRIS dataset Keras Neural Network Concepts for training Multi-class Classification Model Training a neural network for multi-class classification using Keras will require the following seven steps to be taken: Loading Sklearn IRIS dataset Prepare the dataset for training and testing …

Continue reading

Posted in Data Science, Deep Learning, Machine Learning, Python. Tagged with , , , .

How to Setup / Install MLFlow & Get Started

Install MLFLow and get started

In this post, you will learn about how to setup / install MLFlow right from your Jupyter Notebook and get started tracking your machine learning projects. This would prove to be very helpful if you are running an enterprise-wide AI practice where you have a bunch of data scientists working on different ML projects. Mlflow will help you track the score of different experiments related to different ML projects. Install MLFlow using Jupyter Notebook In order to install / set up MLFlow and do a quick POC, you could get started right from within your Jupyter notebook. Here are the commands to get set up. Mlflow could be installed with …

Continue reading

Posted in Machine Learning. Tagged with .

Python – How to Add Trend Line to Line Chart / Graph

Chris Gayle - Rohit Sharma - Dhoni - Virat Kohli IPL Batting Average Score Trendline

In this plot, you will learn about how to add trend line to the line chart / line graph using Python Matplotlib.As a data scientist, it proves to be helpful to learn the concepts and related Python code which can be used to draw or add the trend line to the line charts as it helps understand the trend and make decisions. In this post, we will consider an example of IPL average batting scores of Virat Kohli, Chris Gayle, MS Dhoni and Rohit Sharma of last 10 years, and, assess the trend related to their overall performance using trend lines. Let’s say that main reason why we want to …

Continue reading

Posted in Python, statistics. Tagged with , , .

Top Tutorials – Neural Network Back Propagation Algorithm

neural network back propagation algorithm

Here are the top web pages /videos for learning back propagation algorithm used to compute the gradients in neural network. I will update this page with more tutorials as I do further deep dive on back propagation algorithm. For beginners or expert level data scientists / machine learning enthusiasts, these tutorials will prove to be very helpful. Before going ahead and understanding back propagation algorithm from different pages, lets quickly understand the key components of neural network algorithm: Feed forward algorithm: Feed forward algorithm represents the aspect of how input signals travel through different neurons present in different layers in form of weighted sums and activations, and, result in output / …

Continue reading

Posted in Deep Learning, Machine Learning. Tagged with , , .

Product Manager – Machine Learning Interview Questions

interview questions for machine learning

In this post, you will learn about some of the interview questions which can be asked in the AI / machine learning based product manager / business analyst job.  Some of the questions listed in this post can also prove to be useful for the interview for the job position of director  or vice president, product management. The interview questions can be categorized based on some of the following topics: Machine learning high level concepts Identifying a problem as machine learning problems Identifying business metrics vs value generation Feature engineering Working with data science team in model development lifecycle Monitoring model performance Model performance metrics presentation to key stakeholders Setting up …

Continue reading

Posted in Interview questions, Machine Learning, Product Management. Tagged with , , .