## Spacy Tokenization Python Example

In this post, you will quickly learn about how to use Spacy for reading and tokenising a document read from text file or otherwise. As a data scientist starting on NLP, this is one of those first code which you will be writing to read the text using spaCy. First and foremost, make sure you have got set up with Spacy, and, loaded English tokenizer. The following commands help you set up in Jupyter notebook. Reading text using spaCy: Once you are set up with Spacy and loaded English tokenizer, the following code can be used to read the text from the text file and tokenize the text into words. Pay attention …

## Top 10 Types of Analytics Projects – Examples

In this post, you will learn about some of the most common types of data analytics projects which can be executed by the organization to realise associated business value from analytics projects and, also, gain competitive advantage with respect to the related business functions. Note that analytics projects are different from AI / ML projects. AI / ML or predictive analytics is one part of analytics. Other types of analytics projects include those related with descriptive and prescriptive analytics. You may want to check out one of my related posts on difference between predictive and prescriptive analytics. Here are the key areas of focus for data analytics projects: Cost reduction: …

## Different Success / Evaluation Metrics for AI / ML Products

In this post, you will learn about some of the common success metrics which can be used for measuring the success of AI / ML (machine learning) / DS (data science) initiatives / products. If you are one of the AI / ML stakeholders, you would want to get hold of these metrics in order to apply right metrics in right business use cases. Business leaders do want to know and maximise the return on investments (ROI) from AI / ML investments. Here is the list of success metrics for AI / DS / ML initiatives: Business value metrics / Key performance indicators (KPIs): Business value metrics such as operating …

## Predictive vs Prescriptive Analytics Difference

In this post, you will quickly learn about the difference between predictive analytics and prescriptive analytics. As data analytics stakeholders, one must get a good understanding of these concepts in order to decide when to apply predictive and when to make use of prescriptive analytics in analytics solutions / applications. Without further ado, let’s get straight to the diagram. In the above diagram, you could observe / learn the following: Predictive analytics: In predictive analytics, the model is trained using historical / past data based on supervised, unsupervised, reinforcement learning algorithms. Once trained, the new data / observation is input to the trained model. The output of the model is prediction in form …

## Analytics Maturity Model for Assessing Analytics Practice

In this post, you will learn about data analytics maturity model which you could use to assess where does your business / organization stand on the path of using analytics to drive business value. If you represent decision-making stakeholders group and want to assess your organization readiness / capabilities to deploy analytics in order to create business value creation, you may find this post useful enough. Here is a list of other articles I posted in the recent past in relation to strategic data analytics: Top 10 analytics strategies for great data products Top 5 data analytics methodologies Here are the three broad categories / levels of data analytics maturity model: Analytically …

## Fixed vs Random vs Mixed Effects Models – Examples

In this post, you will learn about the concepts of fixed and random effects models along with when to use fixed effects models and when to go for fixed + random effects (mixed) models. The concepts will be explained with examples. As data scientists, you must get a good understanding of these concepts as it would help you build better linear models such as general linear mixed models or generalized linear mixed models (GLMM). The following are some of the topics covered in this post: What are fixed, random & mixed effects models? When to use fixed effects vs mixed effects models? What are fixed, random & mixed effects models? First, we will take a real world …

## Hierarchical Clustering Explained with Python Example

In this post, you will learn about the concepts of Hierarchical clustering with the help of Python code example. As data scientist / machine learning enthusiasts, you would want to learn the concepts of hierarchical clustering in a great manner. The following topics will be covered in this post: What is hierarchical clustering? Hierarchical clustering Python example What is Hierarchical Clustering? Hierarchical clustering is an unsupervised learning algorithm which is based on clustering data based on hierarchical ordering. Recall that clustering is an algorithm which groups data points within multiple clusters such that data within each cluster are similar to each other while clusters are different each other. The hierarchical clustering can be classified …

## Negative Binomial Distribution Python Examples

In this post, you will learn about the concepts of negative binomial distribution explained using real-world examples and Python code. We will go over some of the following topics to understand negative binomial distribution: What is negative binomial distribution? What is difference between binomial and negative binomial distribution? Negative binomial distribution real-world examples Negative binomial distribution Python example What is Negative Binomial Distribution? Negative binomial distribution is a discrete probability distribution representing the probability of random variable, X, which is number of Bernoulli trials required to have r number of successes. This random variable is called as negative binomial random variable. And, the experiment representing X number of Bernoulli trials required to product r successes is called …

## Generalized Linear Models Explained with Examples

In this post, you will learn about the concepts of generalized linear models (GLM) with the help of Python examples. It is very important for data scientists to understand the concepts of generalized linear models and how are they different from general linear models such as regression or ANOVA models. Some of the following topics have been covered in this post: What are generalized linear models (GLM)? Generalized linear models real-world examples When to use generalized linear models? What are Generalized Linear Models? Generalized linear models represent the class of regression models which models the response variable, Y, and the random error term () based on exponential family of distributions such as normal, Poisson, …

## NLTK – How to Read & Process Text File

In this post, you will learn about the how to read one or more text files using NLTK and process words contained in the text file. As data scientists starting to work on NLP, the Python code sample for reading multiple text files from local storage will be very helpful. Python Code Sample for Reading Text File using NLTK Here is the Python code sample for reading one or more text files. Pay attention to some of the following aspects: Class nltk.corpus.PlaintextCorpusReader reader is used for reading the text file. The constructor takes input parameter such as corpus root and the regular expression representing the files. List of files that are read could be found using method such as fileids List …

## Poisson Distribution Explained with Python Examples

In this post, you will learn about the concepts of Poisson probability distribution with Python examples. As a data scientist, you must get a good understanding of the concepts of probability distributions including normal, binomial, Poisson etc. Poisson distribution is the discrete probability distribution which represents the probability of occurrence of an event r number of times in a given interval of time or space if these events occur with a known constant mean rate and independent of each other. The following is the key criteria that the random variable follows the Poisson distribution. Individual events occur at random and independently in a given interval. This can be an interval of time or …

## Geometric Distribution Explained with Python Examples

In this post, you will learn about the concepts of Geometric probability distribution with the help of real-world examples and Python code examples. It is of utmost importance for data scientists to understand and get an intuition of different kinds of probability distribution including geometric distribution. You may want to check out some of my following posts on other probability distribution. Normal distribution explained with Python examples Binomial distribution explained with 10+ examples Hypergeometric distribution explained with 10+ examples In this post, the following topics have been covered: Geometric probability distribution concepts Geometric distribution python examples Geometric distribution real-world examples Geometric Probability Distribution Concepts Geometric probability distribution is a discrete …

## Top 10 Analytics Strategies for Great Data Products

In this post, you will learn about the top 10 data analytics strategies which will help you create successful data products. These strategies will be helpful in case you are setting up a data analytics practice or center of excellence (COE). As an AI / Machine Learning / Data Science stakeholders, it will be important to understand these strategies in order to deliver analytics solution which creates business value having positive business impact. Here are the top 10 data analytics strategies: Identify top 2-3 business problems Identify related business / engineering organizations Create measurement plan by identifying right KPIs Identify analytics deliverables such as analytics reports, predictions etc Gather data …

## Keras CNN Image Classification Example

In this post, you will learn about how to train a Keras Convolution Neural Network (CNN) for image classification. Before going ahead and looking at the Python / Keras code examples and related concepts, you may want to check my post on Convolution Neural Network – Simply Explained in order to get a good understanding of CNN concepts. Keras CNN Image Classification Code Example First and foremost, we will need to get the image data for training the model. In this post, Keras CNN used for image classification uses the Kaggle Fashion MNIST dataset. Fashion-MNIST is a dataset of Zalando’s article images—consisting of a training set of 60,000 examples and a …

## Data Quality Challenges for Machine Learning Models

In this post, you will learn about some of the key data quality challenges which need to be dealt with in a consistent and sustained manner to ensure high quality machine learning models. Note that high quality models can be termed as models which generalizes better (lower true error with predictions) with unseen data or data derived from larger population. As a data science architect or quality assurance (QA) professional dealing with quality of machine learning models, you must learn some of these challenges and plan appropriate development processes to deal with these challenges. Here are some of the key data quality challenges which need to be tackled appropriately in …

## Convolutional Neural Network (CNN) – Simply Explained

In this post, you will learn about the basic concepts of convolutional neural network (CNN) explained with examples. As data scientists / machine learning / deep learning enthusiasts, you must get a good understanding of convolution neural network as there are many applications of CNN. Before getting into the details on CNN, let’s understand the meaning of Convolution in convolutional neural network. What’s Convolution? What’s intuition behind Convolution? Convolution represents a mathematical operation on two functions. As there can be applied different mathematical operations such as addition or multiplication on two different functions, in the similar manner, convolution operation can be applied on two different functions. Mathematically, the convolution of two different …