Data Science – P-Value Explained with Examples

0

Are you one of the data science/machine learning beginners who wants to learn about P-Value using some examples? Are you one of those who has been hunting different web pages to understand P-Value in a simpler and easier manner?

This post is aimed to present P-VALUE concepts with multiple different examples. The following use cases and related hypothesis made about the population will either be accepted or rejected based on the P-VALUE:

  • Whether a coin is fair
  • Whether a dice is fair

What is P-VALUE?

P-value can be defined as the probability of obtaining a sample “more extreme” than the ones observed in the sample data used for hypothesis testing.  It is measured using techniques such as determining the test statistics such as Z, T or chi-square and calculating P-value using the related distribution tables such as z-distribution, t-distribution or chi-square distribution respectively.

Let’s take a quick example to understand the concept of P-value. Given a school consisting of both boys and girls students, let’s test the hypothesis that boys to girls ratio is not equal to 0.5. In other words, the percentage of boys to the total number of student is greater than 0.5 or 50%. In order to test the hypothesis, as a first step, we will need to formulate the null and alternate hypothesis. In this example, we set the null hypothesis as the ratio of boys to the total student is 0.5 (50%). The alternate hypothesis is that the ratio of the number of boys the number of girls is not equal to 0.5. As part of the test, several random samples of 20 students is taken to count the number of boys/girls. The following output would help understand the definition of P-Value.

Sample
(No. of students)
Outcome
(No. of Boys)
Outcome (Ratio of
Boys/Total Student)
Interpretation
20 12 0.6 There is a high likelihood that the test outcome looks to have happened by chance; Can’t reject the null hypothesis
20 16 0.8 The test outcome looks to be doubtful; Does not look like the outcome happened by chance; However, the evidence is not enough to reject the null hypothesis.
20 18 0.9 With a very high confidence level, it could be stated that the test outcome definitely does not look to have happened by chance; Given that the sample is chosen in a fair and random manner, the null hypothesis is rejected. The alternate hypothesis is accepted which implies that the boys are greater in number in the school. 
20 8 0.4 There is a high likelihood that the test outcome looks to have happened by chance; Can’t reject the null hypothesis
20 2 0.1 The test outcome definitely does not look to have happened by chance; Given that the sample is chosen in a fair and random manner, the null  hypothesis is rejected. The alternate hypothesis is accepted which implies that the boys are greater in number in the school.

 

In above example, the tests with a number of boys counted as 18 and 2 (red) in a random sample of 20 students are at an extreme level. The test outcomes look to be significant enough to indicate that the test results do not look to have happened by chance and that it is incorrect to claim that the ratio of the number of boys to the number of girls is 0.5.  In such cases, the P-Value may/will turn out to be lesser than 0.05. Given that the level of significance is set to be 0.05, the P-value can be used to indicate that the null hypothesis can be rejected. Thus, one could reject the null hypothesis.

 

The P-VALUE is used to represent whether the outcome of a hypothesis test is statistically significant enough to be able to reject the null hypothesis. It lies between 0 and 1.

The threshold value below which the P-VALUE becomes statistically significant is usually set to be 0.05. The threshold value is called the level of significance and is a function of confidence level.  One could choose to set different threshold value (such as 0.025 or 0.01) based on the confidence level based on which one could choose to reject the null hypothesis.

The following diagram represents p-value as the area of the shaded region (with red).

P-value explained with examples

Figure 1. P-Value

Let’s try and understand the intuition behind P-VALUE.

P-Value Explained using Null Hypothesis: The Coin is Fair

In case a coin is fair, it is expected that the probability of heads and tails being rolled out is around (or near to) 50%. In order to prove the claim for the population, multiple different experiments with samples representing 10 tosses of coins are done. The null hypothesis is that the coin is fair. The alternate hypothesis is that the coin is unfair. The following represents the test outcomes and interpretation related to when the hypothesis can be rejected.

Sample
(No. of tosses)
Outcome
(No. of Heads)
Interpretation
10 6 There is a high likelihood that the test outcome looks to have happened by chance; Can’t reject the null hypothesis
10 7 The test outcome looks to be doubtful; Does not look like the outcome happened by chance; However, the evidence is not enough to reject the null hypothesis.
10 9 With a very high confidence level, it could be stated that the test outcome does not look to have happened by chance; Given that the sample is chosen in a fair and random manner, the null hypothesis, that the coin is fair, can be rejected. The alternate hypothesis is accepted which implies that the coin is not fair.
10 4 There is a high likelihood that the test outcome looks to have happened by chance; Can’t reject the null hypothesis
10 1 With a very high confidence level, it could be stated that the test outcome definitely does not look to have happened by chance; Given that the sample is chosen in a fair and random manner, the null hypothesis, that the coin is fair, can be rejected. The alternate hypothesis is accepted which implies that the coin is not fair.

In above example, the tests with a number of heads counted as 9 and 1 (red) in a random sample of 10 tosses are at an extreme level. The test outcomes look to be significant enough to indicate that the test results do not look to have happened by chance and that it is incorrect to claim that the coin is fair.  In such cases, the P-Value may/will turn out to be lesser than 0.05. Given that the level of significance is set to be 0.05, the P-value can be used to indicate that the null hypothesis can be rejected. Thus, one could reject the null hypothesis.

P-Value Explained using Null Hypothesis: The Dice is Fair

In case the dice is fair, it is expected that the probability of getting 6 when the dice is rolled out is around (or near to) 16.67% (Expected value – the probability of 1/6). In order to prove the claim for the population, multiple different experiments with samples representing 50 tosses of dice are done. The null hypothesis is that the dice are fair. The alternate hypothesis is that the dice is unfair. The following represents the test outcomes and interpretation related to when the hypothesis can be rejected.

Sample
(No. of tosses)
Outcome
(No. of 6s)
Interpretation
50 25 There is a high likelihood that the test outcome looks to have happened by chance; Can’t reject the null hypothesis
50 15 The test outcome looks to be doubtful; Does not look like the outcome happened by chance; However, the evidence is not enough to reject the null hypothesis.
50 3 With a very high confidence level, it could be stated that the test outcome does not look to have happened by chance; Given that the sample is chosen in a fair and random manner, the null hypothesis, that the dice are fair, can be rejected. The alternate hypothesis is accepted which implies that the dice are not fair.
50 38 The test outcome looks to be doubtful; Does not look like the outcome happened by chance; However, the evidence is not enough to reject the null hypothesis.
50 47 With a very high confidence level, it could be stated that the test outcome definitely does not look to have happened by chance; Given that the sample is chosen in a fair and random manner, the null hypothesis, that the dice are fair, can be rejected. The alternate hypothesis is accepted which implies that the dice are not fair.

In above example, the tests with a number of 6s counted as 3 and 48 (red) in a random sample of 50 tosses are at an extreme level. The test outcomes look to be significant enough to indicate that the test results do not look to have happened by chance and that it is incorrect to claim that the dice are fair.  In such cases, the P-Value may/will turn out to be lesser than 0.05. Given that the level of significance is set to be 0.05, the P-value can be used to indicate that the null hypothesis can be rejected. Thus, one could reject the null hypothesis.

References

Summary

In this post, you learned about what is P-Value with the help of examples. Understanding P-Value is important for Data Scientists as it is used for hypothesis testing related to whether there is a relationship between a response variable and predictor variables. Hope you liked the details presented in the post. Please leave your comments or feel free to suggest.

Ajitesh Kumar

Ajitesh Kumar

Ajitesh is passionate about various different technologies including programming languages such as Java/JEE, Javascript, PHP, C/C++, mobile programming languages etc, and, computing fundamentals related with cloud-native technologies, application security, cloud computing platforms, mobile apps, big data etc.

He has also authored the book, Building Web Apps with Spring 5 and Angular.
Ajitesh Kumar

Leave A Reply

Time limit is exhausted. Please reload the CAPTCHA.