In the world of data science, understanding the differences between various statistical tests is crucial for accurate data analysis. Three most popular tests – the Z-test, T-test, and Chi-square test – each serve specific purposes. This blog post will delve into their definitions, types, formulas, appropriate usage scenarios, and the Python/R packages that can be used for their implementation, along with real-world examples. Check out a detailed post on the differences between Z-test vs T-test.
Definition: What’s Z-test vs T-test vs Chi-square test?
The following represents the definition of each of the tests along with a real-world example:
Z-test: The Z-test is a statistical test used to determine if there are significant differences between two population means. It’s particularly useful when the variances are known and the sample size is large. For example, a company might use a Z-test to determine if the average productivity of employees this year is different from last year, given they have a large number of employees and historical data on productivity.
T-test: The T-test is used for comparing the means of two groups, especially when the sample sizes are small and the population variance is unknown. For instance, a researcher could use a T-test to compare the effects of two different diets on weight loss, where each diet is tried by a small group of participants, and the variance in weight loss is not known beforehand.
Chi-square Test: The Chi-square test is a statistical method to compare observed results with expected results in categorical data. This test is useful in determining whether there are significant differences or relationships between categories. An example would be using the Chi-square test in marketing to determine if the preference for a product differs by gender, with categories being the different genders and the product preferences.
Different Types & Formula of Z-test, T-Test & Chi-Square Test
The following are different types and formula of tests for t-test, z-test and chi-square test:
Z-test
One-sample Z-test
This test compares the mean of a single sample to a known population mean. For example, a manufacturer might use this test to determine if the average weight of a batch of products is equal to the standard weight.
The following is the formula for z-statistics in one-sample z-test:
$Z = \frac{\bar{x} – \mu}{\sigma/\sqrt{n}}$
Two-sample Z-test
This test compares the means of two independent samples. For instance, a company may use it to compare the average salaries between two different offices. Another example can be that a retail company might use this to compare the average transaction values between two stores. If one store recently underwent renovations, the company could use a two-sample Z-test to evaluate if the renovation had a significant effect on sales compared to a store that wasn’t renovated.
The following is the formula for z-statistics in two-sample z-test:
$ Z = \frac{\bar{x}_1 – \bar{x}_2}{\sqrt{\sigma_1^2 / n_1 + \sigma_2^2 / n_2}} $
Z-test for Proportions
Used to determine whether the proportion of a certain characteristic is different between two groups. For instance, a survey could use this to compare the proportion of male versus female respondents who prefer a particular product. This can be used in election studies. If exit polls suggest 40% of voters in one city favor a certain candidate, and 35% in another, a Z-test for proportions can determine if this difference is statistically significant or due to random chance.
The following is the formula of z-statistics in z-test for proportion:
$Z = \frac{p_1 – p_2}{\sqrt{P(1 – P)(\frac{1}{n_1} + \frac{1}{n_2})}}$
where $P = \frac{p_1n_1 + p_2n_2}{n_1 + n_2}$
T-test
One-sample T-test
This test checks if the mean of a single group is significantly different from a known mean. For example, a school could use it to determine if the average score of a class in a national exam significantly deviates from the national average. In environmental science, this test could assess whether the average level of a pollutant in a lake differs significantly from national safety standards. If the national standard for a pollutant is 10 parts per million, a one-sample T-test can compare this standard against the observed pollutant levels in the lake.
The following is the formula for t-statistics in one-sample t-test:
$t = \frac{\bar{x} – \mu}{s/\sqrt{n}}$
Two-sample T-test
This test compares the means of two independent groups. For example, it could be used to compare the average height of players in two different basketball teams. In medicine, a two-sample T-test might be used to compare the effectiveness of two treatments. If two groups of patients are given different drugs for the same condition, this test can assess if there’s a significant difference in outcomes between the groups.
The following is the formula for t-statistics in two-sample t-test:
$t = \frac{\bar{x}_1 – \bar{x}_2}{\sqrt{s_1^2/n_1 + s_2^2/n_2}}$
Paired sample T-test
This test compares the means of the same group at different times. For instance, in education, a paired sample T-test can evaluate the impact of a new teaching method. By comparing student scores before and after implementing the new method in the same group of students, educators can determine if the method significantly affects student performance.
T-test for Proportions
Similar to the Z-test for proportions, but used when sample sizes are small and variances are unknown. For example, it might be used to compare the proportion of customers satisfied with a service before and after an improvement was implemented. This could be applied in marketing as well. If a company wants to know whether a promotional campaign increased the proportion of customers who buy a certain product, they can compare the proportions of purchasers before and after the campaign using a T-test for proportions.
Chi-square Test
Chi-square Goodness of Fit Test
This test determines if a sample data matches a population. For example, a researcher might use it to test whether the ethnic distribution in a study sample reflects the actual distribution in the general population. In marketing, a company might want to know if their customer demographic distribution matches their targeted marketing demographic. Using the Chi-square Goodness of Fit Test, they can compare their customer demographic data to the desired demographic distribution.
The following is the formula for chi-square statistics in Chi-square goodness of fit test:
$\chi^2 = \sum \frac{(O_i – E_i)^2}{E_i}$
Chi-square Test for Independence
This test examines the relationship between two categorical variables. For instance, this test is widely used in healthcare research. For instance, researchers might use it to investigate if there’s a relationship between smoking status and lung cancer. By categorizing individuals as smokers or non-smokers and those with or without lung cancer, the Chi-square Test for Independence can help determine if these variables are related.
The following is the formula for chi-square statistics in Chi-square test of independence:
$\chi^2 = \sum \frac{(O_{ij} – E_{ij})^2}{E_{ij}}$
where $E_{ij} = \frac{(row\_total) \times (column\_total)}{grand\_total}$
When to use Z-test vs T-test vs Chi-square test
The following represents scenarios / conditions based on which you can decide to use one of the three tests:
Use Z-test when:
- Large Sample Sizes: The Z-test is most appropriate when dealing with large sample sizes, typically considered to be more than 30 observations.
- Known Population Variance: It’s used when the population variance is known, which is a rare scenario in real-world applications.
- Normal Distribution: The data should be normally distributed, especially for smaller sample sizes. However, due to the Central Limit Theorem, the Z-test can be used with larger samples even if the distribution is not perfectly normal.
Use T-test when:
- Small Sample Sizes: The T-test is the preferred choice for smaller sample sizes, typically less than 30.
- Unknown Population Variance: It is used when the population variance is unknown, which is a common scenario in practical applications.
- Approximately Normal Distribution: The T-test is robust to slight deviations from normality, especially with larger sample sizes. However, for very small samples, the normality of data is more critical.
- Paired or Independent Samples: Depending on the type of T-test (paired, one-sample, or two-sample), it can be used for different experimental designs, such as comparing means from the same group at different times or comparing means between two independent groups.
Use Chi-square test when:
- Categorical Data: The Chi-square test is exclusively used for categorical (nominal and ordinal) data.
- Independence or Goodness of Fit: The test is suitable for testing the independence between two variables (Chi-square Test for Independence) or for testing if a sample distribution matches a hypothesized distribution (Chi-square Goodness of Fit Test).
- Sufficiently Large Sample Size: The expected frequencies in each category should ideally be 5 or more to ensure the accuracy of the test.
- Random Sampling and Independent Observations: The data should be collected through random sampling, and the observations should be independent of each other.
Check out my latest book titled as First Principles Thinking: Building winning products using first principles thinking.
- Agentic Reasoning Design Patterns in AI: Examples - October 18, 2024
- LLMs for Adaptive Learning & Personalized Education - October 8, 2024
- Sparse Mixture of Experts (MoE) Models: Examples - October 6, 2024
I found it very helpful. However the differences are not too understandable for me