Assumptions of the t-test are almost never tested, and are almost always violated to some degree
Enumerating them for the quantitative types among us
Samples are of a random, continuous variable.
Measurements within a sample are independent of each other.
Samples are representative of the population from which they are drawn.
The amount of variation in the two samples is similar (homoscedasticity).
The data in each sample are normally distributed.
Samples sample sizes (N from each group) are large enough to detect a difference that does exist with an accepted probability (usually 80%).
What Can Happen if the Assumptions are Violated
If sample measurements are not continuous, the t-test is the wrong test.
If the sample measurements are not random samples from the population (2), the representativeness of the collection of values (of the population) may be compromised.
If the sample measurements are not representative of the population, the inference made about the difference of means may not generalize to the real world.
If the two samples do not share roughly the same amount of variation around their means, characteristics and artifacts in one of the two samples can drive the analysis.
If the data are not normally distributed, they may be quite different and the t-test will miss the whole-distribution difference (both samples skewed toward each other, for example). Alternatively, the t-test might also miss a difference that is real (sample distributions skewed away from each other).
If the sample sizes of the two groups are too small, a real difference might be missed only due to the small sample size. In part, it is because small sample sizes will tend to be spuriously non-representative of the population.
When you go to a study that uses the t-test, what if the study does not show a priori power curves (#6) and the sample size is low? How do we know, if the researchers used p-values-based hypothesis testing, that the inference was not merely due to the small sample size?
At IPAK-EDU, we offer an increasing number of quantitative courses in our Analytics track. In the Spring, we offer Applied Biostats.
You will learn the concepts of and how to execute many analyses. If you never use the methods yourself, you’ll be more empowered to understand and critique studies that you read.
Here’s the link to Register, hope to see you in class!
https://ipak-edu.org/registration/?store-page=Applied-Biostatistics-Analytics-200-p379243785
