Why Assumption Checks Matter
A small-sample test can produce a precise-looking statistic and p-value even when its model does not describe the data well. The reference distribution behind that calculation depends on assumptions about how observations were collected and about the population distributions. Checking those assumptions is part of the analysis, not an optional step after the calculation.
For small samples, plots can reveal conspicuous problems, but they cannot certify that a population is normal or that observations are independent. The study design is the primary evidence for independence. Data and residual plots help assess distribution shape and spread. A quiet-looking plot from a very small sample may simply have little power to reveal a problem.
What to Check for Each Procedure
The relevant assumptions depend on the design and the test. State the model first, then choose diagnostics that address its assumptions. For a one-sample t test, the observations are assumed to be an independent random sample from a normal population when exact small-sample inference is required. A paired t test applies that same condition to the within-pair differences, not separately to the two sets of measurements.
For a pooled two-sample t test, the two samples must be independent of each other, and observations within each sample must be independent. Exact small-sample inference assumes normal populations with a common variance. Welch’s two-sample t procedure allows the population variances to differ. Under normal populations it uses an approximate small-sample t reference distribution, typically with Satterthwaite degrees of freedom; it does not generally have an exact t distribution when variances differ.
A classical F test comparing two population variances assumes independent random samples from normal populations. Its exact reference distribution depends on both normality and independence. In a one-way analysis of variance (ANOVA), the corresponding model assumes independent errors, normal errors within each group, and a common error variance across groups. A violation of one assumption does not automatically make every method useless, but it can make the stated reference distribution or the resulting inference unreliable.
| Procedure | Key small-sample assumptions | What to examine |
|---|---|---|
| One-sample t | Independent observations; a normal population for exact inference | Study design; dotplot or normal probability plot |
| Paired t | Independent pairs; normal distribution of pair differences for exact inference | How pairs were formed; plot of the differences |
| Pooled two-sample t | Independent samples; normal populations; equal variances | Design; group plots; group spreads and residuals |
| Welch two-sample t | Independent samples; normal populations for its usual small-sample approximation; variances may differ | Design; group plots; outliers and skewness |
| Two-sample F test for variances | Independent samples from normal populations | Design and separate plots for both groups |
Study Design and Plots
Independence is primarily a design question. Ask whether the sampling plan selected distinct observational units, whether measurements were repeated on the same unit, or whether observations were collected in clusters such as households or clinics. Repeated measurements on one person are not independent observations for an ordinary two-sample test. They may call for a paired analysis if the pairs are meaningful and the differences are the intended quantities of interest. A plot of measured values cannot turn dependent observations into independent ones.
For a one-sample t procedure, inspect the sample values with a dotplot or another display that shows individual observations. A normal probability plot can help identify strong departures from normal shape: substantial curvature suggests skewness or another distributional mismatch, while an isolated point may indicate an outlier. For a paired t test, make these checks on the differences. The distribution of the differences is the relevant distribution for that test.
For comparisons across groups, inspect each group separately as well as the residuals. A residual here is an observation minus the mean of its group. Group plots can reveal skewness, isolated values, and unequal spreads. If the spreads grow or shrink systematically across groups, that is evidence against a common-variance model. A pooled plot can conceal group-specific problems, so it should not replace separate group displays.
These checks are evidence, not mechanical pass-or-fail rules. With a handful of observations, a normal probability plot may look irregular by chance, and a severe departure can remain hidden. Do not treat a formal-looking plot as proof of normality. Consider the context, the sample size, the potential influence of unusual observations, and whether the chosen procedure’s assumptions are plausible.
Worked Example: An Extreme Value in a One-Sample t Analysis
A hypothetical laboratory records eight independent measurements of a chemical process: 9, 10, 10, 11, 11, 12, 12, and 27 units. The analyst plans a one-sample t test of \(H_0:\mu=11\) against a two-sided alternative. Calculate the sample mean, sample variance, and t statistic, then assess whether the small-sample assumptions are credible.
Method. Calculate the usual one-sample t statistic, then inspect the observations themselves. The normality assumption concerns the population distribution, but the unusually large observation is an important warning sign in this small sample.
The sum of the observations is \(9+10+10+11+11+12+12+27=102\), so \(\bar X=102/8=12.75\). The squared deviations from 12.75 are \(14.0625, 7.5625, 7.5625, 3.0625, 3.0625, 0.5625, 0.5625,\) and \(203.0625\). In particular, the last value is \((27-12.75)^2=203.0625\).
The estimated standard error is \(S/\sqrt{8}\), and the null value is 11. Thus
The arithmetic is consistent: the standard error is about 2.0686 and \(1.75/2.0686\) is about 0.8460. But the value 27 is far separated from the other observations and contributes \(203.0625/239.5\), or about 84.8%, of the total squared deviation. The sample therefore gives a clear reason to question whether a normal-population model is plausible. The t statistic has been computed, but its small-sample reference distribution may not be trustworthy. The mistake to avoid is treating a correctly computed statistic as proof that its assumptions hold.
Why Normality and Equal Variance Affect the Tests
The one-sample t statistic is \(T=(\bar X-\mu_0)/(S/\sqrt n)\), where \(n\) is the sample size and \(\mu_0\) is the hypothesized mean. Under independent sampling from a normal population, the standardized sample mean has a standard normal distribution, and its relationship to the sample variance yields an exact Student t distribution with \(n-1\) degrees of freedom. This is why normality matters especially when \(n\) is small: it supports the exact reference distribution, rather than merely describing how the sample looks.
An outlier or strong skewness can affect both \(\bar X\) and \(S\), and so can change the t statistic as well as the accuracy of its reference distribution. Larger samples often make mean-based inference less sensitive to moderate departures from normality, but that is not a guarantee—particularly when the data are dependent or contain influential outliers.
For a pooled two-sample t test, a common variance is used to estimate the uncertainty in the difference between means. If the population variances differ, that pooled estimate may not represent the uncertainty appropriately, especially when the sample sizes are unequal. Welch’s method instead estimates the standard error using each group’s sample variance separately. Its usual degrees-of-freedom formula is an approximation, not a way to obtain an exact t distribution under unequal variances.
For a two-sample F test, the statistic is a ratio of sample variances. The exact F reference distribution depends strongly on normality; skewness or outliers can greatly affect sample variances and therefore the ratio. Equal observed spreads on a plot do not establish normality, and similar-looking distributions do not establish independence. Each assumption must be considered on its own.
Here \(S_1^2\) and \(S_2^2\) are the sample variances and \(n_1,n_2\) are the sample sizes. The cancellation of \(\sigma^2\) explains why the ratio is useful for comparing variances. It also shows why the distributional claim requires the model assumptions: without normality, the scaled sample variances need not have the stated chi-square distributions.
Worked Example: Assessing a Pooled Two-Sample t Model
A hypothetical study compares a calibration reading from four independently selected devices made by process A with readings from four different devices made by process B. The readings are A: 8, 9, 10, 11 and B: 12, 13, 14, 15. The research question concerns a difference in population means. Compute the pooled t statistic and assess what the small samples can and cannot show about the assumptions.
Method. Since the devices in the two groups are distinct, the samples are independent by the stated design. For illustration, calculate the pooled t statistic under the additional common-variance and normal-population assumptions; then inspect the group values.
The means are \(\bar X_A=(8+9+10+11)/4=9.5\) and \(\bar X_B=(12+13+14+15)/4=13.5\). Within group A, the squared deviations from 9.5 sum to \(2.25+0.25+0.25+2.25=5\). Within group B, the squared deviations from 13.5 also sum to \(5\). Thus \(S_A^2=S_B^2=5/3\), and the pooled variance is
The estimated standard error for \(\bar X_A-\bar X_B\) is \(\sqrt{S_p^2(1/4+1/4)}=\sqrt{5/6}\approx0.9129\). The statistic and degrees of freedom are
Both samples are ordered without an isolated observation or obvious skewness, and their observed spreads match. However, four values per group are too few to establish normality or equal population variances. The calculation is valid under the stated pooled model; the plots provide no strong visible warning, but cannot verify that model. The trap is to interpret matching sample variances as proof that the population variances are equal.
Worked Example: A Variance Ratio and Its Limits
Two hypothetical independent groups each contain four observations. Group C records 4, 5, 6, 7 units; group D records 3, 5, 7, 9 units. An analyst considers the classical two-sample F test for equal population variances. Calculate the variance ratio and identify what must be checked before using an F reference distribution.
Method. Calculate each sample variance using its own sample mean and divide the larger sample variance by the smaller one. This gives an upper-tail ratio for describing the observed spread; it does not by itself establish a valid test result.
Group C has mean \((4+5+6+7)/4=5.5\). Its squared deviations sum to \(2.25+0.25+0.25+2.25=5\), so \(S_C^2=5/3\). Group D has mean \((3+5+7+9)/4=6\). Its squared deviations sum to \(9+1+1+9=20\), so \(S_D^2=20/3\). The ratio is therefore
The degrees of freedom associated with this ordering are 3 in the numerator and 3 in the denominator. Both groups have symmetric, evenly spaced observations, with no obvious outlier. Nevertheless, the samples are very small, and the F test’s exact reference distribution requires independent samples from normal populations. The ratio 4 reports that the observed variance in D is four times the observed variance in C; deciding whether this is statistically significant requires the appropriate test specification and reference distribution. The mistake to avoid is treating a large variance ratio as conclusive while ignoring normality and the sampling design.
A Practical Checking Sequence
Determine whether measurements come from independent units, matched pairs, repeated measures, or clusters. Choose the procedure to match the design.
Specify which population or errors must be normal and whether equal variances are required. For a paired test, specify that the differences are the relevant values.
Use individual-value plots for small samples, and examine groups separately. Look for skewness, outliers, clusters, and unequal spreads.
Explain whether the design and plots support the assumptions, whether the small sample limits the check, and how a visible violation could affect the reference distribution or variance estimate.
Common Mistakes and Exam Traps
- Trying to diagnose independence from a histogram. Independence comes from the sampling and measurement design. A histogram can look regular even when observations are repeated or clustered.
- Checking the wrong values for a paired t test. The normality condition for exact small-sample inference concerns the pair differences, not the two marginal samples separately.
- Assuming Welch’s test is exact under unequal variances. Welch’s procedure allows unequal variances, but its usual small-sample t reference distribution and degrees of freedom are approximate.
- Using an F test as a general-purpose check for equal spread. The classical variance-ratio test is sensitive to departures from normality. Inspect the distributions and design; do not treat the F statistic as assumption-free evidence.
- Equating no visible warning with proof. Small samples can hide problems. Report the limitation rather than claiming that a plot confirms normality or equal variance.
Check Your Understanding
For each problem, identify the relevant assumptions and use the supplied information to assess or calculate what is requested.
- A paired study records a before-and-after reading for each of six independent machines. Which values should be examined for the normality condition of an exact paired t test? (Answer: The six within-machine differences; the pairs should also be independent of one another.)
- Two independent groups each have five observations. Their sample variances are \(S_1^2=9\) and \(S_2^2=4\). What is the larger-over-smaller observed variance ratio, and does that ratio establish that the populations are normally distributed? (Answer: \(9/4=2.25\); no, a variance ratio does not establish normality.)
- A small-sample pooled two-sample t analysis uses separate groups whose sample variances are 16 and 4, with sample sizes 3 and 12. Name one concern about using the pooled procedure and one feature of Welch’s procedure that addresses it. (Answer: The observed spreads differ and the sample sizes are unequal, so pooling may not estimate uncertainty appropriately if population variances differ; Welch’s procedure allows unequal variances and uses an approximate degrees-of-freedom calculation.)
- A technician measures each of ten components three times and enters all 30 readings as independent observations in a one-sample t test. What assumption is in question, and why can a normal probability plot of the 30 readings not resolve it? (Answer: Independence is in question because repeated readings from the same component may be related; a normal probability plot assesses distribution shape, not dependence.)
- In a two-group study, a residual plot shows that the spread around the group means is much larger in one group, and that group also contains an extreme observation. State two assumptions or features that warrant caution for a small-sample pooled t or F analysis. (Answer: Equal variance is questionable, and the extreme observation may undermine normality-based small-sample inference and strongly affect the sample variance.)