When Small Samples Make T Intervals Fragile

Why Sample Shape Matters More When n Is Small

A one-sample t interval uses the sample mean and sample standard deviation to estimate a population mean. Its familiar formula does not change just because the sample is small. What changes is how much confidence we should place in its performance: with few observations, an unusually skewed sample or a single extreme value can strongly affect both the estimated center and spread. The result may be a poor estimate of the population mean, even when the arithmetic is correct.

The key question is not simply whether the sample is small. It is whether the data shape and the conditions for inference make the t procedure reasonable. A small sample with an approximately symmetric shape and no influential outliers can be suitable. A small sample with marked skewness or an extreme observation deserves caution. The t interval is then fragile: its actual coverage may depart from the stated confidence level, and the interval can be strongly influenced by individual observations.

Definition: For $n$ quantitative observations, the one-sample $t$ interval for a population mean $\mu$ is $$ \bar{x}\pm t^* \frac{s}{\sqrt{n}}, $$ where $\bar{x}$ is the sample mean, $s$ is the sample standard deviation, and $t^*$ is the positive critical value from a $t$ distribution with $df=n-1$ degrees of freedom for the chosen confidence level. The quantity $s/\sqrt{n}$ is the standard error $SE$; the margin of error is $t^*SE$.

Before using this interval, the observations should come from a random sample or an appropriate random process, and they should be independent. If sampling without replacement, the population should be at least ten times the sample size for the usual independence approximation. For a small sample, the population distribution should also be approximately normal: the data should show no strong skewness or extreme outliers. These conditions do different jobs. Random sampling supports generalization, independence supports the standard error calculation, and a suitable shape supports the $t$-based reference distribution.

If observations are independent and come from a normal population, the one-sample $t$ procedure has its exact model-based justification. With a nonnormal population, it is an approximation. Larger samples often make inference about a mean less sensitive to population shape, but a small sample provides little such protection. A sample plot is useful evidence, not a guarantee: with few observations, important features of the population may be hard to detect.

A Small Sample With a Suitable Shape

First consider a hypothetical random, independent sample with values that are balanced around the center. A symmetric pattern without isolated values is the kind of sample shape that makes a small-sample t interval more defensible, provided the other conditions are met.

Worked Example: A Nearly Symmetric Sample of Five Measurements

Suppose five independently selected objects have hypothetical measurement values of 8, 9, 10, 11, and 12 units. Assume they were randomly sampled from a population, and that this sample shape is representative of an approximately normal population. Check the shape and calculate a 95% t interval for $\mu$.

The values are balanced around 10, with no apparent extreme observation. This is a plausible shape for a small-sample t interval. The random-sampling and independence assumptions are stipulated in the hypothetical setup; in an actual analysis, they would need to be justified by how the data were collected.

The sample mean is $\bar{x}=10$ units. The sum of squared deviations is $ (8-10)^2+(9-10)^2+(10-10)^2+(11-10)^2+(12-10)^2=10$ units squared. Thus $s=\sqrt{10/(5-1)}=\sqrt{2.5}=1.581$ units, and $SE=s/\sqrt{5}=0.7071$ units. The degrees of freedom are $df=5-1=4$; the 95% critical value is approximately $t^*=2.776$.

\begin{aligned} \bar{x}\pm t^*SE &=10\pm 2.776(0.7071)\\ &=10\pm 1.963\\ &=(8.037,\ 11.963)\text{ units}. \end{aligned}

The interval is centered at the sample mean and extends about 1.963 units in each direction. Its suitability depends on the stated sampling and shape assumptions, not merely on the fact that a $t$ critical value was used. Under the assumed conditions, it is reasonable to use this 95% procedure to estimate the population mean.

Strong Skewness Can Make the Interval Unreliable

Now consider a sample with several small values and one much larger value. The sample mean is pulled toward the large observation, while the sample standard deviation also increases. The symmetric t interval then uses those affected quantities to estimate uncertainty. The formula still produces two endpoints, but it cannot make a highly asymmetric small-sample pattern behave like a normal one.

Worked Example: A Strongly Right-Skewed Sample

Suppose five randomly and independently selected hypothetical observations, measured in units, are 2, 3, 3, 4, and 18. Assume the variable cannot be negative. Calculate the nominal 95% t interval, then assess whether it is trustworthy.

The sum is 30, so $\bar{x}=30/5=6$ units. The value 18 is far above the other four, giving a strongly right-skewed pattern. The random and independence conditions are assumed for this example, but the small-sample shape condition is not supported.

The squared deviations from 6 are 16, 9, 9, 4, and 144, for a total of 182. Therefore $s=\sqrt{182/4}=\sqrt{45.5}=6.745$ units, and $SE=6.745/\sqrt{5}=\sqrt{9.1}=3.017$ units. As before, $df=4$ and $t^*=2.776$.

\begin{aligned} \bar{x}\pm t^*SE &=6\pm 2.776(3.017)\\ &=6\pm 8.374\\ &=(-2.374,\ 14.374)\text{ units}. \end{aligned}

The calculation is internally consistent, but the interval extends below zero even though the variable cannot be negative. That is a warning about interpretation, not a reason to silently replace the lower endpoint with zero. With only five observations and strong skewness, the nominal 95% level does not guarantee 95% coverage. The interval may be substantially affected by the one high value and may not represent the uncertainty well.

An outlier can matter through both parts of the interval. The mean determines its center, and the standard deviation contributes to its width. Increasing $s$ does not automatically repair the problem: the width may grow, but the t interval remains symmetric around a mean that the outlier has shifted. Nor does a wide interval prove that the procedure has correct coverage. Coverage is a property of the procedure over repeated samples under its assumptions.

Why One Extreme Value Has Leverage

A short algebraic result makes the effect of adding an observation precise. Let the original sample have size $n$, mean $\bar{x}$, and sum of squared deviations $SS=\sum_{i=1}^{n}(x_i-\bar{x})^2$. Add a value $M$, and write $d=M-\bar{x}$. The new mean is $\bar{x}'=(n\bar{x}+M)/(n+1)=\bar{x}+d/(n+1)$. For each old observation, its deviation from the new mean is $(x_i-\bar{x})-d/(n+1)$. Squaring and summing these old deviations gives $SS+n d^2/(n+1)^2$, because $\sum_{i=1}^{n}(x_i-\bar{x})=0$. The new observation's deviation from the new mean is $nd/(n+1)$, whose square is $n^2d^2/(n+1)^2$. Adding that squared deviation yields

SS_{\mathrm{new}} =SS+\frac{n}{n+1}(M-\bar{x})^2.

The added term grows with the square of the distance between $M$ and the old mean. Thus a value far from the rest of a small sample can greatly increase the sum of squared deviations and the sample standard deviation. At the same time, the new mean shifts by $(M-\bar{x})/(n+1)$. This explains why examining the actual data shape matters: a single extreme observation can affect both the center and spread used in the interval.

Worked Example: Replacing an Ordinary Value With an Extreme One

Compare two hypothetical samples of five measurements in hours. The first is 9, 10, 10, 11, and 12. In the second, the 12 is replaced by 30, giving 9, 10, 10, 11, and 30. Suppose each sample would otherwise meet the random-sampling and independence conditions. Calculate the 95% t intervals and assess the change.

For the first sample, $\bar{x}=52/5=10.4$ hours. The squared deviations sum to $5.2$, so $s=\sqrt{5.2/4}=\sqrt{1.3}=1.140$ hours and $SE=\sqrt{1.3/5}=\sqrt{0.26}=0.5099$ hours. With $df=4$ and $t^*=2.776$, the margin of error is $2.776(0.5099)=1.415$ hours. The interval is

10.4\pm 1.415=(8.985,\ 11.815)\text{ hours}.

For the second sample, the sum is 70, so $\bar{x}=14$ hours. The squared deviations from 14 sum to $25+16+16+9+256=322$. Therefore $s=\sqrt{322/4}=\sqrt{80.5}=8.972$ hours and $SE=\sqrt{80.5/5}=\sqrt{16.1}=4.012$ hours. The margin of error is $2.776(4.012)=11.139$ hours, giving

14\pm 11.139=(2.861,\ 25.139)\text{ hours}.

Replacing one observation shifted the sample mean from 10.4 to 14 hours and increased the estimated standard error from 0.5099 to 4.012 hours. The second sample has a prominent high value and is strongly right-skewed. Its interval is much wider, but the width alone does not make the interval reliable: the small sample shape still undermines the usual t-based justification. An analyst should investigate the observation and report the limitation rather than treat the numerical interval as decisive.

Making a Careful Judgment

When a small sample is at issue, use the data shape to qualify—not replace—the formal conditions. A nearly symmetric sample with no extreme values supports the normal-population approximation better than a strongly skewed sample does. If a possible outlier appears, first check whether it is a data error or a valid observation. Removing a valid observation solely because it changes the answer is not justified. If it is valid, describe its influence and consider whether the t interval is an adequate summary.

A fragile interval is not automatically useless, and a suspicious shape does not by itself establish what the population distribution is. The correct conclusion is more measured: the stated confidence level has its usual justification only when the required assumptions are reasonable. If a small sample is markedly skewed or contains an influential outlier, report that the interval may not achieve its nominal coverage and avoid presenting its endpoints as more certain than the data warrant.

Mistake: “The formula gives a 95% interval, so it has 95% coverage for any data shape.”
Correction: The 95% interpretation depends on the sampling assumptions and an appropriate model. For small samples, strong skewness or outliers can undermine that model.
Mistake: “The interval is wide, so it must account for the outlier correctly.”
Correction: The outlier affects the estimated mean and standard deviation, but the interval remains symmetric and its nominal coverage is not assured when assumptions fail.
Mistake: “A negative endpoint proves that the calculation is wrong.”
Correction: The arithmetic may be right while the interval is poorly suited to the variable or data shape. Treat an impossible endpoint as a reason to examine the model and assumptions, not to alter the result without justification.

Key takeaway: With a small sample, judge the shape before relying on a one-sample t interval. Strong skewness or an influential outlier can shift $\bar{x}$, inflate $s$, and make the stated confidence level unreliable, even when every calculation is correct.

Check Your Understanding

Use the assumptions and examples above to answer these questions.

Why is an approximately normal population especially important when a one-sample t interval is based on a small sample?
A sample is strongly right-skewed but was randomly selected and has independent observations. Which condition remains a concern, and why?
For the sample 2, 3, 3, 4, 18, why does the interval extending below zero not justify simply changing its lower endpoint to zero?
In the update formula for $SS_{\mathrm{new}}$, how does the distance between a new value $M$ and the old mean affect the added sum of squared deviations?
Why does a wider interval after an extreme observation not, by itself, establish that the nominal confidence level is attained?