Why the Sampling Design Comes First
Two columns of measurements do not, by themselves, tell you which interval for a difference in means to use. The key question is how the observations were collected. If each observation in one sample is naturally linked to a particular observation in the other, the data are paired. If the samples consist of separate, independently selected units, they are independent samples. The distinction changes both the standard error and, for t intervals, the degrees of freedom.
For example, two measurements from the same person, device, or location are often paired. Measurements from two separately sampled groups are typically independent if the sampling designs support that assumption. Treating matched measurements as independent discards their within-pair relationship. Conversely, inventing pairs between unrelated observations does not create a paired design.
Which Standard Error Matches the Design?
Suppose the target is the difference between population means, \(\mu_X-\mu_Y\). For paired data, first form the \(n\) differences \(D_i=X_i-Y_i\). The estimate is \(\bar D\), and the paired standard error is \(s_D/\sqrt n\), where \(s_D\) is the sample standard deviation of the differences. Under independent sampling, the estimated standard error instead combines the variability of the two sample means.
For two independent samples with sizes \(n_X,n_Y\), sample means \(\bar X,\bar Y\), and sample standard deviations \(s_X,s_Y\), the estimated standard error is \(\sqrt{s_X^2/n_X+s_Y^2/n_Y}\). If the population variances are unequal or not assumed equal, the Welch interval uses this standard error and an approximate degrees-of-freedom formula. If independent normal populations have equal variances, a pooled t interval instead estimates their common variance and uses \(n_X+n_Y-2\) degrees of freedom.
The Welch degrees of freedom are generally not an integer. They approximate the reference degrees of freedom for the estimated standard error; they are not the count of observations or pairs. For the pooled method, the equal-variance assumption is important: the pooled variance is not justified merely because both samples are normally distributed.
Why Pairing Changes the Standard Error
The relationship between measurements in a pair is part of the variability of their difference. To see this, suppose each pair has the same marginal variances \(\sigma_X^2\) and \(\sigma_Y^2\), and the covariance within a pair is \(\operatorname{Cov}(X,Y)\). The variance identity for a difference gives
Because independent pairs have independent differences, the variance of their average is \(\operatorname{Var}(D)/n\). Thus the paired standard error estimates the variability of the actual pairwise differences, not the variability of two unrelated sample means. In contrast, for independent samples the covariance between the two sample means is zero, so
This derivation explains why the independent-samples formula cannot simply be applied to matched observations: it omits the within-pair covariance term. Positive within-pair association often makes differences less variable, but the effect is not guaranteed to be a reduction; negative association can make them more variable. The appropriate calculation follows the design, not whichever standard error happens to be smaller.
Ask whether each value in one group is linked to a specific value in the other group by person, object, location, or deliberate matching.
State the direction, such as \(D_i=X_i-Y_i\), and calculate the mean and sample standard deviation of those differences.
Choose Welch unless equal population variances are assumed and the pooled method is appropriate. Use the degrees of freedom belonging to the selected method.
Worked Example: A Paired Interval from Matched Device Readings
Worked Example: A Paired Interval from Matched Device Readings
A technician tests six sensor units under two calibration settings. Each unit is measured under both settings, and the readings are in milliseconds. For this hypothetical study, assume the six units are independent of one another and that the population of within-unit differences is normal. The readings under settings A and B are, respectively, \(10,12,13,14,16,17\) and \(8,8,10,9,15,14\). Construct a 95% interval for the population mean difference, defining the difference as A minus B.
Method. These are paired readings because both settings were applied to each same unit. Calculate the differences \(D_i=A_i-B_i\): \(2,4,3,5,1,3\) milliseconds.
Their sum is \(2+4+3+5+1+3=18\), so \(\bar D=18/6=3\) milliseconds. The squared deviations from 3 are \(1,1,0,4,4,0\), with sum 10. Therefore \(s_D^2=10/(6-1)=2\), \(s_D=\sqrt2\approx1.414\), and \(SE=s_D/\sqrt6=\sqrt{2/6}\approx0.577\) milliseconds.
There are \(6-1=5\) degrees of freedom. The 95% two-sided critical value is \(t_{0.975,5}\approx2.571\). The margin of error is \(2.571(0.57735)\approx1.484\) milliseconds. The interval is \(3\pm1.484\), or \((1.516,4.484)\) milliseconds, rounded to three decimals.
Check. The interval is centered at the observed mean difference of 3, and its half-width is approximately 1.484. The common mistake this method avoids is treating the two readings from each unit as unrelated observations.
Worked Example: The Error from Treating Paired Data as Independent
Worked Example: The Error from Treating Paired Data as Independent
Use the same hypothetical sensor readings from the preceding calculation. A candidate ignores the matching and applies a Welch interval to the two columns as if they came from independent samples. Show what that calculation produces, then identify the correct interval.
Incorrect method. For setting A, the mean is \(82/6=13.667\). The sum of squared deviations from this mean is \(33.333\), so \(s_A^2=33.333/5=6.667\). For setting B, the mean is \(64/6=10.667\); its squared deviations sum to \(47.333\), so \(s_B^2=47.333/5=9.467\). The difference of sample means is \(13.667-10.667=3\).
The incorrectly selected independent-sample standard error is \(SE=\sqrt{6.667/6+9.467/6}=\sqrt{2.689}\approx1.640\). The Welch degrees of freedom are approximately \((2.689)^2/\{(1.111)^2/5+(1.578)^2/5\}\approx9.71\). Using \(t_{0.975,9.71}\approx2.237\), the incorrect margin is approximately \(2.237(1.640)=3.669\), producing the interval \((-0.669,6.669)\) milliseconds.
Correction. The calculation is inappropriate because the settings were measured on the same six units. The proper analysis uses the six differences, giving the paired interval \((1.516,4.484)\) milliseconds from the preceding example. The two methods do not differ because one formula is generally more conservative; they differ because only one reflects the sampling design. The independent calculation fails to use the observed matching.
Check. The difference of the two sample means is still 3, the same as \(\bar D\), but that agreement does not make the standard errors interchangeable. The estimate may match while the uncertainty calculation does not.
Worked Example: A Welch Interval for Independent Groups
Worked Example: A Welch Interval for Independent Groups
In a hypothetical study of a new work-queue process, one randomly selected group of eight staff members uses the current process and a separate group of ten uses the new process. Each person contributes one completion-time measurement, and no person appears in both groups. The groups are independent; assume normal populations, but do not assume equal population variances. The current-process sample has mean 42 minutes and variance 16 minutes squared; the new-process sample has mean 38 minutes and variance 25 minutes squared. Construct a 95% Welch interval for current minus new.
Method. The groups contain different people and are independent, so use Welch’s method. The estimated difference is \(42-38=4\) minutes. The standard error is \(\sqrt{16/8+25/10}=\sqrt{2+2.5}=\sqrt{4.5}\approx2.121\) minutes.
The Welch degrees of freedom are \[ \frac{(2+2.5)^2}{2^2/7+2.5^2/9} =\frac{20.25}{4/7+6.25/9} \approx16.00. \] For about 16 degrees of freedom, \(t_{0.975,16}\approx2.120\). The margin is \(2.120(2.1213)\approx4.497\) minutes, giving the interval \(4\pm4.497=(-0.497,8.497)\) minutes, rounded to three decimals.
Check. The interval is centered at 4, the observed difference in means. This method avoids the opposite design error: forcing a paired analysis when the study has no matching or repeated measurements.
Common Mistakes and Exam Checks
Before calculating, identify the unit of analysis and the sampling relationship. A description that says “each subject was measured twice” signals pairing; a description that says “subjects were randomly assigned to one of two groups” usually signals independent groups, provided no other dependence is introduced. Similar sample sizes or similar measurement types do not create pairs.
| Study design | Calculation | Degrees of freedom |
|---|---|---|
| Same units or matched units, with normal differences | One-sample t interval on the differences | \(n-1\), where \(n\) is the number of pairs |
| Independent groups; equal population variances assumed | Pooled two-sample t interval | \(n_X+n_Y-2\) |
| Independent groups; equal variances not assumed | Welch two-sample t interval | Welch approximate degrees of freedom |
- Counting measurements instead of pairs. With \(n\) matched units measured twice, the paired analysis has \(n\) differences, not \(2n\) independent observations.
- Using \(n_X+n_Y-2\) for paired data. That is the pooled independent-samples degrees-of-freedom formula under its assumptions. A paired t interval has \(n-1\) degrees of freedom.
- Assuming pairing from the variable names. Two columns labeled “before” and “after” are paired only when values are linked to the same units or matched units.
- Assuming the paired method must be more precise. The covariance determines how pairing affects the variance. Choose the method from the design, not by comparing which interval is narrower.
Check Your Understanding
For each problem, identify the design before selecting the calculation. Report interval endpoints to three decimals where needed.
- A lab measures the same five filters before and after a cleaning process. The before-minus-after differences in pressure loss are \(1,3,2,4,5\) units. Assume independent normally distributed differences. Find a 95% paired t interval. (Answer: \(\bar D=3\), \(s_D^2=2.5\), \(SE=\sqrt{0.5}\approx0.707\), df \(=4\); interval \((1.037,4.963)\) units.)
- Six matched pairs produce differences \(1,2,2,3,3,7\). Assume normal differences. Find the standard error and the 95% paired interval. (Answer: \(\bar D=3\), \(s_D^2=22/5=4.4\), \(SE=\sqrt{4.4/6}\approx0.856\), df \(=5\); interval \((0.798,5.202)\).)
- Two independent normal populations have sample sizes 9 and 16, sample means 18 and 15, and sample variances 9 and 16. Do not assume equal variances. Find the Welch standard error and degrees of freedom. (Answer: \(SE=\sqrt2\approx1.414\); df \(=4/(1/8+1/15)=480/23\approx20.87\).)
- Two independent normal samples have sizes 7 and 9, means 11 and 8, and variances 4 and 9. Assume equal population variances. Find the pooled 95% interval for the first mean minus the second. (Answer: \(s_p^2=48/7\), \(SE=\sqrt{256/147}\approx1.320\), df \(=14\); using \(t_{0.975,14}\approx2.145\), interval \((0.170,5.830)\).)
- A study records the operating temperature of each of 12 engines under both a standard setting and a revised setting. A candidate uses the pooled independent-samples degrees of freedom \(12+12-2=22\). What is the design and what degrees of freedom should a paired t interval use, assuming normal differences? (Answer: The design is paired because each engine receives both settings; form 12 within-engine differences and use \(12-1=11\) degrees of freedom.)