Worked Problems in Univariate Normal Sampling

Choosing the Exact Sampling Law

Exact sampling calculations for normal data are built from a small set of linked quantities: a sample mean, a residual sum of squares, and ratios formed from them. The important work in a problem is not just recognizing a familiar statistic. It is checking the model assumptions, identifying which quantities are independent, and matching the numerator and denominator degrees of freedom to the statistic actually calculated.

Throughout, \(X_1,\ldots,X_n\) are independent observations from \(N(\mu,\sigma^2)\), where \(\sigma^2>0\) and \(n\ge 2\), unless a different model is stated. Write \(\bar X=n^{-1}\sum_{i=1}^n X_i\), \(RSS=\sum_{i=1}^n(X_i-\bar X)^2\), and \(S^2=RSS/(n-1)\). The divisor \(n-1\) is essential: it makes the residual sum of squares correspond to \(n-1\) degrees of freedom.

Normal Means and Residual Variation

The exact mean law follows directly from closure of independent normal variables under linear combinations: \(\bar X\sim N(\mu,\sigma^2/n)\). The complementary part of the sample vector is its residual variation around \(\bar X\). For a normal sample, the mean and this residual variation are independent; the residual sum of squares, scaled by \(\sigma^2\), has a chi-square law.

To verify the residual degrees of freedom and independence, standardize the observation vector: \(Z_i=(X_i-\mu)/\sigma\). The vector \(Z\) has independent standard normal coordinates. Choose an orthogonal matrix whose first row is \((1/\sqrt n,\ldots,1/\sqrt n)\). Orthogonal transformations preserve the standard multivariate normal density, so the transformed coordinates are again independent standard normals. The first transformed coordinate is \(\sqrt n(\bar X-\mu)/\sigma\). The sum of squares of the remaining \(n-1\) coordinates is \(RSS/\sigma^2\): an orthogonal change of coordinates preserves total squared length, and removing the first coordinate removes \(n(\bar X-\mu)^2/\sigma^2\). Thus the residual sum is chi-square with \(n-1\) degrees of freedom and is independent of the first coordinate, hence independent of \(\bar X\). This proves the theorem. \(\square\)

One-Sample Student Ratios

When \(\sigma\) is unknown, standardizing \(\bar X-\mu\) by the true standard error \(\sigma/\sqrt n\) is not operational. Replacing \(\sigma\) by \(S\) gives a statistic with an exact Student \(t\) law, because the standardized mean is independent of the scaled residual sum of squares.

For the proof, the mean law gives \(Z=\sqrt n(\bar X-\mu)/\sigma\sim N(0,1)\). The preceding result gives \(Q=(n-1)S^2/\sigma^2\sim\chi^2_{n-1}\), independently of \(Z\). Substitution shows \[ \frac{\bar X-\mu}{S/\sqrt n} =\frac{\sqrt n(\bar X-\mu)/\sigma}{\sqrt{(n-1)S^2/\sigma^2\,/(n-1)}} =\frac{Z}{\sqrt{Q/(n-1)}}. \] By the definition of the Student distribution, this is \(t_{n-1}\). \(\square\)

Two Samples: Pooled t and F Ratios

For two independent normal samples, within-sample residual sums of squares provide separate chi-square quantities. If the population variances are equal, adding those sums produces a pooled variance estimate. The resulting pooled t statistic is exact under equality of the population means. If instead the target is a variance comparison, the ratio of the two scaled sample variances has an exact F distribution under equal population variances.

For justification, each sample separately satisfies \((n_j-1)S_j^2/\sigma_j^2\sim\chi^2_{n_j-1}\), independently of its sample mean. The two samples are independent. Under the common-variance condition, the two independent chi-square variables add to a chi-square with \((n_1-1)+(n_2-1)=n_1+n_2-2\) degrees of freedom, and their sum is independent of the difference of sample means. Under \(\mu_1=\mu_2\), that difference divided by \(\sigma\sqrt{1/n_1+1/n_2}\) is standard normal. Dividing it by the square root of the independent pooled chi-square divided by \(n_1+n_2-2\) yields the stated t law. For the F result, divide each sample variance by its own population variance and its degrees of freedom; the resulting independent quantities are chi-square variables divided by their degrees of freedom, which is exactly the defining F ratio. \(\square\)

Assumptions, Degrees of Freedom, and Common Errors

Exactness depends on the model, not simply on the appearance of a statistic. The one-sample t result requires independent observations from a single normal population; its exactness does not follow for an arbitrary population merely because the sample size is moderate. The pooled two-sample t result additionally requires independent samples and equal population variances. The F variance-ratio result requires independent normal samples. When these assumptions are doubtful, the corresponding exact reference law is not justified by the derivations above.

A frequent mistake is to use \(n\) degrees of freedom for a sample variance because there are \(n\) observations. The residuals satisfy \(\sum_i(X_i-\bar X)=0\), so only \(n-1\) residual coordinates can vary freely. The theorem makes this precise: \(RSS/\sigma^2\) is chi-square with \(n-1\) degrees of freedom. Another error is to treat a sample-variance ratio as \(F\) without accounting for the population variances. Its exact law is \(F\) under \(\sigma_1^2=\sigma_2^2\), or after explicitly dividing each sample variance by its own population variance.