Degrees of Freedom and Sample Variance Conventions

Why Two Variance Conventions Arise

For a sample of \(n\) observations, two familiar quantities are both called sample variance. One divides the residual sum of squares by \(n-1\); the other divides it by \(n\). The difference is not merely a naming convention: it changes the exact distribution and the expectation of the estimate. The degrees of freedom in the residual sum of squares, however, remain \(n-1\) under either convention.

The key is to begin with the normal-sample residual sum of squares and then apply the chosen divisor. This tutorial makes that translation explicit. Throughout, assume \(X_1,\ldots,X_n\) are independent and identically distributed as \(N(\mu,\sigma^2)\), with \(\sigma^2>0\) and \(n\geq 2\), and let \(\bar X=n^{-1}\sum_{i=1}^n X_i\).

The Residual Chi-Square Law

The normal-sample residual result gives the starting distribution. In geometric terms, centering removes the component of the sample vector in the direction of the all-ones vector. The residual vector lies in a subspace of dimension \(n-1\), which accounts for the degrees of freedom.

This result concerns \(Q/\sigma^2\), not \(Q\) itself: \(Q\) has the scale \(\sigma^2\). The two variance conventions follow by dividing the same equality by different constants.

The degrees of freedom do not change when the divisor changes. Dividing by \(n\) changes the scale of the random variable, not the dimension of the residual subspace.

Deriving the Two Distributions

Write \(\nu=n-1\) and \(U=Q/\sigma^2\), so that \(U\sim\chi^2_\nu\). By definition, \(S^2=\sigma^2 U/\nu\) and \(V_n=\sigma^2 U/n\). These identities prove the scaled chi-square statements directly. They also yield density formulas if a density, rather than a scaled-distribution representation, is needed.

The chi-square density is \[ f_U(u)=\frac{u^{\nu/2-1}e^{-u/2}}{2^{\nu/2}\Gamma(\nu/2)},\qquad u>0, \] where \(\Gamma\) is the gamma function. For \(s>0\), the change of variable \(u=\nu s/\sigma^2\) has derivative \(du/ds=\nu/\sigma^2\). Therefore

For \(V_n\), use \(u=nv/\sigma^2\), with derivative \(du/dv=n/\sigma^2\). This gives

Both densities have the same power \(v^{\nu/2-1}\) or \(s^{\nu/2-1}\), reflecting the same degrees of freedom. Their exponential terms and leading constants differ because their scales differ.

Relationship, Bias, and Variability

The two variance estimates are deterministically related for every observed sample:

To derive their moments, first recall the moments of a chi-square variable. If \(U\sim\chi^2_\nu\), its moment-generating function is \(M_U(t)=(1-2t)^{-\nu/2}\) for \(t<1/2\). This follows by writing \(U\) as a sum of squares of \(\nu\) independent standard normal variables: for \(Z\sim N(0,1)\), direct integration of the normal density gives \(E[e^{tZ^2}]=(1-2t)^{-1/2}\), and independence multiplies these factors. Differentiating the moment-generating function at zero gives

Since \(S^2=\sigma^2 U/\nu\), multiplying the mean by \(\sigma^2/\nu\) and the variance by \((\sigma^2/\nu)^2\) yields \(E[S^2]=\sigma^2\) and \(\operatorname{Var}(S^2)=2\sigma^4/\nu\). Since \(V_n=\sigma^2 U/n\), the corresponding calculations give

The unbiasedness of \(S^2\) does not mean that every sample estimate equals \(\sigma^2\); it means that its expectation over repeated samples equals \(\sigma^2\). The divisor-\(n\) version is smaller for every sample with \(Q>0\), and its expected value is correspondingly smaller. In the normal model with \(\mu\) unknown, \(V_n\) is also the maximum likelihood estimator of \(\sigma^2\); that fact does not make it unbiased.

Degrees of Freedom and Common Mistakes

One degree of freedom is used in estimating the common mean through \(\bar X\). The \(n\) deviations \(X_i-\bar X\) satisfy \(\sum_{i=1}^n(X_i-\bar X)=0\), so they cannot vary independently across all \(n\) coordinates. This constraint leaves \(n-1\) residual directions. Choosing a different divisor for their squared length does not restore the missing direction.

• Changing the divisor and changing the degrees of freedom are not the same operation. Both \(S^2\) and \(V_n\) are scaled versions of a chi-square variable with \(n-1\) degrees of freedom.
• Do not reverse an event inequality while scaling by a positive constant. From \(S^2=Q/(n-1)\), the event \(S^2>c\) is \(Q>(n-1)c\), not \(Q>c/(n-1)\).
• Do not call both quantities unbiased. Under the stated normal model, \(E[S^2]=\sigma^2\), while \(E[V_n]=(n-1)\sigma^2/n\).

For \(n=1\), there are no residual degrees of freedom: \(Q=0\) and the divisor-\((n-1)\) variance is undefined. The results for \(S^2\) therefore require \(n\geq2\), as assumed throughout.