Degrees Of Freedom From Parameter Restrictions

Why Parameter Dimensions Determine Degrees of Freedom

A likelihood-ratio test compares the best fit available in a broader model with the best fit under a nested, restricted model. The test statistic measures the loss of fit caused by the restrictions. Under the regular conditions of Wilks’ theorem, that loss has an asymptotic chi-square distribution. Its degrees of freedom are determined by how many independently varying parameter dimensions the restrictions remove.

This count is not necessarily the number of parameter names in the model, nor is it always the number of written equations. The key is the dimension of each parameter space: how many independent quantities can vary while remaining in that space. A single parameter equality can remove one dimension; several equations may remove fewer dimensions if some are redundant.

Definition: For a regular likelihood-ratio test of a null model nested in an alternative model, the degrees of freedom are $$ \nu = \dim(\Theta_{\text{alt}})-\dim(\Theta_0), $$ where $\Theta_{\text{alt}}$ and $\Theta_0$ are the alternative and null parameter spaces. Under the usual regularity conditions, the likelihood-ratio statistic $T=2(\ell_{\text{alt}}-\ell_0)$ has an asymptotic $\chi^2_\nu$ distribution under the null. Here $\ell_{\text{alt}}$ and $\ell_0$ are the maximized log-likelihoods in the respective models.

Counting Dimensions and Restrictions

If an alternative model has a parameter vector with $k$ freely varying components, its dimension is generally $k$. If the null model imposes $r$ independent restrictions and otherwise retains those parameters, its dimension is $k-r$, so the degrees of freedom are $r$. The word independent is essential: restrictions that imply one another do not each remove an additional dimension.

For example, with a three-component parameter vector $(\theta_1,\theta_2,\theta_3)$, imposing $\theta_1=\theta_2$ and $\theta_2=\theta_3$ leaves one freely varying parameter, not three. The alternative has dimension three, the null has dimension one, and the difference is two. In contrast, writing both $\theta_1=\theta_2$ and $\theta_2=\theta_1$ does not impose two independent restrictions; they are the same condition.

More generally, restrictions may be written as equations $g_j(\theta)=0$. At a regular point, the number of independent restrictions is the rank of the matrix of their first derivatives with respect to the model parameters. This local rank gives the dimension removed from the parameter space. The familiar “number of restrictions” shortcut is valid when the restrictions are independent and the null parameter space is regular.

Theorem: If the null model is a regular submodel of the alternative, both models are identifiable, the true null parameter is an interior point, and the required likelihood regularity conditions hold, then Wilks’ theorem gives $T \overset{d}{\longrightarrow} \chi^2_\nu$, with $\nu=\dim(\Theta_{\text{alt}})-\dim(\Theta_0)$. This dimension rule does not by itself establish that the regularity conditions hold; in particular, a restriction at a boundary requires separate care.

Why the Dimension Difference Appears

The reason for the rule can be seen locally. Near the true parameter, a regular log-likelihood behaves approximately like a quadratic function of the parameter estimates. In directions allowed by both models, the null and alternative fits can make the same local adjustment. Those shared directions do not contribute to the asymptotic difference in fit. Only directions available in the alternative but removed by the null remain, and there are $\dim(\Theta_{\text{alt}})-\dim(\Theta_0)$ such directions.

In the local quadratic approximation, each such independent direction contributes one squared, standardized estimation error. The sum of $\nu$ independent squared standard normal quantities has a $\chi^2_\nu$ distribution. This explains why the degrees of freedom count removed dimensions rather than observations, likelihood terms, or parameter labels. The approximation is asymptotic; it does not state that every finite-sample likelihood-ratio statistic has an exact chi-square distribution.

Describe the alternative parameter space.
Count the independent quantities that may vary in the broader model.

Apply the null restrictions.
Determine how many free quantities remain, accounting for the model’s existing parameter constraints.

Subtract dimensions.
Use the alternative dimension minus the null dimension to obtain the Wilks chi-square degrees of freedom.

Worked Examples

Worked Example: Testing a Fixed Mean in a Normal Model

A hypothetical sample consists of four independent observations $1,2,3,4$ from a Normal distribution with unknown mean $\mu$ and unknown variance $\sigma^2>0$. Test the null model $\mu=0$ against the alternative model with unrestricted $\mu$. Find the degrees of freedom and calculate the likelihood-ratio statistic. Use the maximum-likelihood variance estimate, with divisor $n$.

Method and dimensions. The alternative has two free parameters, $\mu$ and $\sigma^2$, so its dimension is two. Under the null, $\mu$ is fixed at zero but $\sigma^2$ remains free, so the null dimension is one. Therefore, $\nu=2-1=1$. The null retains the unknown variance; it is not a model in which both Normal parameters are fixed.

Alternative fit. The sample mean is $\bar{x}=(1+2+3+4)/4=2.5$. The sum of squared deviations from that mean is $ (1-2.5)^2+(2-2.5)^2+(3-2.5)^2+(4-2.5)^2=2.25+0.25+0.25+2.25=5$. Thus $\widehat{\sigma}^2_{\text{alt}}=5/4=1.25$.

Null fit. With $\mu=0$, the sum of squared deviations is $1^2+2^2+3^2+4^2=30$. Therefore $\widehat{\sigma}^2_0=30/4=7.5$.

For a Normal sample with fitted mean and maximum-likelihood variance $S/n$, the maximized log-likelihood is $-\frac{n}{2}[\ln(2\pi)+1+\ln(S/n)]$, where $S$ is the fitted sum of squared deviations. The common terms cancel in the difference, giving

T=2(\ell_{\text{alt}}-\ell_0) =4\ln\left(\frac{7.5}{1.25}\right) =4\ln(6) \approx 7.167.

The statistic is compared with an asymptotic $\chi^2_1$ distribution, subject to the regularity conditions. Check: the alternative variance estimate is smaller because its mean is fitted to the data, and the likelihood-ratio statistic is nonnegative. A common mistake is to count the unknown variance as a restriction; it is free under both models and does not add a degree of freedom.

Worked Example: Equal Rates Across Three Poisson Groups

A hypothetical study records independent Poisson claim counts of $1,4,7$ in three groups, each observed for one equal exposure unit. The alternative assigns a separate rate $\lambda_j$ to each group; the null requires all three rates to be equal. Find the degrees of freedom and the likelihood-ratio statistic.

Method and dimensions. Under the alternative, three group rates vary independently, so the dimension is three. Under the null, the groups share one rate, so the dimension is one. The null has two fewer dimensions and $\nu=3-1=2$. Equivalently, two independent equalities are needed to make three rates equal.

Alternative estimates. With one exposure unit per group, the rate estimates are the observed counts: $\widehat{\lambda}_1=1$, $\widehat{\lambda}_2=4$, and $\widehat{\lambda}_3=7$.

Null estimate. The shared rate estimate is the total count divided by total exposure: $\widehat{\lambda}_0=(1+4+7)/3=4$.

For a Poisson count $x_j$ with rate $\lambda_j$ over one exposure unit, the log-likelihood contribution is $x_j\ln(\lambda_j)-\lambda_j-\ln(x_j!)$. The factorial terms cancel between the fits, and the fitted rate terms $-\lambda_j$ sum to $-12$ under either model. Therefore,

\begin{aligned} \ell_{\text{alt}}-\ell_0 &=1\ln(1/4)+4\ln(4/4)+7\ln(7/4)\\ &=-\ln(4)+7\ln(7/4)\\ &=7\ln(7)-8\ln(4) \approx 2.5310,\\ T&=2(2.5310)\approx 5.0620. \end{aligned}

Thus the reference distribution is asymptotically $\chi^2_2$. Check: the null rate, four, is the pooled count divided by pooled exposure, not the unweighted average of rates if exposures differ. Here exposures are equal, so the pooled estimate is also the average of the three alternative estimates.

Worked Example: Multiple Restrictions in a Multinomial Model

A hypothetical sample of 60 policies is classified into four mutually exclusive categories, with counts $12,8,18,22$. Under the alternative, the four category probabilities are unrestricted subject only to summing to one. The null requires the first two probabilities to be equal and the last two probabilities to be equal. Find the degrees of freedom and calculate the likelihood-ratio statistic.

Method and dimensions. Four probabilities subject to one sum-to-one constraint give the alternative dimension $4-1=3$. Under the null, write the probabilities as $(a,a,b,b)$. Since $2a+2b=1$, only one of $a,b$ can vary freely; the null dimension is one. Hence $\nu=3-1=2$. The two pair-equality restrictions are independent within the three-dimensional probability space.

Alternative estimates. The fitted probabilities are the observed proportions: $12/60=0.2$, $8/60=0.1333\ldots$, $18/60=0.3$, and $22/60=0.3666\ldots$.

Null estimates. The first pair has 20 observations, so each fitted probability in that pair is $20/(2\cdot60)=1/6$. The second pair has 40 observations, so each fitted probability there is $40/(2\cdot60)=1/3$. These probabilities sum to $2(1/6)+2(1/3)=1$.

For multinomial counts $n_i$, the likelihood-ratio statistic is $2\sum_i n_i\ln(\widehat{p}_{i,\text{alt}}/\widehat{p}_{i,0})$; the multinomial coefficient cancels. Substitution gives

\begin{aligned} T=2\bigg[ &12\ln(0.2/(1/6)) +8\ln((8/60)/(1/6))\\ &+18\ln(0.3/(1/3)) +22\ln((22/60)/(1/3)) \bigg]\\ =2[&12\ln(1.2)+8\ln(0.8)+18\ln(0.9)+22\ln(1.1)]\\ &\approx 1.2061. \end{aligned}

The asymptotic reference distribution is $\chi^2_2$. Check: the null fitted probabilities preserve the observed total in each pair, as required by maximizing the likelihood subject to the pair-equality restrictions. A common mistake is to count four alternative probabilities as four dimensions; the sum-to-one condition means there are only three.

Common Mistakes and Limits of the Rule

Counting parameters but ignoring constraints already present. Multinomial probabilities sum to one, so $m$ category probabilities have dimension $m-1$, not $m$.
Counting equations instead of independent restrictions. Duplicate or implied equations do not each reduce the dimension.
Dropping shared nuisance parameters. A parameter that remains free under both models belongs in both dimension counts and cancels in the difference.
Using the sample size as the degrees of freedom. Wilks’ degrees of freedom come from the difference in model dimensions, not the number of observations.
Applying the rule without checking its setting. The usual chi-square conclusion is asymptotic and relies on regularity. A parameter on a boundary or other failure of regularity can change the limiting distribution; a dimension count alone does not settle that issue.

Key takeaway: First identify the free dimensions in the alternative and null models; then subtract. For regular nested models, each independent dimension removed by the null contributes one degree of freedom to the asymptotic likelihood-ratio chi-square distribution.

Check Your Understanding

For each setting, determine the degrees of freedom under the regular nested-model assumptions. Where a statistic is requested, calculate it from the stated data.

A Normal model has three free parameters $(\mu,\sigma^2,\eta)$. The null fixes $\eta=2$, while $\mu$ and $\sigma^2$ remain free. What are the degrees of freedom? (Answer: 1)
A five-category multinomial alternative has unrestricted category probabilities. The null requires $p_1=p_2=p_3$ and $p_4=p_5$. What are the alternative and null dimensions, and the degrees of freedom? (Answer: dimensions 4 and 1; degrees of freedom 3)
Four Poisson groups have separate alternative rates. The null imposes $\lambda_1=\lambda_2$ and $\lambda_3=\lambda_4$, with the two shared rates otherwise free. What are the degrees of freedom? (Answer: 2)
An alternative has three free parameters $(a,b,c)$. The null imposes $b=2a$ and $c=a^2$, with $a$ free in the allowed interior range. Assuming these constraints are independent, what are the null dimension and degrees of freedom? (Answer: null dimension 1; degrees of freedom 2)
A five-dimensional alternative is restricted by equations $h_1(\theta)=0$, $h_2(\theta)=0$, and $h_3(\theta)=h_1(\theta)+h_2(\theta)=0$. If the first two restrictions are independent and the model is regular, what is the null dimension and the degrees of freedom? (Answer: null dimension 3; degrees of freedom 2)

What You'll Learn