Computing Likelihood-Ratio Tests Numerically

Why Numerical Checks Matter

A likelihood-ratio test can be written compactly, but its calculation depends on two separate optimizations: one under the null hypothesis and one under the alternative. A numerical optimizer can report a result even when it has stopped at a poor local solution, approached an excluded parameter value, or optimized a likelihood different from the one used in the other fit. The statistic is only meaningful when both fits represent the intended models and data.

For nested models, the unrestricted model contains every parameter choice allowed by the restricted model. Numerical fitting should preserve this relationship: the alternative fit must be at least as good as the null fit when both maximize the same likelihood over the same observations. That inequality is a useful diagnostic, not a substitute for checking the fits.

Definition: Let $\ell_0$ be the maximized log-likelihood under the null hypothesis and $\ell_1$ the maximized log-likelihood under the alternative. The likelihood-ratio statistic is $$ T=-2\log\left(\frac{\sup_{H_0}L}{\sup_{H_1}L}\right)=2(\ell_1-\ell_0). $$ For nested models with a correctly optimized common likelihood, $\ell_1\geq\ell_0$, so $T\geq0$. The usual chi-square calibration requires the relevant regularity conditions; computing $T$ does not itself establish that those conditions hold.

A Reliable Numerical Workflow

Numerical fitting is usually more stable on the log-likelihood scale. Products of many small probability terms can underflow, whereas their logarithms add. For independent observations, write the log-likelihood as a sum of observation-level contributions and use the same contributions, constants, data, and observation units in both fits. Constants that truly are identical in the two models cancel in the difference, but retaining them consistently helps expose mismatches.

Before optimizing, state which parameters are fixed under the null, which are free under the alternative, and what values are permitted. For example, a rate $\lambda$ must satisfy $\lambda>0$, while a probability $p$ must satisfy $0\leq p\leq1$. A common numerical device for a strictly positive parameter is to optimize over $\eta=\log\lambda$ and set $\lambda=e^\eta$. This guarantees positivity, but it does not permit $\lambda=0$; if zero is an allowed boundary, that possibility must be assessed separately.

Write both likelihoods.
Use the same observations and likelihood definition under the null and alternative, identifying restrictions and parameter domains explicitly.

Fit the restricted model.
Optimize all nuisance parameters that remain free under the null; do not treat them as fixed unless the hypothesis says so.

Fit the alternative.
Use admissible parameter values and, for a non-concave objective, consider different starting values or boundary candidates.

Check and compare.
Inspect convergence status, parameter bounds, and whether the estimates and log-likelihood are plausible. For nested models, confirm that $\ell_1\geq\ell_0$, then calculate $T=2(\ell_1-\ell_0)$.

A convergence flag alone is not enough. Useful checks include whether the final gradient is small for an interior solution, whether a boundary solution was missed, and whether nearby starting values produce a better log-likelihood. Also check that no estimate was rounded before evaluating the likelihood. Keep full working precision through the subtraction and round the final statistic only.

Why the Statistic Uses Twice the Log-Likelihood Difference

The likelihood ratio compares the best likelihood permitted by the null with the best likelihood permitted by the alternative. Taking a logarithm turns the ratio into a difference, and multiplying by $-2$ gives the conventional scale used for likelihood-ratio tests.

\Lambda=\frac{\sup_{H_0}L}{\sup_{H_1}L}, \qquad -2\log\Lambda =-2(\ell_0-\ell_1) =2(\ell_1-\ell_0).

For nested models, every null parameter choice is also available under the alternative. Therefore, the alternative supremum cannot be smaller than the null supremum: $\sup_{H_1}L\geq\sup_{H_0}L$. The ratio satisfies $0<\Lambda\leq1$, which gives $T\geq0$. If a calculation gives a negative statistic, check the order of subtraction, whether both fits used the same likelihood, and whether the alternative was actually optimized over a set containing the null model.

Worked Numerical Fits

Worked Example: A Poisson Rate Restriction

Five independent hypothetical annual claim counts have total count 15. Under $H_0$, each count has a Poisson distribution with fixed mean $\lambda=2$. Under $H_1$, the counts remain independent Poisson variables with a common mean $\lambda>0$, estimated from the data. Find the MLE under $H_1$ and calculate $T$. The method is maximum likelihood on the log scale; the main trap is reporting the log-likelihood difference without multiplying it by two.

For observed counts $x_1,\ldots,x_5$, the log-likelihood is

\ell(\lambda)=\sum_{j=1}^{5} \left(x_j\log\lambda-\lambda-\log(x_j!)\right).

The terms involving the observations enter through their total, $15$. Under the unrestricted alternative,

\frac{d\ell}{d\lambda}=\frac{15}{\lambda}-5, \qquad \widehat{\lambda}_1=\frac{15}{5}=3.

The second derivative is $-15/\lambda^2<0$ for $\lambda>0$, so this stationary point is a maximum. Under the null, $\lambda=2$. The factorial terms are identical in the two fits and cancel:

\ell_1-\ell_0 =15\log\left(\frac{3}{2}\right)-5(3-2) =15(0.4054651)-5 =1.0819766.

Thus $T=2(1.0819766)=2.1639532$, or $T=2.1640$ to four decimal places. The alternative fit has a greater log-likelihood, as required for these nested models. The common mistake is to stop at $1.0819766$; that is $\ell_1-\ell_0$, not the likelihood-ratio statistic.

Worked Example: A Common Probability Versus Separate Group Probabilities

Two hypothetical independent Bernoulli groups have 6 successes in 10 trials and 9 successes in 20 trials. Under $H_0$, both groups share one success probability $p$. Under $H_1$, each group has its own probability. Find the pooled null estimate and the statistic. Use the Bernoulli likelihood for the observed trials; the method avoids treating the two group proportions as if they were the null estimate.

Under the null, there are 15 successes in 30 trials, so the maximum-likelihood estimate is

\widehat p_0=\frac{6+9}{10+20}=\frac{15}{30}=0.5000.

Under the alternative, the two estimates are $\widehat p_1=6/10=0.6$ and $\widehat p_2=9/20=0.45$. For Bernoulli observations, the log-likelihood contribution from a group with $x$ successes and $n-x$ failures is $x\log p+(n-x)\log(1-p)$. Consequently,

\begin{aligned} \ell_1-\ell_0={}& 6\log(0.6/0.5)+4\log(0.4/0.5)\\ &+9\log(0.45/0.5)+11\log(0.55/0.5)\\ ={}&6(0.1823216)+4(-0.2231436) +9(-0.1053605)+11(0.0953102)\\ ={}&0.3015224. \end{aligned}

Therefore $T=2(0.3015224)=0.6030448$, which is $0.6030$ to four decimal places. Both alternative probabilities are inside the allowed interval, and the alternative log-likelihood exceeds the pooled null log-likelihood. Using either group proportion as the null estimate would fail to maximize the common-probability likelihood.

Worked Example: A Normal Mean Test With Known Variance

Three independent hypothetical measurements are $2,4,6$. Assume each has a normal distribution with mean $\mu$ and known variance $\sigma^2=8$. Test $H_0:\mu=3$ against an unrestricted mean under $H_1$. Find the alternative MLE and $T$. This example illustrates a numerical fit that can be checked against the sample mean.

The unrestricted maximum-likelihood estimate is the sample mean:

\widehat{\mu}_1=\frac{2+4+6}{3}=4.

The sum of squared deviations is $11$ under the null and $8$ under the alternative:

\operatorname{RSS}_0=(2-3)^2+(4-3)^2+(6-3)^2=1+1+9=11, $$ $$ \operatorname{RSS}_1=(2-4)^2+(4-4)^2+(6-4)^2=4+0+4=8.

With known variance, the normal log-likelihood difference is $(\operatorname{RSS}_0-\operatorname{RSS}_1)/(2\sigma^2)$. Here it is $(11-8)/(2\cdot8)=3/16=0.1875$, so

$$ T=2(0.1875)=0.3750. $$

The final statistic is $T=0.3750$ to four decimal places. The check is that the fitted mean $4$ is the sample average and reduces the residual sum of squares. Keep the known variance equal to 8 throughout; it is not estimated from these three observations.

Worked Example: Checking a Normal Fit With Unknown Variance

Three independent hypothetical observations are $1,2,4$, each modeled as $N(\mu,\sigma^2)$ with both parameters unknown under the alternative. Compare $H_0:\mu=2$, with $\sigma^2$ still free, against the unrestricted alternative in which both $\mu$ and $\sigma^2$ are estimated. Calculate $T$. This fit demonstrates why the null nuisance parameter must also be optimized.

Under the null, the residual sum of squares is $5$; under the alternative, $\widehat\mu_1=(1+2+4)/3=7/3$, giving

\operatorname{RSS}_0=(1-2)^2+(2-2)^2+(4-2)^2=5, $$ $$ \operatorname{RSS}_1= \left(1-\frac73\right)^2+ \left(2-\frac73\right)^2+ \left(4-\frac73\right)^2 =\frac{16+1+25}{9} =\frac{14}{3}.

For either fit, maximizing over $\sigma^2>0$ gives $\widehat{\sigma}^2=\operatorname{RSS}/n$, provided the residual sum of squares is positive. The estimates are therefore $5/3$ under the null and $14/9$ under the alternative. Substituting these into the normal log-likelihood and cancelling the shared constants gives

T=3\log\left(\frac{\operatorname{RSS}_0}{\operatorname{RSS}_1}\right) =3\log\left(\frac{5}{14/3}\right) =3\log\left(\frac{15}{14}\right) =0.2070.

The statistic is $0.2070$ to four decimal places. If the residual sum of squares at a fixed mean is zero, the formula for a positive variance MLE does not apply: the likelihood is unbounded as $\sigma^2$ approaches zero, and there is no finite MLE over $\sigma^2>0$. This example has positive residual sums of squares in both fits, so the stated calculation is valid.

Common Numerical Traps

The same algebra can produce a wrong result if the model fit or the interpretation of the output is inconsistent. These checks are especially useful when software reports estimates and log-likelihoods but does not show how the statistic was assembled.

Omitting the factor of two. In the Poisson example, $1.0820$ is the log-likelihood difference. The statistic is twice that value: $2.1640$.
Using the wrong null fit. A null model with a shared probability requires maximizing over that shared probability; it is not generally equal to either group's estimate.
Ignoring constraints. An unconstrained numerical step can propose a negative rate or a probability outside $[0,1]$. Such a point is not an admissible fit.
Comparing different likelihoods. Different data, omitted observations, or inconsistent treatment of likelihood terms invalidate the subtraction of maximized log-likelihoods.
Assuming every optimizer result is a maximum. A convergence message does not rule out a boundary maximum, a poor local solution, or a flat direction. Compare plausible starts and inspect the fitted values.
Accepting a negative statistic for nested models. Since the unrestricted parameter space contains the restricted one, a negative value usually signals a fitting or bookkeeping problem.

Key takeaway: Fit the null and alternative on the same data and likelihood, enforce each model's parameter constraints, verify that the alternative maximization is no worse for nested models, and calculate $T$ as twice the log-likelihood difference. Interpret or calibrate that statistic only after checking the test's assumptions.

Check Your Understanding

For each problem, compute the requested estimates and statistic. Round statistics to four decimal places.

Four independent hypothetical Poisson claim counts total 10. Test a fixed null mean $\lambda=2$ against a common freely estimated mean. Find the alternative MLE and $T$. (Answer: $\widehat{\lambda}_1=2.5,\ T=0.4629$)
Two independent hypothetical Bernoulli groups have 4 successes in 8 trials and 8 successes in 12 trials. Compare one common probability with separate probabilities. Find the pooled null estimate and $T$. (Answer: $\widehat p_0=0.6000,\ T=0.5538$)
Independent normal observations are $3,5,8$, with known variance $\sigma^2=9$. Test $H_0:\mu=5$ against a freely estimated mean. Find $\widehat\mu_1$ and $T$. (Answer: $\widehat\mu_1=16/3,\ T=0.0370$)
Six independent hypothetical Poisson counts total 12. Test $H_0:\lambda=2.5$ against a freely estimated common mean. Find the alternative MLE and $T$. (Answer: $\widehat{\lambda}_1=2,\ T=0.6448$)
Independent observations $0,2,7$ follow $N(\mu,\sigma^2)$ with unknown variance. Compare $H_0:\mu=2$, with variance estimated under the null, against the unrestricted alternative. Find both variance estimates and $T$. (Answer: $\widehat{\sigma}_0^2=29/3,\ \widehat{\sigma}_1^2=26/3,\ T=0.3276$)

What You'll Learn