What a Likelihood Ratio Compares
When two statistical models are specified completely, the observed data can be evaluated under each model. A likelihood ratio compares those two evaluations: it measures how many times as large the likelihood is under one specified model as under the other. The ratio describes relative support from the observed data; it is not itself the probability that either hypothesis is true.
For this tutorial, the direction is fixed as alternative divided by null. In other words, a ratio above one means the observed data have greater likelihood under the specified alternative than under the specified null. Some texts define the ratio in the opposite direction, so always check which likelihood is in the numerator before interpreting a value.
A simple hypothesis specifies a complete probability distribution, including all parameter values needed to calculate a likelihood. For example, \(H_0:p=0.5\) versus \(H_1:p=0.8\) for Bernoulli observations is a comparison of two simple hypotheses. By contrast, a statement allowing a range of parameter values does not specify one likelihood by itself.
For discrete data, the likelihood at an observed outcome is its probability under the model. For continuous data, it is the probability density at the observed value; a density value is not the probability of observing that exact point. In either case, the likelihood is treated as a function of the model after the data have been observed. Both hypotheses must assign positive likelihood to the observed data for the ratio to be finite and defined in the ordinary way.
Computing the Ratio from Independent Bernoulli Data
Suppose \(X_1,\ldots,X_n\) are independent Bernoulli observations with success probability \(p\), where each observation is either 1 (success) or 0 (failure). If the observed sample contains \(x\) successes and \(n-x\) failures, its likelihood under a specified value of \(p\) is \(p^x(1-p)^{n-x}\) for the particular ordered sample. If the data are summarized by the number of successes, the binomial likelihood also contains the factor \(\binom{n}{x}\); that factor is the same under both hypotheses and cancels in the ratio.
Worked Example: Six Bernoulli Observations
Six independent Bernoulli observations contain four successes and two failures. Under \(H_0\), \(p=0.5\); under \(H_1\), \(p=0.8\). Calculate \(R=L_1/L_0\) and identify which specified model has the larger likelihood.
Method. Use the Bernoulli product likelihood, with four success factors and two failure factors. This avoids confusing a likelihood ratio with a test decision or reversing its defined direction.
Under the null model, $$ L_0=(0.5)^4(1-0.5)^2=(0.5)^6=0.015625. $$ Under the alternative model, $$ L_1=(0.8)^4(1-0.8)^2=(0.4096)(0.04)=0.016384. $$ Therefore, $$ R=\frac{0.016384}{0.015625}=1.048576. $$ Rounded to four decimal places, \(R=1.0486\). Because \(R>1\), the observed sample has slightly greater likelihood under \(H_1\). As a check, \(1.048576(0.015625)=0.016384\), the alternative likelihood calculated above.
The binomial likelihoods would each include \(\binom{6}{4}=15\); since it appears in both numerator and denominator, it would leave this ratio unchanged.
For Bernoulli data, the likelihood ratio can be expressed directly in terms of the observed success count: $$ R=\frac{p_1^x(1-p_1)^{n-x}}{p_0^x(1-p_0)^{n-x}} =\left(\frac{p_1}{p_0}\right)^x \left(\frac{1-p_1}{1-p_0}\right)^{n-x}, $$ where \(p_0\) and \(p_1\) are the success probabilities specified by \(H_0\) and \(H_1\), respectively. This expression shows why both the success count and the failure count matter; using only the successes omits factors from the observed failures.
Worked Example: A Bernoulli Ratio with a Different Success Count
A quality audit records eight independent Bernoulli outcomes, of which six meet a stated standard. Compare \(H_0:p=0.25\) with \(H_1:p=0.5\). Calculate the likelihood ratio and interpret it.
Method. Substitute the six successes and two failures in the product likelihood. This method avoids treating the count as though it were the only contribution and forgetting the failure probabilities.
The likelihoods for a particular ordered sequence are $$ L_0=(0.25)^6(0.75)^2 =\left(\frac{1}{4}\right)^6\left(\frac{3}{4}\right)^2 =\frac{9}{65536}, $$ and $$ L_1=(0.5)^6(0.5)^2=(0.5)^8=\frac{1}{256} =\frac{256}{65536}. $$ Thus, $$ R=\frac{L_1}{L_0} =\frac{256/65536}{9/65536} =\frac{256}{9} =28.4444\ldots. $$ To four decimal places, \(R=28.4444\). The observed outcomes have about 28.4444 times the likelihood under \(H_1\) as under \(H_0\). A check is that \((256/9)(9/65536)=1/256=L_1\).
Computing the Ratio from Normal Samples
Let \(X_1,\ldots,X_n\) be independent observations from a normal distribution. Under each simple hypothesis, both the mean and variance needed for the density must be specified. If the model is \(N(\mu,\sigma^2)\), the likelihood of observed values \(x_1,\ldots,x_n\) is $$ L(\mu,\sigma^2)=(2\pi\sigma^2)^{-n/2} \exp\left(-\frac{1}{2\sigma^2}\sum_{j=1}^{n}(x_j-\mu)^2\right). $$ The notation \(N(\mu,\sigma^2)\) uses \(\sigma^2\) as the variance. For two hypotheses with the same variance, the leading factor and the variance denominator are common, so the comparison depends on the two residual sums of squares. With different variances, the leading factors do not cancel.
Worked Example: Comparing Two Normal Means
Three independent measurements are \(1,2,4\). Under \(H_0\), each observation has distribution \(N(2,1)\); under \(H_1\), each has distribution \(N(3,1)\). Calculate the likelihood ratio \(L_1/L_0\).
Method. Use the normal sample likelihood with the common variance retained in the formula, then compare the residual sums of squares. This avoids using the sample mean alone without accounting for the specified model.
Under \(H_0\), the sum of squared deviations is $$ (1-2)^2+(2-2)^2+(4-2)^2=1+0+4=5. $$ Under \(H_1\), it is $$ (1-3)^2+(2-3)^2+(4-3)^2=4+1+1=6. $$ Both likelihoods have the same leading factor \((2\pi)^{-3/2}\). Therefore, $$ R=\frac{L_1}{L_0} =\frac{(2\pi)^{-3/2}\exp(-6/2)} {(2\pi)^{-3/2}\exp(-5/2)} =\exp\left(-\frac{1}{2}\right) =0.606530\ldots. $$ To four decimal places, \(R=0.6065\). Since the ratio is below one, \(H_0\) assigns the greater likelihood to these observations. As a check, the alternative residual sum of squares is larger, so with equal variances its likelihood is smaller.
Worked Example: Comparing Normal Variances
Two independent observations are \(0\) and \(1\). Under \(H_0\), they come from \(N(0,1)\); under \(H_1\), they come from \(N(0,4)\). Calculate \(L_1/L_0\), rounding to four decimal places.
Method. Apply the normal likelihood under each specified variance. This example avoids the mistake of canceling the normalizing factor when the variances differ.
The sum of squared deviations from the common mean zero is \(0^2+1^2=1\). Under \(H_0\), $$ L_0=(2\pi)^{-1}\exp\left(-\frac{1}{2}\right). $$ Under \(H_1\), the variance is 4 and the standard deviation is 2, so $$ L_1=(2\pi\cdot4)^{-1} \exp\left(-\frac{1}{2\cdot4}\right) =(8\pi)^{-1}\exp\left(-\frac{1}{8}\right). $$ The ratio is $$ R=\frac{L_1}{L_0} =\frac{1}{4}\exp\left(\frac{1}{2}-\frac{1}{8}\right) =\frac{1}{4}\exp(0.375) =0.363747\ldots. $$ To four decimal places, \(R=0.3637\). Since \(R<1\), the null model has the larger likelihood for these data. The factor \(1/4\) comes from the different normalizing constants; omitting it would give an incorrect comparison.
Why Ratios and Log Ratios Work
The ratio compares the two likelihood values directly. Dividing by the null likelihood expresses the alternative likelihood relative to the null, which explains the interpretation of values above and below one. In many models, the likelihoods contain factors common to both hypotheses. If \(C(\mathbf{x})\) is such a positive factor, then $$ \frac{C(\mathbf{x})A_1(\mathbf{x})}{C(\mathbf{x})A_0(\mathbf{x})} =\frac{A_1(\mathbf{x})}{A_0(\mathbf{x})}. $$ That is why the binomial coefficient cancels for a success count and why common normal density factors cancel when the variances are equal. A factor that differs between the two hypotheses cannot be canceled.
Taking logarithms often makes products easier to calculate. Since the natural logarithm is strictly increasing, taking logs preserves the ordering of positive likelihood ratios: $$ \log R=\log L_1-\log L_0,\qquad R>1\ \Longleftrightarrow\ \log R>0. $$ For independent observations, a product likelihood becomes a sum of log likelihood contributions. This is a computational simplification, not a change in the direction or meaning of the comparison.
Common Direction and Calculation Traps
A frequent error is to calculate \(L_0/L_1\) but interpret the result as though it were \(L_1/L_0\). For the six-observation example above, the correct convention gives \(R=1.048576\), so the alternative likelihood is larger. The inverse ratio is \(1/1.048576=0.953674\ldots\); it is below one because it is the null likelihood divided by the alternative likelihood. Both calculations are valid ratios, but their interpretations must match their directions.
Another common error is to omit failures in a Bernoulli likelihood or to cancel normalizing constants when variances differ. Before simplifying, write both likelihoods from the specified model. Then verify that every canceled factor is identical in numerator and denominator, and check that the final interpretation agrees with whether the ratio is above or below one.
Check Your Understanding
For each problem, use \(R=L_1/L_0\). Interpret the result as a comparison of the two specified likelihoods, not as a probability that a hypothesis is true.
- Five independent Bernoulli observations contain three successes. Compare \(H_0:p=0.4\) with \(H_1:p=0.6\). Calculate \(R\) and state which model assigns the greater likelihood. (Answer: \(R=1.5000\); \(H_1\) assigns the greater likelihood.)
- Seven independent Bernoulli observations contain two successes. Compare \(H_0:p=0.2\) with \(H_1:p=0.5\). Include both successes and failures in your calculation. (Answer: \(R=0.5960\); \(H_0\) assigns the greater likelihood.)
- Two independent normal observations are \(1\) and \(3\). Compare \(H_0:N(2,1)\) with \(H_1:N(1,1)\), where the second parameter is the variance. Calculate \(R\) to four decimal places. (Answer: \(R=e^{-1}=0.3679\); \(H_0\) assigns the greater likelihood.)
- Four independent observations are \(0,1,1,2\). Compare \(H_0:N(0,1)\) with \(H_1:N(0,2)\), where the second parameter is the variance. Calculate \(R\) to four decimal places and account for the different normalizing factors. (Answer: \(R=\frac14 e^{1.5}=1.1204\); \(H_1\) assigns the greater likelihood.)
- Ten independent Bernoulli observations contain seven successes. Compare \(H_0:p=0.7\) with \(H_1:p=0.5\). Calculate \(R\) to four decimal places and explain why the result is below or above one. (Answer: \(R=\frac{0.5^{10}}{0.7^7(0.3)^3}=0.4392\); \(H_0\) assigns the greater likelihood.)