Reparameterization and Data-Only Factors in Likelihood Ratios

Why Simplifying a Likelihood Ratio Needs Care

For two simple hypotheses, the Neyman–Pearson test compares the likelihood of the observed data under one hypothesis with its likelihood under the other. The formulas can look much more complicated than the decision rule: a product of densities may contain factors that depend on the data but are exactly the same under both hypotheses. Those common factors can be removed from the comparison, leaving a simpler statistic that gives the same decisions.

A related simplification comes from changing how the same information is expressed. A likelihood ratio can be written as its logarithm, or a test based on a sample sum can be written using the sample mean. These changes preserve decisions only when the cutoff is translated correctly. In particular, a decreasing transformation reverses the direction of an inequality.

Definition: Suppose the data $Z$ have probability mass functions or densities $f_0(z)$ and $f_1(z)$ under two simple hypotheses $H_0$ and $H_1$. The likelihood ratio in favor of $H_1$ over $H_0$ is $$ \Lambda(z)=\frac{f_1(z)}{f_0(z)}. $$ A Neyman–Pearson test rejects $H_0$ when $\Lambda(z)>c$, accepts $H_0$ when $\Lambda(z)<c$, and may randomize at equality to attain a specified size. The constant $c$ is chosen for the desired test size. This tutorial uses the $f_1/f_0$ convention; reversing the ratio requires reversing the inequality.

The probability mass functions or densities must be specified with respect to the same measure. For discrete data, use the corresponding probability mass functions; for continuous data, use densities. On points where both likelihoods are positive, a factor common to numerator and denominator cancels. If a point is possible under one hypothesis but impossible under the other, handle it separately: the ratio is infinite when $f_0(z)=0<f_1(z)$, and zero when $f_1(z)=0<f_0(z)$. A formal cancellation of a shared zero would give an undefined $0/0$, not a likelihood ratio.

Common Data-Only Factors

Suppose the likelihoods can be expressed as $$ f_1(z)=h(z)g_1(z), \qquad f_0(z)=h(z)g_0(z), $$ where $h(z)>0$ is the same under both hypotheses on the common support. Then $$ \Lambda(z)=\frac{h(z)g_1(z)}{h(z)g_0(z)} =\frac{g_1(z)}{g_0(z)}. $$ The factor $h(z)$ may depend on the observed data; it need not be constant across possible samples. It cancels because it has the same value for the particular observed data under both hypotheses.

This is a comparison of two likelihoods for the same observed data, not a claim that the common factor is irrelevant to every statistical calculation. It may matter for other quantities. Also, if the supports differ between hypotheses, do not discard support information simply because the density formulas appear to share an algebraic factor.

Worked Example: Canceling a Binomial Coefficient

Let $X$ be the number of successes in six independent Bernoulli trials. Test $H_0:p=0.25$ against $H_1:p=0.50$. The observed count is $x=4$. Find the likelihood ratio and determine whether this observation falls in the rejection region $\Lambda(x)>1$.

Method. Use the binomial probability mass functions and cancel the factor that is common to both hypotheses: $$ f_p(x)=\binom{6}{x}p^x(1-p)^{6-x}. $$ For $x=4$, the numerator under $H_1$ is $$ f_1(4)=\binom{6}{4}(0.50)^4(0.50)^2 =15(0.0625)(0.25)=0.234375. $$ The denominator under $H_0$ is $$ f_0(4)=\binom{6}{4}(0.25)^4(0.75)^2 =15(0.00390625)(0.5625)=0.032958984375. $$ Thus, canceling the common $\binom{6}{4}=15$ gives $$ \Lambda(4)=\frac{(0.50)^4(0.50)^2}{(0.25)^4(0.75)^2} =2^4\left(\frac{2}{3}\right)^2 =\frac{64}{9} \approx 7.1111. $$ Since $7.1111>1$, this observation is in the stated rejection region.

Check. Dividing the uncanceled likelihoods gives $0.234375/0.032958984375=7.1111$, agreeing with the simplified calculation. The common mistake is to cancel $\binom{6}{4}$ from just one side, or to omit it in only one likelihood. It must either appear in both likelihoods or be canceled from both.

One-to-One Transformations and the Rejection Rule

Suppose a test rejects when a statistic $T(z)$ exceeds a threshold. If $U=g(T)$ and $g$ is strictly increasing, then $T>k$ is equivalent to $U>g(k)$. The ordering of observations has not changed. If $g$ is strictly decreasing, then $T>k$ is equivalent to $U<g(k)$, so the inequality reverses. A one-to-one transformation preserves the decision when the rejection region is transformed along with the statistic; it does not guarantee that the same numerical cutoff or inequality direction still applies.

The logarithm is strictly increasing on positive numbers. Therefore, for a positive likelihood ratio, comparing $\Lambda(z)$ with $c>0$ is equivalent to comparing $\log\Lambda(z)$ with $\log c$. This is often easier because products become sums. The test decisions, including equality at the boundary, remain the same.

Theorem: If $g$ is one-to-one on the values of a statistic $T$, the rule $T\in R$ is equivalent to $g(T)\in g(R)$. If $g$ is strictly increasing, upper-tail and lower-tail inequalities keep their direction. If $g$ is strictly decreasing, their direction reverses. The cutoff must be transformed along with the statistic.

Worked Example: A Normal Likelihood Ratio and the Log Scale

Suppose $X_1,X_2,X_3$ are independent normal observations with known variance $4$. Test $H_0:\mu=1$ against $H_1:\mu=3$. The observed values are $2,3,1$. Express the likelihood ratio using the sample sum, and check whether the data satisfy the test rule $\Lambda>2$.

Method. Under a normal model with mean $\mu$ and known variance $\sigma^2$, the joint density is proportional to $$ \exp\left(-\frac{1}{2\sigma^2}\sum_{i=1}^n(x_i-\mu)^2\right). $$ The factor $(2\pi\sigma^2)^{-n/2}$ is common to both hypotheses and cancels. Expanding the squares in the ratio gives $$ \log\Lambda =\frac{\mu_1-\mu_0}{\sigma^2}\sum_{i=1}^n x_i -\frac{n(\mu_1^2-\mu_0^2)}{2\sigma^2}. $$ Here, $\mu_0=1$, $\mu_1=3$, $\sigma^2=4$, and $n=3$. Therefore, $$ \log\Lambda =\frac{2}{4}\sum_{i=1}^3x_i -\frac{3(9-1)}{2(4)} =0.5\sum_{i=1}^3x_i-3. $$ The observed sum is $2+3+1=6$, so $$ \log\Lambda=0.5(6)-3=0, \qquad \Lambda=e^0=1. $$ Since $1$ is not greater than $2$, this observation does not satisfy the rejection rule.

To translate the cutoff, $\Lambda>2$ is equivalent, by the increasing logarithm, to $\log\Lambda>\log 2$. Thus $$ 0.5\sum_{i=1}^3x_i-3>\log 2 \quad\Longleftrightarrow\quad \sum_{i=1}^3x_i>6+2\log 2 \approx 7.3863. $$ The sample mean is the sum divided by three, a strictly increasing transformation, so the same rule is $\bar X>2.4621$. The observed mean is $6/3=2$, again giving no rejection. The check confirms that the sum, mean, likelihood-ratio, and log-likelihood-ratio versions agree when their cutoffs are translated.

Common mistake. Comparing $\log\Lambda$ directly with $2$ would change the test. The correct log-scale cutoff is $\log 2$, not $2$.

Reparameterizing the Model

A parameterization names the model distributions using a particular parameter. For example, an exponential distribution can be described by its rate $\lambda$ or its mean $\eta=1/\lambda$. If this map is one-to-one over the parameter values in question, then the same two distributions can be labeled in either way. The likelihood ratio compares those distributions at the observed data, so changing their labels does not change the ratio.

There is an important distinction between reparameterizing a model and transforming the observed data. If the data are transformed, densities may acquire a Jacobian factor; for a common one-to-one transformation of the data, the same Jacobian appears in both densities and cancels in their ratio. Merely replacing a parameter label, such as $\lambda$ with $\eta=1/\lambda$, is not a transformation of the observations and does not call for a data-density Jacobian. A prior distribution on parameters is not part of the simple-hypothesis likelihood ratio considered here.

Worked Example: The Same Exponential Test in Two Parameterizations

Let $X_1,X_2,X_3$ be independent exponential observations with rate $\lambda$, so each density is $f(x;\lambda)=\lambda e^{-\lambda x}$ for $x\geq 0$. Test $H_0:\lambda=1$ against $H_1:\lambda=2$. The observed values are $0.2,0.4,0.6$, and the rejection rule is $\Lambda>2$. Re-express the hypotheses using the mean $\eta=1/\lambda$ and verify the decision.

Method. First calculate the observed sum: $$ T=0.2+0.4+0.6=1.2. $$ For $n=3$, the likelihood under rate $\lambda$ is $\lambda^3e^{-\lambda T}$. Hence $$ \Lambda =\frac{2^3e^{-2(1.2)}}{1^3e^{-1(1.2)}} =8e^{-1.2} \approx 2.4096. $$ Since $2.4096>2$, reject $H_0$ under the stated rule.

Now write $\eta=1/\lambda$. The same model has density $f(x;\eta)=\eta^{-1}e^{-x/\eta}$. The original $H_0$ is $\eta=1$, and the original $H_1$ is $\eta=0.5$. Using those values gives $$ \Lambda =\frac{(0.5)^{-3}e^{-1.2/0.5}}{1^{-3}e^{-1.2/1}} =8e^{-2.4+1.2} =8e^{-1.2} \approx 2.4096. $$ The likelihood ratio and decision are unchanged. The check is that $\eta=1/\lambda$ maps rate $1$ to mean $1$, and rate $2$ to mean $0.5$; it has not changed either hypothesis distribution. The common mistake is to compare rate $2$ with mean $1$ as if these were the two original hypotheses. That would compare different distributions.

When a Transformation Reverses the Inequality

The ratio itself can also be replaced by a one-to-one function. For instance, $R=1/\Lambda$ is a strictly decreasing transformation when $\Lambda>0$. Thus $$ \Lambda>c \quad\Longleftrightarrow\quad \frac{1}{\Lambda}<\frac{1}{c}. $$ The ratio $1/\Lambda$ is the likelihood ratio in favor of $H_0$ over $H_1$. It gives the same rejection decisions only when the inequality is reversed. More generally, a one-to-one but nonmonotone function can represent the statistic without losing information, but it may not turn a threshold rule into a single upper- or lower-tail threshold.

Write the two likelihoods.
Use the same data representation and the correct model under each simple hypothesis.

Check support and factors.
Identify points where the hypotheses assign zero probability or density. Cancel only a common positive factor on the common support.

Simplify the ratio.
Reduce the expression to a function of the observed data, a statistic, or its logarithm.

Translate the cutoff.
Apply the same transformation to the threshold. Preserve the inequality for an increasing map and reverse it for a decreasing map.

Check the decision.
Substitute the observed data and verify that the transformed and original rules agree.

Common Exam Traps and Why the Simplification Matters

Canceling data-only factors makes a test easier to compute and often reveals which feature of the sample determines the decision. For example, a binomial likelihood ratio can depend on the success count rather than the full mass-function expression, while a normal likelihood ratio can depend on the sample sum. Rewriting the rule in terms of a convenient statistic does not change the test if the translation is one-to-one and the cutoff is handled correctly.

Canceling a factor that is not common. A term depending on the parameter cannot be canceled just because it has a similar appearance in both likelihoods. Compare the full terms under each hypothesis.
Ignoring unequal supports. If one hypothesis assigns zero density where the other assigns positive density, the ratio is zero or infinite there. That information is part of the test.
Using the wrong ratio convention. If the ratio is $f_1/f_0$, large values favor $H_1$. If it is $f_0/f_1$, the direction is reversed.
Keeping the old cutoff after transforming. A test $\Lambda>c$ becomes $\log\Lambda>\log c$, not $\log\Lambda>c$.
Failing to reverse a decreasing transformation. Replacing $\Lambda$ by $1/\Lambda$ turns an upper-tail rule into a lower-tail rule.
Confusing a parameter relabeling with new hypotheses. A one-to-one reparameterization changes the labels, not the distributions being tested. The transformed parameter values must represent the original hypotheses.

Key takeaway: A positive factor shared by both likelihoods cancels on their common support. Equivalent parameterizations leave the likelihood ratio unchanged, while one-to-one transformations preserve decisions only when the rejection region and cutoff are translated correctly. For a decreasing transformation, reverse the inequality.

Check Your Understanding

For each problem, simplify the likelihood ratio or translate the rejection rule as requested.

A sample of four Bernoulli trials has $x=3$ successes. Test $p=0.20$ against $p=0.50$. Compute $f_{0.50}(3)/f_{0.20}(3)$, canceling the common binomial coefficient. (Answer: $9.7656$, to four decimal places.)
Two independent observations are normal with known variance $1$. Test $\mu=0$ against $\mu=1$, and suppose the observed sum is $1.5$. Find $\log\Lambda$ and $\Lambda$. (Answer: $\log\Lambda=0.5$, $\Lambda\approx1.6487$.)
Two independent exponential observations have rate $\lambda$. Test $\lambda=3$ against $\lambda=1$, with observed sum $T=0.4$. Find the likelihood ratio $f_1/f_0$. (Answer: $\frac{1}{9}e^{0.8}\approx0.2473$.)
A test rejects when $\Lambda>5$, and $R=1/\Lambda$. Give the equivalent rejection rule in terms of $R$. (Answer: reject when $R<0.2$.)
Two independent exponential observations have rate $\lambda$. Test $\lambda=1$ against $\lambda=4$, and observe $T=0.5$. Compute $f_4/f_1$. State the equivalent parameter values if the model is instead described by its mean $\eta=1/\lambda$. (Answer: $16e^{-1.5}\approx3.5701$; the hypotheses are $\eta=1$ and $\eta=0.25$, respectively.)

What You'll Learn