Monotone Likelihood Ratios in One-Parameter Families

Introduction

A likelihood ratio compares how well two parameter values account for an observation. In some families, this comparison changes in only one direction as the observed value increases: larger observations consistently favor the larger parameter. This structure is called a monotone likelihood ratio (MLR), and it gives a useful way to understand how evidence in a sample is ordered across parameter values.

One-parameter exponential families make the calculation especially transparent. When their natural parameter increases and their sufficient statistic is increasing in the observation, the likelihood ratio has a simple exponential form. The key is to state the common-support and parameter-ordering conditions carefully; the formula alone does not settle the direction of the comparison.

Definition and Common-Support Conditions

Definition: Let \(f_{\eta}\) be densities or mass functions indexed by a real parameter \(\eta\), all with respect to the same dominating measure and on a common support. The family has a monotone likelihood ratio in a statistic \(T(x)\) if, for every \(\eta_2>\eta_1\), the ratio \(f_{\eta_2}(x)/f_{\eta_1}(x)\), wherever defined on the common support, is a nondecreasing function of \(T(x)\). If \(T(x)\) is nondecreasing in \(x\), the ratio is also nondecreasing in \(x\).

A convenient common-support setup is a fixed measurable set \(S\) on which the distributions have densities with respect to a measure \(\nu\), and where the densities are positive. Points at which every density is zero can be omitted. Having a common support matters: if one density is positive where the other is zero, the ratio may be undefined or infinite there, and the usual ratio calculation requires additional care.

Consider the one-parameter exponential-family form

f_{\eta}(x)=h(x)\exp\{\eta T(x)-A(\eta)\}, \qquad x\in S.

Here \(h(x)\) is a nonnegative function that does not depend on \(\eta\), \(T(x)\) is a statistic, and \(A(\eta)\) is the log-normalizing function. The natural parameter \(\eta\) ranges over an interval \(\mathcal{H}\), and the normalizing integral is finite for every \(\eta\in\mathcal{H}\). The same support and dominating measure are used throughout the family. To obtain the MLR conclusion in the observation, we additionally require that \(T(x)\) is nondecreasing on \(S\).

The Likelihood-Ratio Formula

For two parameter values \(\eta_2>\eta_1\), the factor \(h(x)\) cancels in the ratio. The normalizing terms remain, but they do not depend on \(x\). Thus

\frac{f_{\eta_2}(x)}{f_{\eta_1}(x)} = \exp\{(\eta_2-\eta_1)T(x)-[A(\eta_2)-A(\eta_1)]\}.

Because \(\eta_2-\eta_1>0\), the exponent increases with \(T(x)\). This proves that the ratio is nondecreasing in \(T(x)\), and hence in \(x\) whenever \(T\) is nondecreasing in \(x\).

Theorem: Suppose \(f_{\eta}(x)=h(x)\exp\{\eta T(x)-A(\eta)\}\) is a normalized family on a common support, with \(h\) independent of \(\eta\), and let its natural parameter range over an interval. If \(T(x)\) is nondecreasing in \(x\), then the family has a monotone likelihood ratio in \(T(x)\). For \(\eta_2>\eta_1\), the likelihood ratio is nondecreasing in \(x\).

Proof. Fix any \(\eta_2>\eta_1\). On the common support where \(h(x)>0\), division gives

\frac{f_{\eta_2}(x)}{f_{\eta_1}(x)} = \exp\{(\eta_2-\eta_1)T(x)-A(\eta_2)+A(\eta_1)\}.

Take \(x_1<x_2\) in the ordered support. Since \(T\) is nondecreasing, \(T(x_1)\le T(x_2)\). Therefore

\frac{f_{\eta_2}(x_2)/f_{\eta_1}(x_2)} {f_{\eta_2}(x_1)/f_{\eta_1}(x_1)} = \exp\{(\eta_2-\eta_1)[T(x_2)-T(x_1)]\} \ge 1.

The ratio is consequently nondecreasing in \(x\), and it is a nondecreasing function of \(T(x)\). If \(T\) is strictly increasing, the ratio is strictly increasing whenever \(x_1<x_2\). If \(T\) takes the same value at two points, the ratio is equal at those points. \(\square\)

The same calculation gives a useful equivalent inequality. For \(x_1<x_2\) and \(\eta_1<\eta_2\),

f_{\eta_2}(x_2)f_{\eta_1}(x_1) \ge f_{\eta_2}(x_1)f_{\eta_1}(x_2).

Indeed, on the positive common support, this is just the nondecreasing-ratio inequality after multiplying by the positive quantity \(f_{\eta_1}(x_1)f_{\eta_1}(x_2)\). It expresses the same ordering without writing a quotient.

Worked Examples

Worked Example: Binomial Success Probabilities

Let \(X\sim\operatorname{Binomial}(5,p)\), with \(0<p<1\). Show that the family has an MLR in \(X\) as \(p\) increases, and compare \(p_1=0.3\) with \(p_2=0.6\).

The mass function is

P_p(X=x)=\binom{5}{x}p^x(1-p)^{5-x}, \qquad x=0,1,2,3,4,5.

Set \(\eta=\log\{p/(1-p)\}\), so \(p=e^\eta/(1+e^\eta)\). Rewriting the mass function gives

P_{\eta}(X=x) = \binom{5}{x}\exp\left\{\eta x-5\log(1+e^\eta)\right\}.

Thus \(T(x)=x\), \(h(x)=\binom{5}{x}\), and \(A(\eta)=5\log(1+e^\eta)\). Since \(T(x)=x\) is increasing, the theorem applies. For the specified probabilities, \(\eta_1=\log(3/7)\), \(\eta_2=\log(3/2)\), and \(\eta_2-\eta_1=\log(7/2)>0\). Also,

A(\eta_2)-A(\eta_1) = 5\log\left(\frac{1+3/2}{1+3/7}\right) = 5\log(7/4).

Therefore the ratio at \(x\) is

\frac{P_{0.6}(X=x)}{P_{0.3}(X=x)} = \exp\{x\log(7/2)-5\log(7/4)\} = (7/2)^x(4/7)^5.

Increasing \(x\) by one multiplies the ratio by \(7/2>1\). Each additional observed success therefore raises the likelihood ratio in favor of \(p=0.6\) over \(p=0.3\).

Worked Example: Poisson Means

Let \(X\sim\operatorname{Poisson}(\lambda)\), where \(\lambda>0\). Compare \(\lambda_1=2\) with \(\lambda_2=5\).

For \(x=0,1,2,\ldots\),

P_{\lambda}(X=x)=\frac{e^{-\lambda}\lambda^x}{x!} = \frac{1}{x!}\exp\{x\log\lambda-\lambda\}.

The natural parameter is \(\eta=\log\lambda\), so \(T(x)=x\), \(h(x)=1/x!\), and \(A(\eta)=e^\eta\). Both distributions have the same support, the nonnegative integers. The ratio is

\frac{P_{5}(X=x)}{P_{2}(X=x)} = \exp\{x(\log 5-\log 2)-(5-2)\} = e^{-3}(5/2)^x.

The factor for increasing \(x\) by one is \(5/2>1\), so the ratio increases with the count. This is the MLR property in \(X\), with the larger Poisson mean favored increasingly as the observed count grows.

Worked Example: Normal Means With Known Variance

Suppose \(X\sim N(\mu,1)\), where the variance is known to be one and \(\mu\) varies over the real line. Compare \(\mu_1=0\) with \(\mu_2=1\).

The density can be written as

f_{\mu}(x) = \frac{1}{\sqrt{2\pi}} \exp\{-x^2/2+\mu x-\mu^2/2\}.

This has \(T(x)=x\), \(h(x)=(2\pi)^{-1/2}\exp(-x^2/2)\), \(\eta=\mu\), and \(A(\mu)=\mu^2/2\). The common support is the real line. Directly,

\frac{f_{1}(x)}{f_{0}(x)} = \exp\{x-1/2\}.

If \(x\) increases by a positive amount \(d\), this ratio is multiplied by \(e^d>1\). Hence the family has an MLR in \(X\) as \(\mu\) increases. For example, at \(x=0\) the ratio is \(e^{-1/2}\), while at \(x=1\) it is \(e^{1/2}\), confirming the increase numerically.

What the Normalizing Function Adds

The MLR conclusion depends on the difference \(\eta_2-\eta_1\), not on the shape of \(A\): the normalizing terms affect the level of the ratio but not how it varies with the observation. Under standard differentiability conditions, however, \(A\) also describes how the distribution changes as \(\eta\) varies.

Result: Suppose \(Z(\eta)=\int h(x)e^{\eta T(x)}\,d\nu(x)\) is finite on an open interval, \(A(\eta)=\log Z(\eta)\), and differentiation twice under the integral is valid there. Then

A'(\eta)=E_{\eta}[T(X)], \qquad A''(\eta)=\operatorname{Var}_{\eta}(T(X)).

Proof. The stated differentiation condition gives

Z'(\eta)=\int T(x)h(x)e^{\eta T(x)}\,d\nu(x), \qquad Z''(\eta)=\int T(x)^2h(x)e^{\eta T(x)}\,d\nu(x).

Dividing the first expression by \(Z(\eta)\) gives the expectation under the density \(f_\eta\), so \(A'(\eta)=Z'(\eta)/Z(\eta)=E_\eta[T(X)]\). Differentiating the logarithm a second time yields

A''(\eta) = \frac{Z''(\eta)}{Z(\eta)} - \left(\frac{Z'(\eta)}{Z(\eta)}\right)^2 = E_\eta[T(X)^2]-\{E_\eta[T(X)]\}^2 = \operatorname{Var}_\eta(T(X)).

Since variance is nonnegative, the mean of \(T(X)\) is nondecreasing in the natural parameter wherever these derivatives exist. This is consistent with the MLR direction: increasing \(\eta\) gives increasing weight to larger values of \(T\). \(\square\)

Direction, Interpretation, and Common Pitfalls

The direction of the MLR statement depends on both the parameterization and the statistic. The theorem orders distributions by the natural parameter \(\eta\). If a model is indexed by another parameter \(\theta\), the same direction applies when \(\eta(\theta)\) increases with \(\theta\). If \(\eta(\theta)\) decreases instead, increasing \(\theta\) reverses the comparison. Likewise, if \(T(x)\) decreases in \(x\), the ratio decreases with \(x\) for \(\eta_2>\eta_1\).

A frequent mistake is to notice that the family is exponential and conclude that the ratio increases in the observation without checking \(T\). The formula gives increase in \(T(x)\), not automatically in \(x\). For instance, if the chosen statistic is \(T(x)=-x\), the ratio decreases with \(x\) as the natural parameter increases. Another mistake is to treat the cancellation of \(h\) as valid when supports differ; the ratio calculation above is justified on a common positive support.

The property is useful because it gives a consistent ordering of the evidence supplied by observations: larger values of an increasing statistic favor larger natural parameters relative to smaller ones. It does not say that every individual observation identifies the parameter, nor does it replace checking the model’s support and parameterization. The ratio formula makes those assumptions visible and lets the direction be verified directly.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

In a one-parameter exponential family, what happens to the likelihood ratio when \(T(x)\) increases and \(\eta_2>\eta_1\)?
For the binomial family, identify \(T(x)\), the natural parameter, and the common support.
For the Poisson comparison with means 2 and 5, by what factor does the likelihood ratio change when \(x\) increases by one?
What happens to the direction of the MLR in \(x\) if \(T(x)\) is decreasing rather than increasing?
Under the stated differentiability conditions, what do \(A'(\eta)\) and \(A''(\eta)\) represent?