Likelihood-Ratio Order and Monotone Density Ratios

Why Compare Density Ratios?

Two distributions can differ in more than their means or their tail probabilities. Likelihood-ratio order captures a particular kind of systematic change: as the observation increases, the relative weight assigned by one distribution compared with another does not reverse direction. This comparison is stronger and more structured than simply saying that one variable tends to be larger.

The basic object is the ratio $f_Y(x)/f_X(x)$, or its discrete counterpart. Its behavior must be checked across the full relevant support, not just on a convenient subinterval. When supports differ or a denominator can be zero, an equivalent cross-product condition gives a precise definition without dividing by zero.

Definition for Densities and Mass Functions

Definition: Let $X$ and $Y$ have densities $f_X$ and $f_Y$ with respect to the same measure. We say $X$ is smaller than $Y$ in likelihood-ratio order, written $X\le_{\mathrm{lr}}Y$, if versions of the densities can be chosen such that, for every $x_1<x_2$, $$ f_Y(x_2)f_X(x_1)\ \ge\ f_Y(x_1)f_X(x_2). $$ For probability mass functions, the same condition is imposed at every ordered pair of support points $x_1<x_2$.

The cross-product definition is useful because it remains meaningful when one of the densities or masses is zero. On a common support $I$ where both densities are positive, it is equivalent to saying that the ratio $f_Y(x)/f_X(x)$ is nondecreasing on all of $I$. In particular, if the densities are positive on the same support $I$ and vanish outside $I$, the ratio must be checked throughout that entire support.

To see the equivalence on a common positive support, divide the cross-product inequality by the positive quantity $f_X(x_1)f_X(x_2)$. The result is $$ \frac{f_Y(x_1)}{f_X(x_1)} \ \le\ \frac{f_Y(x_2)}{f_X(x_2)} \qquad (x_1<x_2). $$ Conversely, multiplying this ratio inequality by $f_X(x_1)f_X(x_2)$ gives the cross-product condition. For supports that differ, use the cross-product definition: it specifies the required comparison without silently ignoring points where a ratio is undefined.

Densities are defined only up to sets of measure zero, so the definition is stated in terms of versions satisfying the inequality. In ordinary examples with continuous density formulas, the displayed formulas themselves provide the versions to check. The same convention applies to probability mass functions, which are specified at individual points.

Worked Example: A Uniform Law Compared With an Increasing Density

Let $X\sim\operatorname{Uniform}(0,1)$, and let $Y$ have density $f_Y(x)=2x$ for $0<x<1$, with both densities zero elsewhere. First check that $f_Y$ is a probability density: $$ \int_0^1 2x\,dx=[x^2]_0^1=1. $$

On their full common support $(0,1)$, $f_X(x)=1$, so $$ \frac{f_Y(x)}{f_X(x)}=2x. $$ This ratio is increasing: if $0<x_1<x_2<1$, then $2x_1<2x_2$. Equivalently, the cross-products are $2x_2$ and $2x_1$, and $2x_2\ge2x_1$. Outside the support, both densities vanish, so the cross-product inequality also holds for pairs involving only exterior points; for a pair with one point inside and one outside, the inequality follows directly because both densities are zero at the exterior point. Thus $X\le_{\mathrm{lr}}Y$.

Worked Example: Bernoulli Mass-Ratio Order

Let $X\sim\operatorname{Bernoulli}(0.3)$ and $Y\sim\operatorname{Bernoulli}(0.7)$. Both have support $\{0,1\}$. Their mass ratios are $$ \frac{P(Y=0)}{P(X=0)}=\frac{0.3}{0.7}=\frac{3}{7}, \qquad \frac{P(Y=1)}{P(X=1)}=\frac{0.7}{0.3}=\frac{7}{3}. $$

Since $3/7<7/3$, the mass ratio is nondecreasing across the ordered support points, and $X\le_{\mathrm{lr}}Y$. Checking with cross-products gives the same result: $$ P(Y=1)P(X=0)=0.7(0.7)=0.49 \ \ge\ P(Y=0)P(X=1)=0.3(0.3)=0.09. $$ There is only one pair of distinct ordered support points to check.

Worked Example: Normal Distributions With a Common Variance

Let $X\sim N(0,1)$ and $Y\sim N(1,1)$. Their densities are positive on the whole real line, so the full common support is $\mathbb{R}$. The density ratio is $$ \frac{f_Y(x)}{f_X(x)} = \frac{\exp\{-(x-1)^2/2\}}{\exp\{-x^2/2\}} = \exp\left\{-\frac{(x-1)^2}{2}+\frac{x^2}{2}\right\} = \exp(x-\tfrac12). $$

The exponent $x-\tfrac12$ is increasing in $x$, and the exponential function is increasing. Hence the ratio is increasing everywhere on $\mathbb{R}$, so $X\le_{\mathrm{lr}}Y$. The comparison is global: the calculation verifies the ratio over the full support, rather than merely over a selected interval.

Monotone Ratios and Monotone Likelihood Ratios

Likelihood-ratio order for a pair of random variables is the pairwise version of a broader idea used for families of distributions. In statistical models, a likelihood ratio compares the probability of the observed data under two parameter values. When that ratio is monotone in a statistic, the model is said to have a monotone likelihood ratio in that statistic.

Definition: Let $f_\theta(x)$ be densities with respect to a common measure, indexed by an ordered parameter $\theta$, and let $T(x)$ be a real-valued statistic. The family has a monotone likelihood ratio in $T$ if, for every $\theta_2>\theta_1$, the ratio $f_{\theta_2}(x)/f_{\theta_1}(x)$ is nondecreasing as a function of $T(x)$ wherever the ratio is defined. When the family has a common positive support, this means that $T(x_1)\le T(x_2)$ implies $$ \frac{f_{\theta_2}(x_1)}{f_{\theta_1}(x_1)} \ \le\ \frac{f_{\theta_2}(x_2)}{f_{\theta_1}(x_2)}. $$

If the observation itself is the statistic, $T(x)=x$, then the condition for each pair $\theta_1<\theta_2$ says exactly that the distribution at $\theta_1$ is below the distribution at $\theta_2$ in likelihood-ratio order, provided the densities have the common positive support needed for the ratio formulation. For a general statistic $T$, the ratio need only be ordered according to $T(x)$; it need not be monotone in every coordinate of the original observation.

A Useful Exponential-Family Criterion

A common way to verify monotone likelihood ratios is through a one-parameter exponential-family representation. The following derivation states the conditions that make the conclusion valid.

Theorem: Suppose a family has densities $$ f_\theta(x)=h(x)\exp\{\eta(\theta)T(x)-A(\theta)\} $$ with respect to a common measure, on a common support where $h(x)>0$, and suppose $\eta(\theta)$ is increasing in $\theta$. Then the family has a monotone likelihood ratio in $T$.

Proof. Take any $\theta_2>\theta_1$. On the common support, the factors $h(x)$ cancel: $$ \frac{f_{\theta_2}(x)}{f_{\theta_1}(x)} = \exp\left\{ [\eta(\theta_2)-\eta(\theta_1)]T(x) -[A(\theta_2)-A(\theta_1)] \right\}. $$ Because $\eta$ is increasing, $\eta(\theta_2)-\eta(\theta_1)\ge0$. The term involving $A$ is constant in $x$. Thus the exponent is nondecreasing as a function of $T(x)$, and applying the increasing exponential function preserves that ordering. The likelihood ratio is therefore nondecreasing in $T(x)$. $\square$

The common-support condition matters: this ratio calculation compares the two densities at the same points and cancels the same factor $h(x)$. If a model has parameter-dependent support, this displayed argument alone does not establish the property; the support and zero-density cases must be handled using a suitable full definition.

Worked Example: Bernoulli Family Has a Monotone Likelihood Ratio

For $X\sim\operatorname{Bernoulli}(p)$, the mass function is $f_p(x)=p^x(1-p)^{1-x}$ on $\{0,1\}$. For $0<p_1<p_2<1$, the ratio at $x\in\{0,1\}$ is $$ \frac{f_{p_2}(x)}{f_{p_1}(x)} = \left(\frac{p_2}{p_1}\right)^x \left(\frac{1-p_2}{1-p_1}\right)^{1-x}. $$ At $x=0$ this equals $(1-p_2)/(1-p_1)$, and at $x=1$ it equals $p_2/p_1$. Since $p_2>p_1$, $$ \frac{1-p_2}{1-p_1}<1<\frac{p_2}{p_1}. $$ Thus the ratio increases from $x=0$ to $x=1$, so the Bernoulli family has a monotone likelihood ratio in $T(x)=x$.

The same result follows from the exponential-family criterion: take $h(x)=1$ on $\{0,1\}$, $T(x)=x$, $\eta(p)=\log\{p/(1-p)\}$, and $A(p)=-\log(1-p)$. The function $\eta(p)$ is increasing for $0<p<1$, since its derivative is $1/[p(1-p)]>0$.

Why Full-Support Checks Matter

A nondecreasing ratio expresses a consistent change in relative density: as $x$ rises, $Y$'s density becomes no smaller relative to $X$'s. If the ratio rises on one interval and falls on another, the distributions do not satisfy likelihood-ratio order merely because a selected piece of the support looks ordered. This is why specifying the full support, or using the cross-product condition globally, is part of the definition rather than a technical afterthought.

For example, let $X\sim\operatorname{Uniform}(0,2)$, and let $Y$ have density $0.75$ on $(0,1)$ and $0.25$ on $(1,2)$, zero elsewhere. The density integrates to $0.75+0.25=1$, and $f_X=0.5$ on $(0,2)$. The ratio $f_Y/f_X$ is $1.5$ on $(0,1)$, then $0.5$ on $(1,2)$. It drops across the support, so it is not nondecreasing and $X\le_{\mathrm{lr}}Y$ does not hold. Checking only the first interval would miss the failure.

Common mistake: Verify that $f_Y/f_X$ is increasing on some interval where both densities are positive, then conclude likelihood-ratio order. Correction: Check the ratio over the entire common positive support and account for points outside it through the cross-product definition.
Common mistake: Divide by $f_X(x)$ at points where $f_X(x)=0$. Correction: Use cross-products at such points; do not treat division by zero as an ordinary ratio.
Common mistake: Treat any likelihood ratio as a monotone likelihood ratio. Correction: The defining feature is monotonicity in the observation or specified statistic, for every ordered parameter pair in the family.

Key takeaway: Likelihood-ratio order is a global monotonicity condition on relative density or mass. On a common positive support it is checked by the ratio; in general, the cross-product inequality states the comparison without relying on undefined ratios.

Check Your Understanding

For each question, identify the support and the precise ratio or cross-product comparison that is relevant.

For two densities positive on the same interval and zero outside it, what ratio condition is equivalent to likelihood-ratio order?
For probability mass functions on $\{0,1,2\}$, which ordered pairs need to satisfy the cross-product condition?
Why is checking a density ratio on only part of the common support insufficient?
In an exponential family, which condition on $\eta(\theta)$ makes the likelihood ratio nondecreasing in $T$?
For the Bernoulli family, compare the likelihood ratio at $x=0$ and $x=1$ when $p_2>p_1$.