Choosing an Order for a Distributional Comparison

Start With the Question Being Compared

Two distributions can be compared in different ways because “larger” can refer to different features. One model may produce larger values more often, have a lower failure rate at each time, have a monotone density ratio relative to another model, or be more dispersed at the same mean. No single ordering answers all four questions. The useful first step is to name the feature of interest, then choose an order whose definition directly concerns that feature.

The four orders considered here are related but not interchangeable. Likelihood-ratio order implies hazard-rate order, and hazard-rate order implies usual stochastic order under their standard conditions. Convex order addresses a different question: it compares expectations of convex functions and requires equal means. These implications help organize the choices, but the direction of comparison must still be checked using the definitions.

Four Orders, Four Questions

Definition: Write \(X\le_{\mathrm{st}}Y\) (usual stochastic order) if \(F_X(t)\ge F_Y(t)\) for every real \(t\), or equivalently \(P(X>t)\le P(Y>t)\) for every \(t\). This says that \(Y\) has at least as much probability in every upper tail as \(X\).

Usual stochastic order is the direct choice when the question concerns threshold exceedance probabilities, upper-tail risk, or whether one variable tends to take larger values in the sense of all increasing summaries. It does not say that every realized value of \(Y\) exceeds every realized value of \(X\).

Definition: For nonnegative lifetime variables \(X\) and \(Y\), write \(X\le_{\mathrm{hr}}Y\) (hazard-rate order) when the survival ratio \(S_X(t)/S_Y(t)\), where \(S_Z(t)=P(Z>t)\), is nonincreasing over the range where the ratio is defined. For absolutely continuous lifetimes with positive survival probabilities, the hazard rate is \(h_Z(t)=f_Z(t)/S_Z(t)\); the condition \(h_X(t)\ge h_Y(t)\) for all \(t\) is a standard way to establish this order.

Hazard-rate order is designed for questions about lifetime risk across time. It compares instantaneous failure rates conditional on survival up to time \(t\), rather than only comparing the accumulated probabilities of exceeding each threshold. In particular, this is a stronger kind of comparison than simply asking which lifetime has the larger survival probability at every fixed time.

Definition: Suppose \(X\) and \(Y\) have densities \(f_X\) and \(f_Y\) that are positive on the same interval. Write \(X\le_{\mathrm{lr}}Y\) (likelihood-ratio order) if \(f_Y(t)/f_X(t)\) is nondecreasing on that interval. Thus the relative density weight of \(Y\) compared with \(X\) rises as \(t\) rises. For distributions with different supports, additional support conditions are needed; a ratio on a shared interval alone may not settle the comparison.

Likelihood-ratio order is the targeted choice when the structure of the density ratio matters—for example, when an argument depends on how evidence in the observed value shifts relative plausibility between two models. Under the usual regularity conditions, likelihood-ratio order implies hazard-rate order, which in turn implies usual stochastic order. This hierarchy is useful, but a weaker order should not be mistaken for evidence of a stronger one.

Definition: If \(E[|X|]\) and \(E[|Y|]\) are finite, write \(X\le_{\mathrm{cx}}Y\) (convex order) when \(E[\phi(X)]\le E[\phi(Y)]\) for every convex function \(\phi\) for which these expectations exist. Convex order requires \(E[X]=E[Y]\); equivalently, one may include equality of the means as part of its definition.

Convex order asks whether one distribution is more spread out in a way detected by every convex function, while keeping the mean fixed. It is useful for dispersion and mean-preserving-spread questions—not as a general statement that one variable tends to be larger. In particular, \(X\le_{\mathrm{cx}}Y\) does not generally imply either usual stochastic ordering.

Worked Comparisons

Worked Example: Choosing Hazard-Rate Order for Lifetimes

Let \(X\) have a gamma distribution with shape 2 and scale 1, and let \(Y\) have a gamma distribution with shape 2 and scale 2. Their densities, for \(t>0\), are \(f_X(t)=t e^{-t}\) and \(f_Y(t)=\frac{t}{4}e^{-t/2}\). We ask whether one lifetime has a consistently lower hazard rate.

For a gamma distribution with shape 2 and scale \(\theta>0\), integrating the density gives \(S_\theta(t)=e^{-t/\theta}(1+t/\theta)\). Thus \(h_\theta(t)=f_\theta(t)/S_\theta(t)=t/[\theta(\theta+t)]\). For every \(t>0\),

h_X(t)=\frac{t}{1+t}>\frac{t}{2(2+t)}=h_Y(t),

so \(X\le_{\mathrm{hr}}Y\): the shape-2, scale-1 lifetime has the higher hazard at every positive time. The density ratio also checks directly:

\frac{f_Y(t)}{f_X(t)} =\frac{1}{4}e^{t/2},

which is increasing, so \(X\le_{\mathrm{lr}}Y\) as well. This example illustrates when hazard-rate order is informative: the comparison concerns relative failure rates throughout follow-up, not just a single survival probability.

Worked Example: Choosing Usual Stochastic Order for Thresholds

Let \(X\sim\operatorname{Bernoulli}(0.2)\) and \(Y\sim\operatorname{Bernoulli}(0.65)\). Their values are 0 and 1. For \(0\le t<1\), \(P(X>t)=0.2\) and \(P(Y>t)=0.65\). For \(t<0\), both probabilities are 1; for \(t\ge1\), both are 0. Hence \(P(X>t)\le P(Y>t)\) for every real \(t\), and \(X\le_{\mathrm{st}}Y\).

If the practical question is whether the outcome exceeds a threshold, this is exactly the needed comparison. For any threshold between 0 and 1, the exceedance probability for \(Y\) is 0.65, compared with 0.2 for \(X\). No density-ratio or hazard-rate analysis is required to answer that tail question.

Worked Example: Choosing Likelihood-Ratio Order for a Density Ratio

Let \(X\) have the uniform distribution on \([0,1]\), with density \(f_X(t)=1\) there. Let \(Y\) have density \(f_Y(t)=2t\) on the same interval. The density ratio is

\frac{f_Y(t)}{f_X(t)}=2t,\qquad 0<t<1.

Since \(2t\) is increasing, \(X\le_{\mathrm{lr}}Y\). The order says the relative density of \(Y\) grows with the observed value. It also gives a usual stochastic comparison: for \(0\le t\le1\), \(F_X(t)=t\) and \(F_Y(t)=t^2\), so \(F_X(t)\ge F_Y(t)\), and thus \(X\le_{\mathrm{st}}Y\). Here the density-ratio comparison answers a more structural question than the resulting tail comparison.

Worked Example: Choosing Convex Order for Dispersion

Let \(X=1\) with probability 1, and let \(Y\) equal 0 or 2 with probability \(1/2\) each. Both means are 1. For any convex \(\phi\), convexity gives

E[\phi(X)]=\phi(1) \le \frac{\phi(0)+\phi(2)}{2} =E[\phi(Y)].

Therefore \(X\le_{\mathrm{cx}}Y\). The comparison captures the fact that \(Y\) spreads probability away from the common mean. It does not give a usual stochastic ordering: for \(0\le t<1\), \(P(X>t)=1\) but \(P(Y>t)=1/2\), whereas for \(1\le t<2\), \(P(X>t)=0\) but \(P(Y>t)=1/2\). The tail comparison changes direction as the threshold changes.

Two Useful Checks

A distributional order is valuable only if it guarantees the type of conclusion needed. The following results make two of the distinctions precise.

Theorem: \(X\le_{\mathrm{st}}Y\) if and only if \(E[g(X)]\le E[g(Y)]\) for every bounded, nondecreasing, measurable function \(g:\mathbb{R}\to\mathbb{R}\).

Proof. Define the generalized quantile \(Q_Z(u)=\inf\{x:F_Z(x)\ge u\}\), for \(0<u<1\). If \(F_X(t)\ge F_Y(t)\) for all \(t\), then \(Q_X(u)\le Q_Y(u)\) for each \(u\): whenever \(F_Y(t)\ge u\), also \(F_X(t)\ge u\), so the first level at which \(F_X\) reaches \(u\) is no larger. If \(U\) is uniform on \((0,1)\), \(Q_X(U)\) has distribution \(F_X\), and \(Q_Y(U)\) has distribution \(F_Y\). Thus a nondecreasing \(g\) satisfies \(g(Q_X(U))\le g(Q_Y(U))\) pointwise; taking expectations proves the stated inequality.

Conversely, suppose the expectation inequality holds for every such \(g\). For any real \(t\), take \(g(x)=\mathbf{1}\{x>t\}\), a bounded nondecreasing measurable function. Then \(P(X>t)\le P(Y>t)\), which is the usual stochastic order. This proves both directions. \(\square\)

Result: If \(X\le_{\mathrm{cx}}Y\) and both variables have finite first moments, then \(E[X]=E[Y]\). If they also have finite second moments, then \(\operatorname{Var}(X)\le\operatorname{Var}(Y)\).

Proof. The functions \(\phi(x)=x\) and \(\phi(x)=-x\) are both convex. Applying the defining convex-order inequality to each gives \(E[X]\le E[Y]\) and \(-E[X]\le -E[Y]\), respectively. Hence \(E[X]=E[Y]\). If second moments exist, \(\phi(x)=x^2\) is convex, so \(E[X^2]\le E[Y^2]\). Subtracting the same squared mean from both sides yields \(\operatorname{Var}(X)\le\operatorname{Var}(Y)\). \(\square\)

The variance consequence is useful but does not define convex order: two distributions can have ordered variances without their expectations being ordered for every convex function. Convex order is the stronger, function-wide dispersion comparison.

A Practical Selection Guide

Question of interest	Order to consider	What the order directly controls
Which distribution has larger upper-tail probabilities?	Usual stochastic	Every threshold exceedance probability, and expectations of bounded increasing functions
Which lifetime has lower conditional failure risk across time?	Hazard-rate	A survival-ratio or, under regularity, a pointwise hazard comparison
Does the relative density increase with the observed value?	Likelihood-ratio	Monotonicity of the density ratio, subject to support conditions
Which equal-mean distribution is more dispersed?	Convex	Expectations of all convex functions

A practical workflow is to specify the quantity that matters, write the corresponding definition, and check only what is needed for the conclusion. If the goal is a tail probability, a hazard comparison may be stronger than necessary. If the goal is lifetime risk at every time, a single mean or a few tail probabilities do not establish that comparison. If the goal is dispersion at a common mean, stochastic order is not a substitute for convex order.

Common Mistakes

Using “larger” without saying what it means. Larger upper tails, lower hazards, and greater dispersion are different claims. Name the target before selecting an order.
Treating implications as equivalences. The standard hierarchy from likelihood-ratio to hazard-rate to usual stochastic order runs in one direction. A usual stochastic comparison alone does not establish a monotone density ratio.
Using convex order to claim larger values. Convex order compares convex summaries at a common mean; it need not order tail probabilities in one direction.
Ignoring support in a density-ratio argument. A ratio on an overlap does not by itself settle likelihood-ratio order when supports differ. Check that the conditions for the order being used are satisfied.

Key takeaway: Choose the order by the question: usual stochastic for tails, hazard-rate for lifetime risk, likelihood-ratio for monotone density ratios, and convex for equal-mean dispersion. Stronger orders can provide extra conclusions, but a conclusion should never be inferred from an order that does not guarantee it.

Check Your Understanding

For each question, identify the most directly relevant order and state one conclusion it guarantees.

A safety analyst wants to compare the probability that two models produce a value above every possible threshold. Which order is designed for this comparison?
Why is hazard-rate order a natural choice for comparing conditional failure risk over time?
If \(f_Y(t)/f_X(t)\) is increasing on a shared interval of positive densities, which order does this establish under the stated support conditions?
Two variables have equal means, and one has larger expectations for every convex function for which the expectations exist. Which order describes this comparison?
Does \(X\le_{\mathrm{cx}}Y\) by itself imply \(X\le_{\mathrm{st}}Y\)? Explain what feature each order compares.