Order Statistics Under Monotone Transformations

Why Monotone Transformations Affect Ranks

Order statistics describe the sorted values in a sample. A monotone transformation changes the values, but its effect on their ranks is especially simple: an increasing function preserves their order, while a decreasing function reverses it. This observation lets us derive distributions for transformed order statistics without sorting a transformed sample from scratch.

The distinction between increasing and decreasing matters even when the original observations are continuous. With discrete observations, ties can occur; the same rank identities still hold for monotone functions, but transformed distributions may have atoms, and a density formula may not apply.

Order Statistics and the Rank Identities

Definition: Given real-valued observations \(X_1,\ldots,X_n\), their order statistics are the sorted values \(X_{(1)}\le \cdots \le X_{(n)}\). For a function \(g\), the transformed observations are \(Y_i=g(X_i)\), and their order statistics are \(Y_{(1)}\le \cdots \le Y_{(n)}\).

The next result is a pointwise statement: it holds for each realized sample, not merely with probability one. It therefore covers samples with ties as well as samples without ties.

Theorem: If \(g\) is nondecreasing on a set containing all the observations, then \(Y_{(k)}=g(X_{(k)})\) for \(k=1,\ldots,n\). If \(g\) is nonincreasing on that set, then \(Y_{(k)}=g(X_{(n-k+1)})\) for \(k=1,\ldots,n\).

Proof. In the nondecreasing case, \(X_{(i)}\le X_{(j)}\) whenever \(i\le j\), so \(g(X_{(i)})\le g(X_{(j)})\). Thus the list \(g(X_{(1)}),\ldots,g(X_{(n)})\) is already sorted, including any equal values. Its \(k\)th entry is \(g(X_{(k)})\).

In the nonincreasing case, \(X_{(i)}\le X_{(j)}\) implies \(g(X_{(i)})\ge g(X_{(j)})\). The transformed values in the order \(g(X_{(n)}),g(X_{(n-1)}),\ldots,g(X_{(1)})\) are therefore nondecreasing. The \(k\)th entry in that sorted list is \(g(X_{(n-k+1)})\). This argument also permits equalities in either the observations or their transformed values. \(\square\)

Strict monotonicity is not needed for the rank identities. It is needed, however, if we want to use an ordinary inverse to express a transformed CDF as a function of the original CDF. If \(g\) is non-strict, distinct input values may be collapsed to the same output value.

Distribution and Density Under a Strictly Monotone Map

Let \(X_1,\ldots,X_n\) be independent and identically distributed with continuous CDF \(F\), and write \(F_{(r)}\) and \(f_{(r)}\) for the CDF and density of \(X_{(r)}\). For reference, the order-statistic formulas are

F_{(r)}(x)=\sum_{j=r}^{n}\binom{n}{j}F(x)^j[1-F(x)]^{n-j}, \qquad f_{(r)}(x)=\frac{n!}{(r-1)!(n-r)!}F(x)^{r-1}[1-F(x)]^{n-r}f(x).

The CDF formula follows by counting how many of the \(n\) observations are at most \(x\): \(X_{(r)}\le x\) exactly when at least \(r\) observations are at most \(x\). The density formula is its derivative when \(F\) has density \(f\).

Theorem: Suppose \(g\) is strictly monotone and continuously differentiable on the support of \(X_{(r)}\), with nonzero derivative there and a differentiable inverse on the interior of its image. For \(Y_{(k)}=g(X_{(r)})\), its density on that image is \[ f_{Y_{(k)}}(y)=f_{(r)}(g^{-1}(y))\left|(g^{-1})'(y)\right|. \] For an increasing \(g\), take \(r=k\); for a decreasing \(g\), take \(r=n-k+1\).

Proof. The rank identity reduces \(Y_{(k)}\) to a monotone transformation of the single random variable \(X_{(r)}\). If \(g\) is increasing, then \(P(Y_{(k)}\le y)=P(X_{(r)}\le g^{-1}(y))=F_{(r)}(g^{-1}(y))\). Differentiation by the chain rule gives \(f_{(r)}(g^{-1}(y))(g^{-1})'(y)\); the derivative is positive in this case.

If \(g\) is decreasing, then \(P(Y_{(k)}\le y)=P(X_{(r)}\ge g^{-1}(y))\). Since \(X_{(r)}\) has a continuous distribution, this is \(1-F_{(r)}(g^{-1}(y))\). Differentiating gives \(f_{(r)}(g^{-1}(y))[-(g^{-1})'(y)]\), which equals the claimed expression because \((g^{-1})'(y)<0\). The density is zero outside the image of the support. \(\square\)

For a discrete \(X_{(r)}\), the same CDF event relations can be used, but the continuous complement identity needs care: \(P(X_{(r)}\ge x)=1-P(X_{(r)}<x)\), not necessarily \(1-P(X_{(r)}\le x)\). The transformed law is then obtained by mapping the probability masses, and need not have a density.

Worked Examples

Worked Example: Taking Logarithms of an Exponential Sample

Let \(X_1,X_2,X_3\) be iid exponential random variables with rate \(2\), so \(F(x)=1-e^{-2x}\) and \(f(x)=2e^{-2x}\) for \(x>0\). Set \(Y_i=\log X_i\). Find the CDF and density of \(Y_{(2)}\).

The logarithm is strictly increasing on \((0,\infty)\), so \(Y_{(2)}=\log X_{(2)}\). For \(y\in\mathbb R\), the event \(Y_{(2)}\le y\) is equivalent to \(X_{(2)}\le e^y\). For \(n=3,r=2\), at least two observations must be at most \(e^y\), giving

F_{Y_{(2)}}(y)=3F(e^y)^2-2F(e^y)^3 =3(1-e^{-2e^y})^2-2(1-e^{-2e^y})^3.

The density of \(X_{(2)}\) is \(6F(x)[1-F(x)]f(x)\). Substituting the exponential CDF and density gives \(f_{(2)}(x)=6(1-e^{-2x})e^{-2x}(2e^{-2x})=12(1-e^{-2x})e^{-4x}\). Since \(x=e^y\) and \(dx/dy=e^y\), the transformed density is

f_{Y_{(2)}}(y)=12e^y(1-e^{-2e^y})e^{-4e^y},\qquad -\infty<y<\infty.

The support is all of \(\mathbb R\), as expected: the logarithm maps every positive value to a real number. The density calculation uses the positive Jacobian \(e^y\).

Worked Example: A Decreasing Reciprocal Transformation

Let \(X_1,\ldots,X_4\) be iid Uniform\((0,1)\), and transform each observation by \(Y_i=1/(1+2X_i)\). Find the density of \(Y_{(2)}\).

The map \(g(x)=1/(1+2x)\) is strictly decreasing, with image \((1/3,1)\). Rank reversal gives \(Y_{(2)}=g(X_{(3)})\), not \(g(X_{(2)})\). For the third order statistic in a sample of four,

f_{(3)}(x)=\frac{4!}{2!0!}x^2(1-x)=12x^2(1-x),\qquad 0<x<1.

The inverse transformation is \(x=u=(1-y)/(2y)\), with \(|dx/dy|=1/(2y^2)\). Therefore,

f_{Y_{(2)}}(y)=12u^2(1-u)\frac{1}{2y^2} =\frac{6u^2(1-u)}{y^2},\qquad \frac13<y<1.

Substitute \(u=(1-y)/(2y)\), so \(u^2=(1-y)^2/(4y^2)\) and \(1-u=(3y-1)/(2y)\). Hence

f_{Y_{(2)}}(y) =\frac{6}{y^2}\frac{(1-y)^2}{4y^2}\frac{3y-1}{2y} =\frac{3(1-y)^2(3y-1)}{4y^5},\qquad \frac13<y<1.

As a check on the transformation and the algebra, its integral over the support is one: substituting back to \(x\) gives \(\int_{1/3}^{1} f_{Y_{(2)}}(y)\,dy=\int_0^1 12x^2(1-x)\,dx=12(1/3-1/4)=1\).

Worked Example: Squaring the Sample Minimum

Let \(X_1,X_2\) be iid exponential with rate \(1\), and set \(Y_i=X_i^2\). Find the CDF and density of \(Y_{(1)}\).

Squaring is strictly increasing on the support \((0,\infty)\), so \(Y_{(1)}=X_{(1)}^2\). The minimum has CDF \(F_{(1)}(x)=1-[1-F(x)]^2=1-e^{-2x}\) for \(x>0\). Thus for \(y>0\),

F_{Y_{(1)}}(y)=P(X_{(1)}^2\le y) =F_{(1)}(\sqrt y)=1-e^{-2\sqrt y}.

Differentiating on \(y>0\) yields

f_{Y_{(1)}}(y)=e^{-2\sqrt y}\frac{1}{\sqrt y},\qquad y>0.

The support is \((0,\infty)\). Although this density becomes unbounded as \(y\) approaches zero, it is integrable: its CDF tends to zero at zero and to one as \(y\) tends to infinity.

Using the Rank Identity Correctly

The identities are useful whenever a transformed sample is easier to understand through the original order statistics. For an increasing map, the sample minimum remains the minimum and the maximum remains the maximum. For a decreasing map, the minimum becomes the transformed original maximum, and the maximum becomes the transformed original minimum. Intermediate ranks reverse in the same way.

For instance, if \(g\) is strictly decreasing and the original sample has no ties, \(Y_{(1)}=g(X_{(n)})\) and \(Y_{(n)}=g(X_{(1)})\). The event for a decreasing map also reverses its inequality: \(g(X_{(r)})\le y\) is equivalent to \(X_{(r)}\ge g^{-1}(y)\). Forgetting either reversal leads to using the wrong order statistic or the wrong tail probability.

Common pitfall: Applying a decreasing transformation to the \(k\)th order statistic and calling it the \(k\)th transformed order statistic. In general, the correct identity is \(Y_{(k)}=g(X_{(n-k+1)})\). Then, when deriving a density, use the density for rank \(n-k+1\) and multiply by the absolute inverse derivative. The absolute value handles the Jacobian sign; it does not correct a rank chosen incorrectly.

A second caution concerns ties. A strictly monotone function preserves equality as well as order, so ties remain ties. A merely non-strict function can introduce new ties by mapping distinct values to the same value. In either situation the pointwise rank identities remain valid, but a continuous density transformation is not a substitute for calculating masses when the transformed variable has atoms.

Check Your Understanding

Try these questions using the rank identities and transformation rules.

If \(g\) is strictly increasing, express \(Y_{(n)}\) in terms of \(X_{(n)}\). What changes if \(g\) is strictly decreasing?
For a decreasing transformation and a sample of size \(n\), which original order statistic determines \(Y_{(k)}\)?
Why does the density transformation use \(\left|(g^{-1})'(y)\right|\), including for a decreasing map?
If \(X_{(r)}\) has atoms, why can \(1-F_{(r)}(x)\) fail to equal \(P(X_{(r)}\ge x)\)?
For the reciprocal example, explain why \(Y_{(2)}\) depends on \(X_{(3)}\), and verify the support of \(Y_{(2)}\).