Common Errors in Order Statistic Calculations

Why Order-Statistic Calculations Are Easy to Misread

An order statistic is defined by rank, but its probability is usually calculated by counting observations on one side of a threshold. Many errors occur in the translation between those two descriptions: an index is shifted by one, “at least” is replaced by “exactly,” or a strict inequality is used where an atom makes it different. Algebra can then disguise the mistake.

The safest checks are structural. State the event in words, translate it into a count, identify whether ties are possible, and keep the support visible. For a density, also check that the factorial coefficient follows from the selected rank and that the proposed density integrates to one. These checks are especially useful when a transformation reverses order.

Definitions and the Threshold-Count Rule

Definition: For observations \(X_1,\ldots,X_n\), their order statistics are the observations arranged in nondecreasing order, written \(X_{(1)}\le\cdots\le X_{(n)}\). Thus \(X_{(k)}\) is the \(k\)th smallest observation, counting multiplicities if values are tied. The sample minimum is \(X_{(1)}\), and the sample maximum is \(X_{(n)}\).

For any real \(t\), the key translation is that \(X_{(k)}\le t\) exactly when at least \(k\) observations are at most \(t\). The event \(X_{(k)}<t\) instead counts observations strictly less than \(t\). This distinction matters whenever there is positive probability of observing \(t\).

Theorem: If \(X_1,\ldots,X_n\) are independent and identically distributed with distribution function \(F\), then

P(X_{(k)}\le t) =\sum_{j=k}^{n}\binom{n}{j}F(t)^j\{1-F(t)\}^{n-j}.

Indeed, the number of observations at most \(t\) has a binomial distribution with parameters \(n\) and \(F(t)\), and \(X_{(k)}\le t\) means this number is at least \(k\). This probability formula remains valid when the distribution has atoms. For the strict event, replace \(F(t)\) by \(P(X_1<t)\); that probability need not equal \(F(t)\).

Worked Examples: Counts, Coefficients, and Support

Worked Example: The Second Order Statistic of a Bernoulli Sample

Let \(X_1,X_2,X_3\) be independent Bernoulli random variables with \(P(X_i=1)=0.3\), and consider the median \(X_{(2)}\). Find \(P(X_{(2)}\le 0)\) and \(P(X_{(2)}<0)\).

The event \(X_{(2)}\le0\) requires at least two observations to be zero. Since each observation is zero with probability \(0.7\), its probability is

P(X_{(2)}\le0) =\binom{3}{2}(0.7)^2(0.3)+(0.7)^3 =3(0.49)(0.3)+0.343 =0.441+0.343 =0.784.

By contrast, no observation is strictly less than zero, so \(P(X_{(2)}<0)=0\). The difference, \(0.784\), is the probability that the median equals zero. Applying a continuous-distribution formula that treats \(<\) and \(\le\) as interchangeable would lose this mass.

Worked Example: Auditing a Density on a Bounded Support

Let \(X_1,\ldots,X_4\) be independent uniform observations on \([2,5]\). Derive the density of \(X_{(2)}\), the second-smallest observation.

For \(2<x<5\), the parent distribution has \(F(x)=(x-2)/3\) and \(f(x)=1/3\). The order-statistic density for an absolutely continuous parent distribution is

f_{X_{(k)}}(x) =\frac{n!}{(k-1)!(n-k)!} F(x)^{k-1}\{1-F(x)\}^{n-k}f(x).

Here \(n=4\) and \(k=2\), so the factorial coefficient is \(4!/(1!2!)=12\), not \(4!\) and not \(\binom{4}{2}\). Substitution gives

\begin{aligned} f_{X_{(2)}}(x) &=12\left(\frac{x-2}{3}\right) \left(1-\frac{x-2}{3}\right)^2\frac{1}{3}\\ &=12\left(\frac{x-2}{3}\right) \left(\frac{5-x}{3}\right)^2\frac{1}{3}\\ &=\frac{4(x-2)(5-x)^2}{27},\qquad 2<x<5. \end{aligned}

The numerical factors deserve an explicit check. The product has four factors of \(3\) in its denominator: one from \(F(x)\), two from \(\{1-F(x)\}^2\), and one from \(f(x)\). Thus the polynomial coefficient is \(12/3^4=12/81=4/27\), approximately \(0.148148\). The intermediate value \(12/3^3=4/9\), approximately \(0.4444\), is not the coefficient of \((x-2)(5-x)^2\); one further factor of \(1/3\) remains. The density is zero outside \([2,5]\), apart from irrelevant choices at endpoints.

A normalization check confirms the result. With \(y=x-2\), the integral of the polynomial factor over the support is

\int_2^5(x-2)(5-x)^2\,dx =\int_0^3 y(3-y)^2\,dy =\left[\frac{9y^2}{2}-2y^3+\frac{y^4}{4}\right]_0^3 =\frac{27}{4}.

Multiplying by \(4/27\) gives total integral \(1\). Using \(0.4444\) as the polynomial coefficient instead would give total integral \(3\), so it cannot be the density coefficient.

Worked Example: A Decreasing Transformation Reverses Ranks

Suppose the observed values are \(1,3,5,7\), so their order statistics are \(X_{(1)}=1\), \(X_{(2)}=3\), \(X_{(3)}=5\), and \(X_{(4)}=7\). Transform each value by the decreasing function \(g(x)=10-x\). What are the order statistics of the transformed observations?

The transformed observations corresponding to the original ordered values are \(9,7,5,3\), which are in decreasing order. Rearranging them increasingly gives \(3,5,7,9\). Therefore the second-smallest transformed value is \(5=g(X_{(3)})\), not \(g(X_{(2)})\).

In general, if \(g\) is strictly decreasing, then for any sample,

Y_{(k)}=g\!\left(X_{(n-k+1)}\right), \qquad Y_i=g(X_i).

The reversal follows because \(X_{(1)}\le\cdots\le X_{(n)}\) implies \(g(X_{(1)})\ge\cdots\ge g(X_{(n)})\). Thus the \(k\)th smallest transformed value comes from the \((n-k+1)\)th smallest original value. For a strictly increasing \(g\), by contrast, \(Y_{(k)}=g(X_{(k)})\).

Why the Density Coefficient Has Its Particular Form

The density formula can be understood by asking what must happen near a value \(x\) for \(X_{(k)}\) to fall in a short interval around \(x\). For independent observations with an absolutely continuous common distribution, one observation must fall in that interval, \(k-1\) must fall below it, and \(n-k\) must fall above it. The observation in the interval can be chosen in \(n\) ways; the observations below can be chosen in \(\binom{n-1}{k-1}\) ways. Consequently, the count of arrangements is

n\binom{n-1}{k-1} =\frac{n!}{(k-1)!(n-k)!}.

For small interval width \(h>0\), the probability of this arrangement is, to first order,

\frac{n!}{(k-1)!(n-k)!} F(x)^{k-1}\{1-F(x)\}^{n-k} f(x)h.

The probability that two or more observations fall in that interval is of smaller order than \(h\), because the observations are independent and each has probability of order \(h\) of landing there. Dividing the first-order probability by \(h\) yields the density formula at points where the parent density and distribution function have the required regularity. More directly, differentiating the finite binomial-tail expression gives the same result; no interchange of an infinite sum and a limit is involved.

This reasoning also explains common coefficient errors. The selected observation contributes the factor \(n\); choosing which \(k-1\) of the remaining observations are below contributes \(\binom{n-1}{k-1}\). Their product is the factorial coefficient. A coefficient that omits the selected observation or counts arrangements twice will generally fail the total-integral check.

Common Mistakes and Reliable Checks

Shifting the rank. The event \(X_{(k)}\le t\) requires at least \(k\) observations at most \(t\), not at least \(k-1\). Test the statement at \(k=1\): the minimum is at most \(t\) when at least one observation is at most \(t\).
Replacing “at least” by “exactly.” The event \(X_{(k)}\le t\) includes \(k,k+1,\ldots,n\) observations at most \(t\). Keeping only exactly \(k\) misses valid outcomes.
Ignoring atoms. In general, \(P(X_{(k)}\le t)\) and \(P(X_{(k)}<t)\) differ. Use \(P(X_1\le t)\) for the first threshold count and \(P(X_1<t)\) for the second.
Dropping the support restriction. A formula derived for \(a<x<b\) does not define a density outside the parent support \([a,b]\). Check the support before evaluating or integrating a proposed density.
Miscounting factorials or scale factors. Substitute \(n\) and \(k\) into \(n!/((k-1)!(n-k)!)\) separately, then simplify the distribution and density factors. Keep each scale factor visible until the last line.
Failing to reverse ranks. A decreasing transformation reverses the ordering. For \(Y_i=g(X_i)\), use \(g(X_{(n-k+1)})\) for \(Y_{(k)}\), not \(g(X_{(k)})\).

A proposed density should pass two immediate checks: it must be nonnegative on its stated support, and its integral over that support must be one. These checks do not prove a formula is correct, but they quickly expose many index, coefficient, and support errors. For discrete or mixed distributions, use probability masses or the threshold-count distribution rather than assuming an ordinary density formula applies.

Key takeaway: Translate ranks into counts before calculating. Use strict or non-strict threshold probabilities according to the event, preserve the parent support, derive the factorial coefficient from the arrangement count, and reverse the rank under a decreasing transformation. Finish by checking normalization whenever a density is claimed.

Check Your Understanding

For each question, identify the event or calculation that prevents the common error.

For \(X_{(3)}\le t\), how many observations must be at most \(t\), at minimum?
Why can \(P(X_{(k)}<t)\) differ from \(P(X_{(k)}\le t)\) for a discrete distribution?
For a sample of size \(5\), what is the factorial coefficient in the density of \(X_{(3)}\)?
If \(g\) is strictly decreasing, which original order statistic determines \(Y_{(1)}\) when \(Y_i=g(X_i)\)?
A proposed density on \([2,5]\) has integral \(1.4\). What does this tell you before any further interpretation?