The Density of the Kth Order Statistic

From a Rank CDF to a Density

An order statistic describes a particular rank in a sample after the observations are sorted. Its CDF answers how likely that rank is to fall below a threshold. When the parent distribution has a density, differentiating that CDF gives a compact formula for the density of the rank. The formula combines three ingredients: the parent density at the threshold, the probability of observations below it, and the probability of observations above it.

Let \(X_1,\ldots,X_n\) be independent and identically distributed (iid) random variables with common CDF \(F\). Write their values in ascending order as \(X_{(1)}\leq\cdots\leq X_{(n)}\), so \(X_{(k)}\) is the observation in rank \(k\). The CDF expression for this rank follows by counting how many sample observations are at most \(x\). We use that expression here to obtain the density.

For any fixed \(x\), the number of observations at most \(x\) has a binomial distribution with parameters \(n\) and \(F(x)\). The event \(X_{(k)}\leq x\) occurs precisely when at least \(k\) observations are at most \(x\). Therefore,

The index \(j\) counts observations at most \(x\). To find the density, we differentiate this finite sum. The resulting terms cancel in a telescoping pattern, leaving just one term involving the chance of exactly \(k-1\) observations below \(x\).

The Density Formula

The factorial coefficient can also be written \(n\binom{n-1}{k-1}\). It accounts for the number of ways to place the \(k-1\) observations below the rank, the observation at the rank, and the remaining \(n-k\) observations above it. The powers of \(F(x)\) and \(1-F(x)\) encode those lower and upper observations, while \(f(x)\) describes the observation at \(x\).

Proof by Differentiating the Binomial Tail

Define the polynomial \(G(u)=\sum_{j=k}^{n}\binom{n}{j}u^j(1-u)^{n-j}\) for \(0\leq u\leq1\). For each term, differentiation and the identities \(j\binom{n}{j}=n\binom{n-1}{j-1}\) and \((n-j)\binom{n}{j}=n\binom{n-1}{j}\) give

When these derivatives are summed over \(j=k,\ldots,n\), the positive term indexed by \(j\) cancels the negative term indexed by \(j-1\), wherever both occur. The positive term at \(j=k\) remains; the negative term at \(j=n\) is zero. Thus

The CDF of \(X_{(k)}\) is \(G(F(x))\). Because \(F\) is absolutely continuous and \(G\) is a polynomial, \(G\circ F\) is absolutely continuous on every bounded interval. The chain rule for absolutely continuous functions gives \(\frac{d}{dx}G(F(x))=G'(F(x))f(x)\) almost everywhere. Substituting the expression for \(G'\) proves the density formula. Also, \(G(0)=0\) and \(G(1)=1\), so the CDF has the correct limits at the two tails; its nonnegative derivative integrates to one. \(\square\)

Worked Densities

Interpreting and Checking the Result

For an observation near \(x\) to occupy rank \(k\), exactly \(k-1\) of the other observations must lie below it, and \(n-k\) must lie above it. The density formula makes this structure visible: \(F(x)^{k-1}\) and \(\{1-F(x)\}^{n-k}\) represent those lower and upper positions, while \(f(x)\) supplies local density at the ranked observation. This is why the density is not generally just a multiple of the parent density.

For an absolutely continuous parent distribution, the probability of any pair of sample observations being equal is zero. Indeed, two independent variables with densities have probability zero of landing on the same point; a finite union over all pairs still has probability zero. Thus the rank interpretation has no tie ambiguity with positive probability. The density formula itself remains an almost-everywhere statement, which is the appropriate level of precision when a density or derivative is defined only almost everywhere.

A useful validation is to check the support and integral. The order statistic cannot fall outside the parent distribution's support, and the formula must integrate to one. The theorem's CDF proof ensures normalization generally; direct integration, as in the examples, can catch an omitted factor or an incorrect exponent in a particular calculation.

Common Mistake

Another frequent error is reversing the exponents. For the ascending rank \(k\), there are \(k-1\) observations below the ranked value and \(n-k\) above it. Therefore the powers are \(F(x)^{k-1}\) and \(\{1-F(x)\}^{n-k}\), respectively. A quick check at the rank endpoints is helpful: for \(k=1\), the exponent of \(F\) is zero; for \(k=n\), the exponent of \(1-F\) is zero.