Density Of An Iid Kth Order Statistic

From a Rank Event to a Density

The kth order statistic describes the observation at rank $k$ after a sample is arranged from smallest to largest. Its distribution depends both on the parent distribution and on how many sample observations must lie on either side of a value. Turning its cdf into a density makes those two ingredients explicit: the parent density supplies local probability near the value, while a combinatorial coefficient counts the possible rank arrangements.

Assume throughout that $X_1,\ldots,X_n$ are independent and identically distributed continuous random variables with cdf $F_X$ and density $f_X$. Write $X_{(k)}$ for the kth smallest observation, where $1\leq k\leq n$. Continuity means that ties have probability zero, so the ordered observations have unambiguous ranks with probability one. The formula below is for an iid sample from a continuous parent distribution; it should not be applied unchanged to a sample with atoms, where ties can occur.

Definition: For $n$ observations, $X_{(k)}$ is the kth value in the sample after arranging the observations in nondecreasing order. Thus $X_{(1)}$ is the minimum and $X_{(n)}$ is the maximum. For a continuous iid parent distribution, the density of $X_{(k)}$, wherever the parent density is defined, is $$ f_{X_{(k)}}(x)=\frac{n!}{(k-1)!(n-k)!}\,[F_X(x)]^{k-1}[1-F_X(x)]^{n-k}f_X(x). $$ The density is zero outside the support of the parent distribution. Equivalently, the coefficient is $n\binom{n-1}{k-1}$.

The factors have a direct interpretation. For an observation to occupy rank $k$ near $x$, exactly $k-1$ of the other observations must be below $x$, and $n-k$ must be above $x$. The powers of $F_X(x)$ and $1-F_X(x)$ describe those probabilities. The factor $f_X(x)$ accounts for the observation that falls in a small interval around $x$. The coefficient counts which sample observations take those roles.

Why Differentiating the CDF Gives This Formula

For a fixed real number $x$, let $N_x$ be the number of sample observations that are at most $x$. Independence and identical distribution imply $N_x\sim\operatorname{Bin}(n,F_X(x))$. The event $X_{(k)}\leq x$ occurs exactly when at least $k$ observations are at most $x$. Therefore the cdf of the order statistic is a binomial tail:

F_{X_{(k)}}(x)=P(X_{(k)}\leq x) =\sum_{j=k}^{n}\binom{n}{j}[F_X(x)]^j[1-F_X(x)]^{n-j}.

To differentiate, first regard the parent cdf value $p=F_X(x)$ as the variable. In the derivative of the sum, terms cancel in pairs. The identity $j\binom{n}{j}=n\binom{n-1}{j-1}$ converts the positive derivative terms into terms indexed from $k-1$ to $n-1$. The identity $(n-j)\binom{n}{j}=n\binom{n-1}{j}$ converts the negative terms into terms indexed from $k$ to $n-1$. Every term with index $k,\ldots,n-1$ appears once with each sign; only the index $k-1$ term remains:

\frac{d}{dp}\sum_{j=k}^{n}\binom{n}{j}p^j(1-p)^{n-j} =n\binom{n-1}{k-1}p^{k-1}(1-p)^{n-k}.

Since $dp/dx=f_X(x)$ wherever the cdf is differentiable, the chain rule gives the density formula. Finally, $n\binom{n-1}{k-1}=n!/((k-1)!(n-k)!)$, which explains the displayed factorial coefficient. This derivation also explains why the exponent of $F_X(x)$ is $k-1$, not $k$: exactly $k-1$ observations are below the observation occupying rank $k$.

Applying the Formula

A reliable calculation follows the same sequence each time: identify $n$ and $k$, write down $F_X$ and $f_X$, substitute into the general formula, and simplify only after all factors are present. Check that the result is supported where the parent variable can occur. If a probability is requested as well as a density, use the cdf or integrate the density; a density value itself is not a probability.

Worked Example: A Middle Rank from a Uniform Sample

Suppose $X_1,\ldots,X_5$ are iid uniform on $(0,10)$. Find the density of $X_{(3)}$, evaluate it at $x=4$, and check that the formula gives a valid density.

Method. Use the iid order-statistic density. Here $n=5$, $k=3$, $F_X(x)=x/10$, and $f_X(x)=1/10$ for $0<x<10$. Outside that interval, the density is zero.

The rank coefficient is $\frac{5!}{(3-1)!(5-3)!}=\frac{120}{2!2!}=30$. Substitution gives

f_{X_{(3)}}(x) =30\left(\frac{x}{10}\right)^2\left(1-\frac{x}{10}\right)^2\frac{1}{10} =0.0003x^2(10-x)^2,\qquad 0<x<10.

At $x=4$, the parent cdf is $4/10=0.4$, its complement is $0.6$, and its density is $0.1$. Thus $f_{X_{(3)}}(4)=30(0.4)^2(0.6)^2(0.1)=30(0.16)(0.36)(0.1)=0.1728$. This is a density value, not the probability that the order statistic equals 4.

Check. With the change of variable $p=x/10$, integrating the density over $(0,10)$ becomes $30\int_0^1 p^2(1-p)^2\,dp$. Expanding the integrand gives $30(1/3-1/2+1/5)=30/30=1$, as required. A common mistake is to omit the parent density $1/10$; doing so would produce a function whose integral over this ten-unit interval is 10 rather than 1.

Worked Example: The Second Order Statistic from Exponential Lifetimes

Suppose four hypothetical devices have independent, identically distributed lifetimes with exponential rate $2$ per time unit. Find the density of the second-smallest lifetime. Then find the probability that this lifetime is at most $t$, given that $F_X(t)=0.3$.

Method. For an exponential distribution with rate $\lambda=2$, $F_X(x)=1-e^{-2x}$ and $f_X(x)=2e^{-2x}$ for $x>0$. Here $n=4$ and $k=2$, so the coefficient is $4!/(1!2!)=12$. Substituting,

\begin{aligned} f_{X_{(2)}}(x) &=12(1-e^{-2x})e^{-4x}(2e^{-2x})\\ &=24(e^{-6x}-e^{-8x}),\qquad x>0. \end{aligned}

The density is zero for $x\leq0$. For the probability, use the rank event rather than treating the density value as a probability. At the stated threshold, the number of lifetimes at most $t$ has a binomial distribution with $n=4$ and success probability $0.3$. The second order statistic is at most $t$ if at least two lifetimes are at most $t$:

\begin{aligned} P(X_{(2)}\leq t) &=\binom{4}{2}(0.3)^2(0.7)^2+\binom{4}{3}(0.3)^3(0.7)+(0.3)^4\\ &=6(0.09)(0.49)+4(0.027)(0.7)+0.0081\\ &=0.2646+0.0756+0.0081\\ &=0.3483. \end{aligned}

Check. The probability is between 0 and 1 and is larger than the probability that all four lifetimes are at most $t$, namely $0.3^4=0.0081$, as it must be. The common mistake is to include only the case of exactly two lifetimes below the threshold. The event requires at least two, so the cases of three and four must also be included.

Worked Example: A Rank Density When the Parent Density Is Large

Let six hypothetical measurements be iid $N(0.5,1/(72\pi))$, using the convention that the second normal parameter is the variance. Find the density of $X_{(4)}$ at $x=0.5$.

Method. The parent mean is $0.5$, and the standard deviation is $\sqrt{1/(72\pi)}=1/(6\sqrt{2\pi})$. By symmetry, $F_X(0.5)=0.5$. The normal density at its mean is $1/(\sigma\sqrt{2\pi})=6$, so $f_X(0.5)=6$. These parent quantities must both be included in the order-statistic formula.

For $n=6$ and $k=4$, the coefficient is $\frac{6!}{3!2!}=\frac{720}{12}=60$. Therefore

\begin{aligned} f_{X_{(4)}}(0.5) &=60(0.5)^3(1-0.5)^2(6)\\ &=60\left(\frac18\right)\left(\frac14\right)(6)\\ &=\frac{60\cdot6}{32} =11.25. \end{aligned}

Check. The rank factor alone is $60/32=1.875$; multiplying by the parent density $6$ gives $11.25$. This illustrates why the density of an order statistic is not just the rank factor. A density can exceed 1, so the value 11.25 is not itself a probability. The common mistake is to omit $f_X(0.5)$ when it is not equal to 1.

What the Coefficient Does—and Does Not Do

The coefficient may be written in either of two equivalent ways: $n!/((k-1)!(n-k)!)$ or $n\binom{n-1}{k-1}$. In the second expression, choose which of the $n$ observations falls in a small interval around $x$, then choose which $k-1$ of the remaining $n-1$ observations lie below it. The other $n-k$ observations lie above it. This counts every possible assignment of sample labels to these roles.

For example, when $n=6$ and $k=4$, the coefficient is $6\binom{5}{3}=6(10)=60$, not $6\binom{6}{3}$. One observation occupies the rank interval; the binomial choice is made among the remaining five observations. The powers in the formula provide the probabilities of the lower and upper groups, while the coefficient accounts for their possible identities.

At the endpoints of a continuous parent distribution, the cdf may equal 0 or 1. The formula’s powers still describe the limiting behavior, with the density interpreted on the parent support and almost everywhere. For $k=1$, it reduces to $n[1-F_X(x)]^{n-1}f_X(x)$, the minimum density. For $k=n$, it reduces to $n[F_X(x)]^{n-1}f_X(x)$, the maximum density. These are useful checks on whether the exponents have been assigned to the correct ranks.

Key takeaway: For an iid continuous sample, the kth-order density is the parent density multiplied by the probability factors for $k-1$ observations below and $n-k$ above, and by the number of ways to assign the sample observations to those roles. Keep all three parts: coefficient, cdf powers, and parent density.

Common Exam Traps

Using the wrong rank. The kth smallest has $k-1$ observations below it and $n-k$ above it. For the fourth smallest in six observations, those counts are three and two.
Dropping the parent density. The cdf powers alone give a rank probability factor, not a density with respect to $x$. The chain rule contributes $f_X(x)$.
Using a probability interpretation for a density value. For a continuous variable, $P(X_{(k)}=x)=0$. A density value describes probability per unit of the measurement scale near $x$.
Changing the iid model without changing the formula. The binomial count argument uses identical success probability $F_X(x)$ for every observation, as well as independence. If the observations are not identically distributed or ties are possible, this derivation does not apply unchanged.

Check Your Understanding

For each problem, identify the sample size and rank before substituting into the density formula. Probabilities and densities are reported as indicated.

Five iid observations are uniform on $(0,4)$. Find the density of the second order statistic at $x=2$. (Answer: $0.3125$)
Three iid lifetimes have exponential rate $1.5$ per time unit. Find the density of the maximum at $x=(\ln 2)/1.5$. (Answer: $0.5625$)
Four iid observations have a continuous parent distribution with $F_X(t)=0.3$. Find $P(X_{(2)}\leq t)$, reporting four decimals. (Answer: $0.3483$)
Six iid observations have a continuous parent distribution with $F_X(0.5)=0.5$ and $f_X(0.5)=6$. Find the density of the fourth order statistic at $0.5$. (Answer: $11.25$)
Five iid observations are uniform on $(0,1)$. Find $P(0.25<X_{(3)}\leq0.75)$, reporting four decimals. (Answer: $0.7930$)

What You'll Learn