Order Statistics from a Uniform Sample

Uniform Samples and Their Order Statistics

Let $X_1,\ldots,X_n$ be independent observations from a continuous uniform distribution on $(a,b)$, where $a<b$, and write $L=b-a$. Arrange the observations in increasing order: $X_{(1)}<\cdots<X_{(n)}$, with probability one. The ordered observations are not independent: knowing one rank constrains where the others can lie. For a uniform sample, however, their densities have a particularly simple form. A single order statistic has a beta distribution after rescaling the interval, and joint densities can be used to calculate probabilities involving several ranks.

The key reduction is to set $U_i=(X_i-a)/L$. Then the $U_i$ are independent uniform random variables on $(0,1)$, and $U_{(k)}=(X_{(k)}-a)/L$. Thus we can derive formulas on the unit interval and transform back to $(a,b)$.

Definition: For $1\leq k\leq n$, the $k$th order statistic $X_{(k)}$ is the $k$th smallest of $X_1,\ldots,X_n$. If the observations are iid uniform on $(a,b)$, then the rescaled statistic $U_{(k)}=(X_{(k)}-a)/(b-a)$ has a beta distribution with parameters $k$ and $n+1-k$.

The Beta Density and Moments of One Rank

For positive integers $r,s$, the beta density with parameters $r,s$ is $u^{r-1}(1-u)^{s-1}/B(r,s)$ for $0<u<1$, where $B(r,s)=\int_0^1 u^{r-1}(1-u)^{s-1}\,du$. For integer parameters, $B(r,s)=(r-1)!(s-1)!/(r+s-1)!$. The density of a particular rank follows by counting how many observations must fall on either side of it.

Theorem: If $X_1,\ldots,X_n$ are iid uniform on $(a,b)$, then for $a<x<b$, $$ f_{X_{(k)}}(x)= \frac{n!}{(k-1)!(n-k)!} \frac{(x-a)^{k-1}(b-x)^{n-k}}{(b-a)^n}. $$ Equivalently, $U_{(k)}\sim\operatorname{Beta}(k,n+1-k)$.

To see the density, consider $X_{(k)}$ in a small interval of width $h$ near $x$. For it to be there, exactly $k-1$ observations must be below $x$, one must be in that small interval, and the remaining $n-k$ must be above $x$. The number of assignments of observations to these groups is $n!/((k-1)!(n-k)!)$. Since a uniform observation has density $1/L$, the probability is, to first order in $h$, $$ \frac{n!}{(k-1)!(n-k)!} \left(\frac{x-a}{L}\right)^{k-1} \frac{h}{L} \left(\frac{b-x}{L}\right)^{n-k}. $$ Dividing by $h$ and taking the limit gives the stated density. The beta form follows by substituting $u=(x-a)/L$; the density of $U_{(k)}$ is the beta density with $r=k$ and $s=n+1-k$.

Result: The mean and variance of the $k$th order statistic are $$ E[X_{(k)}]=a+(b-a)\frac{k}{n+1}, \qquad \operatorname{Var}(X_{(k)})=(b-a)^2 \frac{k(n+1-k)}{(n+1)^2(n+2)}. $$

For completeness, the beta moments follow from the beta integral. For positive integers $r,s$, integration by parts gives $B(r+1,s)=\frac{r}{r+s}B(r,s)$; applying this relation twice yields $B(r+2,s)=\frac{r(r+1)}{(r+s)(r+s+1)}B(r,s)$. Hence, if $U$ has the beta density with parameters $r,s$, $$ E[U]=\frac{B(r+1,s)}{B(r,s)}=\frac{r}{r+s}, \qquad E[U^2]=\frac{r(r+1)}{(r+s)(r+s+1)}. $$ Subtracting $E[U]^2$ from $E[U^2]$ gives $\operatorname{Var}(U)=rs/((r+s)^2(r+s+1))$. Setting $r=k$, $s=n+1-k$, then using $X_{(k)}=a+LU_{(k)}$, gives the stated mean and variance.

Worked Example: A Quantile of a Uniform Sample

Let $X_1,\ldots,X_5$ be iid uniform on $(0,1)$. Find the density and mean of $X_{(2)}$, and calculate $P(X_{(2)}>0.4)$.

For $n=5$ and $k=2$, the density is $$ f_{X_{(2)}}(x)=\frac{5!}{1!3!}x(1-x)^3=20x(1-x)^3, \qquad 0<x<1. $$ This is the beta density with parameters $2,4$, and the mean formula gives $E[X_{(2)}]=2/6=1/3$.

The event $X_{(2)}>0.4$ occurs exactly when at most one of the five observations is at most $0.4$. The number of observations at most $0.4$ has a binomial distribution with success probability $0.4$, so $$ P(X_{(2)}>0.4) =\binom{5}{0}(0.6)^5+\binom{5}{1}(0.4)(0.6)^4 =0.07776+0.2592 =0.33696. $$ The equivalent density calculation is $\int_{0.4}^1 20x(1-x)^3\,dx=0.33696$, in agreement with the count argument.

Joint Densities for Two Ranks

A joint density is useful when the event constrains two order statistics at once. For $1\leq i<j\leq n$, the observations must divide into four groups: $i-1$ below $x$, one near $x$, $j-i-1$ between $x$ and $y$, one near $y$, and $n-j$ above $y$. This gives the following formula.

Theorem: For iid uniform observations on $(a,b)$, the joint density of $X_{(i)},X_{(j)}$ is zero except when $a<x<y<b$, and on that region it is $$ f_{X_{(i)},X_{(j)}}(x,y)= \frac{n!}{(i-1)!(j-i-1)!(n-j)!} \frac{(x-a)^{i-1}(y-x)^{j-i-1}(b-y)^{n-j}}{(b-a)^n}. $$

To derive it, take small intervals of widths $h$ and $q$ around $x$ and $y$, with the intervals disjoint and $x<y$. There are $n!/((i-1)!(j-i-1)!(n-j)!)$ assignments to the groups below $x$, near $x$, between $x$ and $y$, near $y$, and above $y$. Multiplying the probabilities for these groups gives, to first order in $hq$, $$ \frac{n!}{(i-1)!(j-i-1)!(n-j)!} \left(\frac{x-a}{L}\right)^{i-1} \frac{h}{L} \left(\frac{y-x}{L}\right)^{j-i-1} \frac{q}{L} \left(\frac{b-y}{L}\right)^{n-j}. $$ Dividing by $hq$ and letting both widths tend to zero gives the displayed joint density. It is zero when $x\geq y$, because two ordered ranks cannot occur in that order there.

Worked Example: A Probability Involving Two Ranks

Let $X_1,\ldots,X_4$ be iid uniform on $(0,1)$. Find $P(X_{(1)}>0.2,\ X_{(3)}<0.7)$ using the joint density.

For $i=1,j=3,n=4$, the joint density is $24(y-x)(1-y)$ on $0<x<y<1$. The requested event restricts $x$ to $0.2<x<0.7$ and $y$ to $x<y<0.7$. Therefore $$ \begin{aligned} P(X_{(1)}>0.2,\ X_{(3)}<0.7) &=24\int_{0.2}^{0.7}\int_x^{0.7}(y-x)(1-y)\,dy\,dx\\ &=24\int_{0.2}^{0.7}(y-x)(1-y)\text{ integrated over }x\text{ first}\\ &=12\int_{0.2}^{0.7}(y-0.2)^2(1-y)\,dy. \end{aligned} $$ In the last line the order of integration was reversed: for each $y$, $0.2<x<y$, and $\int_{0.2}^y(y-x)\,dx=(y-0.2)^2/2$. Set $t=y-0.2$; then $0<t<0.5$ and $1-y=0.8-t$. Thus $$ 12\int_0^{0.5}t^2(0.8-t)\,dt =12\left(\frac{1}{30}-\frac{1}{64}\right) =\frac{17}{80} =0.2125. $$ As a check, all four observations must exceed $0.2$, and at least three must be below $0.7$. The interval $(0.2,0.7)$ has probability $0.5$, and $(0.7,1)$ has probability $0.3$; the probability is $\binom{4}{3}(0.5)^3(0.3)+(0.5)^4=0.15+0.0625=0.2125$.

The Range of a Uniform Sample

For $n\geq2$, define the sample range by $R=X_{(n)}-X_{(1)}$. The joint density formula for the minimum and maximum has $i=1,j=n$, so $$ f_{X_{(1)},X_{(n)}}(x,y) =\frac{n(n-1)(y-x)^{n-2}}{L^n}, \qquad a<x<y<b. $$ This specialized joint law makes the uniform range distribution particularly direct.

Theorem: If $X_1,\ldots,X_n$ are iid uniform on an interval of length $L=b-a$, then $$ F_R(r)= \begin{cases} 0, & r<0,\\ n(r/L)^{n-1}-(n-1)(r/L)^n, & 0\leq r\leq L,\\ 1, & r>L. \end{cases} $$ For $0<r<L$, its density is $$ f_R(r)=\frac{n(n-1)}{L^n}r^{n-2}(L-r). $$

For $0\leq r\leq L$, integrate the joint density over pairs whose separation is at most $r$. For a separation $s=y-x$, the left endpoint $x$ ranges from $a$ to $b-s$, a length $L-s$. Hence $$ \begin{aligned} P(R\leq r) &=\int_0^r\int_a^{b-s}\frac{n(n-1)s^{n-2}}{L^n}\,dx\,ds\\ &=\frac{n(n-1)}{L^n}\int_0^r s^{n-2}(L-s)\,ds\\ &=n(r/L)^{n-1}-(n-1)(r/L)^n. \end{aligned} $$ Differentiating this expression for $0<r<L$ gives the stated density. At $r=L$ the CDF is $n-(n-1)=1$, as required.

Worked Example: Exact Range Probability and Mean

Suppose four iid observations are uniform on $(2,8)$, so $L=6$. Find the probability that their range is at most $3$, and find the expected range.

Here $n=4$, and $3/L=1/2$. The range CDF gives $$ P(R\leq3)=4(1/2)^3-3(1/2)^4 =\frac{1}{2}-\frac{3}{16} =\frac{5}{16}. $$ To obtain the expected range, use the density and integrate over its support: $$ \begin{aligned} E[R] &=\int_0^L r\,\frac{n(n-1)}{L^n}r^{n-2}(L-r)\,dr\\ &=\frac{n(n-1)}{L^n} \left(\frac{L^{n+1}}{n}-\frac{L^{n+1}}{n+1}\right)\\ &=L\frac{n-1}{n+1}. \end{aligned} $$ For $n=4$ and $L=6$, this is $E[R]=6(3/5)=18/5=3.6$. The mean exceeding $3$ is consistent with a range distribution that places substantial probability above $3$.

Common Mistakes and Why the Formulas Matter

The beta parameters for the $k$th rank are $k$ and $n+1-k$, not $k$ and $n-k$. There are $k-1$ observations below the selected rank, one observation at the rank, and $n-k$ above it; the rank itself accounts for the additional $1$ in the second beta parameter. For example, the maximum has beta parameters $n,1$, not $n,0$.

A second error is to use a unit-interval density without transforming its scale. If $U_{(k)}=(X_{(k)}-a)/L$, then means transform by $a+L E[U_{(k)}]$, while variances transform by $L^2\operatorname{Var}(U_{(k)})$. A density must also include its scale factor: the density of $X_{(k)}$ contains $L^{-n}$ together with the powers of the distances from the endpoints.

These formulas are useful whenever a sample is genuinely uniform or can be reduced to a uniform sample by a probability integral transform. They provide exact finite-sample probabilities and moments without a large-sample approximation. For multiple ranks, the joint density also makes explicit why their values are dependent and how the interval between them contributes to the calculation.

Key takeaway: On a uniform interval, rescaling turns the $k$th order statistic into a beta random variable. The beta parameters determine its moments, while the joint density for two ranks and the density of the extremes support exact calculations involving several ordered observations and the sample range.

Check Your Understanding

Use the formulas in this tutorial to answer the following questions.

For a sample of size $8$ from $(0,1)$, identify the beta distribution of $X_{(5)}$ and calculate its mean.
For a sample of size $6$ from $(2,10)$, state the support and beta parameters of the rescaled third order statistic.
For $n=3$ on $(0,1)$, write the joint density of $X_{(1)}$ and $X_{(3)}$, including its support.
For $n=5$ on an interval of length $4$, calculate $P(R\leq2)$.
Explain why the variance of a rank on $(a,b)$ scales by $(b-a)^2$, rather than by $b-a$.