The Beta Distribution Of Uniform Order Statistics

Why Uniform Order Statistics Have a Beta Distribution

When a sample is sorted, the value in a specified rank is not generally uniform, even when every observation in the original sample is uniform. For example, the second-smallest value in a sample tends to be larger than the smallest value. The beta distribution gives a compact description of this rank effect: its two parameters are determined directly by the rank and the sample size.

This connection is useful because beta densities have simple formulas for their means and variances. It also gives a direct way to calculate probabilities about ranks in a uniform sample. Throughout this tutorial, the observations are independent and identically distributed, and ties have probability zero.

Definition: Let $U_1,\ldots,U_n$ be independent random variables, each with the uniform distribution on $(0,1)$. Write their ordered values as $U_{(1)}\le \cdots \le U_{(n)}$. The random variable $U_{(k)}$ is the $k$th order statistic, where $1\le k\le n$.

Deriving the Beta Density

For $U_{(k)}$ to lie close to a particular value $u$, one observation must fall in a short interval around $u$, exactly $k-1$ observations must lie below that interval, and the remaining $n-k$ must lie above it. The uniform density is 1 on $(0,1)$, so the probability of an observation falling in an interval of width $du$ near $u$ is approximately $du$. The probability of being below $u$ is $u$, and the probability of being above $u$ is $1-u$.

There are $n$ choices for the observation in the short interval. Among the other $n-1$ observations, there are $\binom{n-1}{k-1}$ ways to choose which $k-1$ lie below it. Thus, after dividing by $du$ and taking the limit as $du$ approaches zero, the density is

f_{U_{(k)}}(u) = n\binom{n-1}{k-1}u^{k-1}(1-u)^{n-k} = \frac{n!}{(k-1)!(n-k)!}u^{k-1}(1-u)^{n-k}, \quad 0<u<1.

The density is zero outside $(0,1)$. The coefficient counts the possible assignments of observations to the three positions: below the rank value, in the short interval, and above it. The exponents record the probabilities of the observations below and above.

The beta density with parameters $a>0$ and $b>0$ is $u^{a-1}(1-u)^{b-1}/B(a,b)$ on $(0,1)$, where $B(a,b)$ is the beta function. For positive integers $a,b$, $B(a,b)=(a-1)!(b-1)!/(a+b-1)!$. Comparing the density with this form identifies the parameters.

Theorem: For an iid sample of size $n$ from the uniform distribution on $(0,1)$, the $k$th order statistic has a beta distribution: $$ U_{(k)}\sim \operatorname{Beta}(k,n-k+1). $$ Its density is $u^{k-1}(1-u)^{n-k}/B(k,n-k+1)$ for $0<u<1$.

The parameter $k$ counts the rank from the bottom; the parameter $n-k+1$ counts from the top, including the observation at rank $k$. This second parameter is a frequent source of off-by-one errors.

Mean and Variance From the Beta Parameters

For a beta random variable $V\sim\operatorname{Beta}(a,b)$, its density and the beta integral give

E[V]=\frac{B(a+1,b)}{B(a,b)}=\frac{a}{a+b}, \qquad E[V^2]=\frac{B(a+2,b)}{B(a,b)} =\frac{a(a+1)}{(a+b)(a+b+1)}.

These ratios follow from $B(a,b)=\Gamma(a)\Gamma(b)/\Gamma(a+b)$ and the identity $\Gamma(x+1)=x\Gamma(x)$. Subtracting the square of the mean from the second moment gives

\operatorname{Var}(V) =E[V^2]-E[V]^2 =\frac{ab}{(a+b)^2(a+b+1)}.

Substituting $a=k$ and $b=n-k+1$ gives the moments for a uniform order statistic:

E[U_{(k)}]=\frac{k}{n+1}, \qquad \operatorname{Var}(U_{(k)}) =\frac{k(n-k+1)}{(n+1)^2(n+2)}.

The mean $k/(n+1)$, rather than $k/n$, reflects that the expected positions of the ordered observations sit between the two endpoints 0 and 1. The formula also behaves sensibly at the extremes: the minimum has mean $1/(n+1)$, while the maximum has mean $n/(n+1)$.

Worked Example: Density and Moments of a Fourth Order Statistic

Let $U_1,\ldots,U_6$ be an iid sample from the uniform distribution on $(0,1)$. Find the density, mean, and variance of $U_{(4)}$.

Method. Match the sample size and rank to the beta parameters. Here $n=6$ and $k=4$, so $a=k=4$ and $b=n-k+1=3$. Thus $U_{(4)}\sim\operatorname{Beta}(4,3)$.

The density coefficient is $6!/[(4-1)!(6-4)!]=720/(3!\,2!)=720/12=60$. Therefore

f_{U_{(4)}}(u)=60u^3(1-u)^2,\quad 0<u<1.

The mean is $4/(4+3)=4/7$. The variance is

\operatorname{Var}(U_{(4)}) =\frac{4\cdot3}{(4+3)^2(4+3+1)} =\frac{12}{49\cdot8} =\frac{3}{98}.

As a check, the density integrates to $60B(4,3)=60(3!\,2!/6!)=60(1/60)=1$. The mean is between 0 and 1 and is above $1/2$, as expected for the fourth-smallest of six observations. Common mistake avoided: using $n-k$ as the second beta parameter would give 2, omitting the observation at rank $k$ from the count above the lower ranks.

Calculating Probabilities

A probability involving $U_{(k)}$ can be found by integrating its beta density. For integer parameters, it can also be useful to interpret the event as a count. For any $0\le t\le1$, $U_{(k)}\le t$ exactly when at least $k$ of the $n$ observations are at most $t$. Each observation is at most $t$ with probability $t$, independently, so the count has a binomial distribution with parameters $n$ and $t$. Therefore

P(U_{(k)}\le t) =\sum_{j=k}^{n}\binom{n}{j}t^j(1-t)^{n-j}.

This count representation agrees with integrating the beta density. It is especially convenient for integer sample sizes, while the density integral is often direct for an interval probability. For $0\le r<s\le1$, subtract the cumulative probability at $r$ from that at $s$.

Worked Example: Probability That the Second Observation Falls in an Interval

Five independent measurements are sampled from the uniform distribution on $(0,1)$. Find $P(0.2<U_{(2)}\le0.6)$. Give the probability to four decimal places.

Method. The second order statistic has distribution $\operatorname{Beta}(2,4)$, with density $20u(1-u)^3$ on $(0,1)$, since $5!/(1!\,3!)=20$. Its cumulative probability at $t$ is the probability that at least two of the five measurements are at most $t$. The complement is zero or one measurement at most $t$, so

P(U_{(2)}\le t) =1-(1-t)^5-5t(1-t)^4.

At $t=0.6$, the two complement terms are $(1-0.6)^5=0.4^5=0.01024$ and $5(0.6)(0.4)^4=3(0.0256)=0.0768$. Hence $P(U_{(2)}\le0.6)=1-0.01024-0.0768=0.91296$.

At $t=0.2$, the complement terms are $0.8^5=0.32768$ and $5(0.2)(0.8)^4=1(0.4096)=0.4096$. Hence $P(U_{(2)}\le0.2)=1-0.32768-0.4096=0.26272$. Subtracting gives $P(0.2<U_{(2)}\le0.6)=0.91296-0.26272=0.65024$, or 0.6502 to four decimal places.

The endpoint distinction does not affect the result because a continuous random variable has zero probability at any single point. Common mistake avoided: the event $U_{(2)}\le t$ means at least two observations are at most $t$, not exactly two.

Uniform Samples on Other Intervals

If $X_1,\ldots,X_n$ are iid uniform on $(A,B)$, where $A<B$, standardize each observation by setting $U_i=(X_i-A)/(B-A)$. Each $U_i$ is uniform on $(0,1)$, and this increasing transformation preserves the order. Thus $X_{(k)}=A+(B-A)U_{(k)}$. The standardized order statistic has the beta distribution above; the original order statistic is a scaled and shifted version of it.

For any constants $c$ and $d$, $E[c+dV]=c+dE[V]$ and $\operatorname{Var}(c+dV)=d^2\operatorname{Var}(V)$. Consequently,

E[X_{(k)}]=A+(B-A)\frac{k}{n+1}, \qquad \operatorname{Var}(X_{(k)}) =(B-A)^2\frac{k(n-k+1)}{(n+1)^2(n+2)}.

Worked Example: A Middle Order Statistic on a Wider Interval

Five independent measurements are uniform on $(10,30)$. Find the mean and variance of the third order statistic, and check that its mean has a plausible location.

Method. Standardize using $U_i=(X_i-10)/20$. The third standardized order statistic has distribution $\operatorname{Beta}(3,5-3+1)=\operatorname{Beta}(3,3)$. Its mean is $3/6=1/2$, and its variance is

\frac{3\cdot3}{6^2\cdot7} =\frac{9}{252} =\frac{1}{28}.

Transform back to the measurement scale. The mean is $10+20(1/2)=20$. The variance is $20^2(1/28)=400/28=100/7$, or 14.2857 to four decimal places.

The mean is the midpoint of the interval, as expected for the third of five observations in a symmetric sample. The variance is in squared measurement units. Common mistake avoided: multiplying the variance by 20 instead of $20^2$; variance scales with the square of the change in measurement units.

Common Traps and a Reliable Procedure

The beta connection turns a rank question into a parameter-identification and calculation problem. Keep the rank and sample size explicit before substituting values.

Off-by-one parameter: the parameters are $k$ and $n-k+1$, not $k$ and $n-k$. For $n=5,k=2$, the correct distribution is $\operatorname{Beta}(2,4)$, not $\operatorname{Beta}(2,3)$. The incorrect parameters would give mean $2/5$, whereas the correct mean is $2/6=1/3$.
Reversing the event: $U_{(k)}\le t$ means at least $k$ observations are at most $t$. It does not mean at most $k$ observations are at most $t$.
Forgetting the scale: when a uniform interval has length $B-A$, the order-statistic variance is multiplied by $(B-A)^2$, not by $B-A$.
Using the continuous result when ties can occur: the derivation assumes a continuous uniform distribution. If the observations are discrete, ties can have positive probability and the density argument does not apply as stated.

Record the sample size and rank.
Write $n$ and $k$, checking that $1\le k\le n$.

Identify the beta parameters.
For a standard uniform sample, use $a=k$ and $b=n-k+1$.

Select the calculation.
Use beta moments for means and variances; use a density integral or a count of observations below a threshold for probabilities.

Check the support and scale.
The standardized rank lies in $(0,1)$. For an interval other than $(0,1)$, transform back and scale the variance by the interval length squared.

This result is aligned with the SOA Exam P (Probability) syllabus because it combines order-statistic rank, continuous densities, beta parameters, and probability calculations. The central check is simple: a rank farther up the sample should have a larger expected value, and every calculated probability must lie between 0 and 1.

Check Your Understanding

Use the beta parameters or the observation-count interpretation as appropriate.

For an iid sample of size 8 from the uniform distribution on $(0,1)$, find the mean and variance of $U_{(5)}$. (Answer: mean $5/9$; variance $2/81$)
Four iid observations are uniform on $(0,1)$. Find $P(U_{(1)}>0.25)$. (Answer: $0.3164$, to four decimal places)
Six iid observations are uniform on $(0,1)$. Find $P(U_{(3)}\le0.5)$. (Answer: $0.6563$, to four decimal places)
For an iid sample of size 5 from the uniform distribution on $(0,1)$, find the mean and variance of $U_{(4)}$. (Answer: mean $2/3$; variance $2/63$)
Three iid measurements are uniform on $(2,14)$. Find the mean and variance of the second order statistic. (Answer: mean $8$; variance $7.2$)

What You'll Learn