Joint Density Of All Iid Order Statistics

Why the Joint Density of the Entire Ordered Sample Matters

Sorting a sample changes the way its values can occur together. If $X_1,\ldots,X_n$ are observations, their ordered values are $X_{(1)}\leq X_{(2)}\leq\cdots\leq X_{(n)}$, where $X_{(1)}$ is the sample minimum and $X_{(n)}$ is the sample maximum. The coordinates of this ordered vector cannot appear in an arbitrary order: every possible vector lies in an ordered region.

For an independent, identically distributed continuous sample, the joint density on that region is the product of the original densities multiplied by $n!$. The product describes the density for one particular arrangement of the original observations; the factorial accounts for every original arrangement that sorts to the same ordered vector. This result is useful whenever a probability condition restricts several ranks at once.

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed continuous random variables, each with probability density function $f_X$ on an interval $I$. Write their order statistics as $X_{(1)}<\cdots<X_{(n)}$. Their joint density is $$ f_{X_{(1)},\ldots,X_{(n)}}(x_1,\ldots,x_n) = \begin{cases} n!\displaystyle\prod_{j=1}^{n} f_X(x_j), & x_1<x_2<\cdots<x_n,\quad x_j\in I,\\ 0, & \text{otherwise.} \end{cases} $$ For a continuous sample, ties have probability zero, so using strict inequalities rather than weak inequalities does not change probabilities.

The Ordered Support and the Factorial

The support is the set of values on which the density can be nonzero. Here it is the part of $I^n$ satisfying $x_1<x_2<\cdots<x_n$. The density is zero at points outside this region because a sorted list cannot have, for example, its second value below its first.

The factor $n!$ is not an adjustment to the parent density. It counts the $n!$ ways to assign $n$ distinct observed values to the original labels $X_1,\ldots,X_n$. Each assignment has the same density product because the observations are identically distributed. Independence makes the density of the original sample the product of the individual densities.

Start with the labelled sample.
The joint density of the original independent sample is $\prod_{j=1}^{n} f_X(x_j)$.

Restrict to one ordering.
Consider the region where the original coordinates occur in one specified order. Sorting maps this region to the ordered support. The map only permutes coordinates, so its Jacobian has absolute value $1$.

Count all possible label assignments.
There are $n!$ coordinate orderings. The regions do not overlap except at ties, which have probability zero, and each contributes the same product density.

More formally, let $B$ be any measurable subset of the ordered support. The event that the ordered vector lies in $B$ is the union of the events that the original vector lies in each of the $n!$ corresponding permuted regions. Independence gives the product density in every region; identical distributions make the integrals equal. Therefore,

P\big((X_{(1)},\ldots,X_{(n)})\in B\big) = n!\int_B \prod_{j=1}^{n} f_X(x_j)\,dx_1\cdots dx_n.

This identity gives the stated joint density. It also checks the normalization: the $n!$ ordering regions partition the original sample space except for ties, so the integral of the product density over any one ordered region is $1/n!$. Multiplying by $n!$ makes the integral of the order-statistic density equal to $1$.

Using the Joint Density

For an event described by ranges for the ordered values, integrate the joint density over the part of the ordered support that satisfies the event. When the ranges are disjoint and appear in increasing order, the ordering is automatic throughout the ranges. The integral then separates into one integral for each rank, with the $n!$ factor retained.

Worked Example: Three Uniform Observations in Ordered Bands

Let $X_1,X_2,X_3$ be independent and identically distributed uniform random variables on $(0,10)$. Find the probability that $X_{(1)}$ is between $0$ and $2$, $X_{(2)}$ is between $4$ and $6$, and $X_{(3)}$ is between $8$ and $10$.

Model and method. Each observation has density $f_X(x)=1/10$ on $(0,10)$. The three specified bands are disjoint and increasing, so any choice of one observation from each band is already in the required sorted order. Integrate the full joint density over these three bands.

f_{X_{(1)},X_{(2)},X_{(3)}}(x_1,x_2,x_3) =3!\left(\frac{1}{10}\right)^3 =\frac{6}{1000} =0.006

The density is constant on the specified region. Its volume is the product of the band lengths: $2\cdot2\cdot2=8$. Thus

P(0<X_{(1)}<2,\ 4<X_{(2)}<6,\ 8<X_{(3)}<10) =0.006(8) =0.0480.

Check and common mistake avoided. Each original observation has probability $2/10=0.2$ of falling in any one of these bands. Assigning the three labelled observations to the three bands can be done in $3!=6$ ways, giving $6(0.2)^3=0.0480$, as before. Omitting the factorial would give $0.008$, which counts only one assignment of labels to bands.

Worked Example: Exponential Observations in Separated Ranges

Suppose $X_1,X_2,X_3$ are independent exponential random variables with rate parameter $2$ per unit. Find the probability that $0<X_{(1)}<0.2$, $0.5<X_{(2)}<0.8$, and $X_{(3)}>1$.

Model and method. The density is $f_X(x)=2e^{-2x}$ for $x>0$, and its cumulative distribution function is $F_X(x)=1-e^{-2x}$. The three ranges are ordered and do not overlap. The joint-density integral therefore equals $3!$ times the product of the three parent probabilities for those ranges.

For the first range, the probability is

P(0<X<0.2)=F_X(0.2)-F_X(0) =1-e^{-0.4} \approx 0.3297.

For the second range, it is

P(0.5<X<0.8)=F_X(0.8)-F_X(0.5) =e^{-1}-e^{-1.6} \approx 0.1660.

For the final range, it is

P(X>1)=1-F_X(1)=e^{-2}\approx 0.1353.

Multiplying these probabilities and accounting for the $3!$ assignments gives

\begin{aligned} P(0<X_{(1)}<0.2,\ 0.5<X_{(2)}<0.8,\ X_{(3)}>1) &=6(1-e^{-0.4})(e^{-1}-e^{-1.6})e^{-2}\\ &\approx 6(0.32967995)(0.16598292)(0.13533528)\\ &\approx 0.0444. \end{aligned}

Check and common mistake avoided. The value $0.0444$ is a probability, rounded to four decimal places. Without the factor $6$, the calculation would give about $0.0074$, the probability for just one specified assignment of the three labelled observations to the ranges. The exponential rate is $2$ per unit throughout; replacing $e^{-2x}$ by $e^{-x}$ would change every interval probability.

Worked Example: A Nonuniform Parent Density

Let $X_1,X_2,X_3$ be iid with density $f_X(x)=2x$ for $0<x<1$, and zero otherwise. Find the probability that $X_{(1)}\in(0.1,0.3)$, $X_{(2)}\in(0.4,0.6)$, and $X_{(3)}\in(0.8,1)$.

Model and method. The density is valid because $\int_0^1 2x\,dx=1$. Its cumulative distribution function on $(0,1)$ is $F_X(x)=x^2$. As the bands are disjoint and increasing, integrate the order-statistic density across their product region, using $3!$ times the product of the three band probabilities.

\begin{aligned} P(0.1<X<0.3)&=0.3^2-0.1^2=0.08,\\ P(0.4<X<0.6)&=0.6^2-0.4^2=0.20,\\ P(0.8<X<1)&=1^2-0.8^2=0.36. \end{aligned}

Consequently,

\begin{aligned} P\big(X_{(1)}\in(0.1,0.3),\ X_{(2)}\in(0.4,0.6),\ X_{(3)}\in(0.8,1)\big) &=6(0.08)(0.20)(0.36)\\ &=6(0.00576)\\ &=0.03456. \end{aligned}

Check and common mistake avoided. The interval probabilities sum to $0.08+0.20+0.36=0.64$, which is less than $1$, as required for three disjoint portions of the parent distribution. Using interval widths alone would incorrectly substitute $0.2$, $0.2$, and $0.2$: the parent density $2x$ is not constant, so widths are not probabilities. The correct result is $0.03456$.

Assumptions and Exam Traps

The formula requires both independence and identical distributions. If observations are independent but have different densities, the density product for a particular sorted vector depends on which original observation occupies each rank; in general, it is not $n!$ times one common product. The assumption of continuity also matters: ties have probability zero under a continuous distribution. For discrete observations, ties can have positive probability, and the strict ordered-support formula does not describe their joint distribution.

A frequent error is to include the factorial twice. Once the joint density $n!\prod_j f_X(x_j)$ is being integrated over the ordered support, the factor $n!$ is already present. Do not multiply the resulting integral by another $n!$. Conversely, if a calculation starts from probabilities for the original labelled observations in specified disjoint bands, multiply by the number of assignments only when every assignment is a distinct way to realize the event.

A second trap is to forget that the joint density applies only on ordered points. For example, a proposed vector with $x_2<x_1$ is outside the support, even if each coordinate separately lies in the parent distribution's support. The density of the full ordered vector is a joint model, not a product of unrestricted marginal densities for its ranks.

Key takeaway: For an iid continuous sample, the joint density of the complete ordered vector equals $n!$ times the product of the parent densities on the ordered support, and equals zero elsewhere. The factorial accounts for the distinct labelled-sample orderings that produce the same sorted values.

Check Your Understanding

For each problem, use the full ordered-sample density or an equivalent count of labelled assignments. Answers are rounded to four decimal places where appropriate.

Let two iid observations be uniform on $(0,5)$. What is the joint density of $(X_{(1)},X_{(2)})$, and what is the probability that $X_{(1)}\in(0,1)$ and $X_{(2)}\in(2,4)$? (Answer: density $2/25$ on $0<x_1<x_2<5$; probability $0.1600$.)
Three iid observations are uniform on $(0,1)$. Find the probability that $X_{(1)}\in(0.1,0.2)$, $X_{(2)}\in(0.5,0.7)$, and $X_{(3)}\in(0.8,0.9)$. (Answer: $0.0120$.)
Let $X_1,X_2$ be iid exponential random variables with rate $3$ per unit. Evaluate the joint density of $(X_{(1)},X_{(2)})$ at $(0.2,0.6)$. (Answer: $18e^{-2.4}\approx 1.6329$.)
Three iid observations have density $f_X(x)=3x^2$ on $(0,1)$. Find the probability that $X_{(1)}\in(0,0.5)$, $X_{(2)}\in(0.6,0.8)$, and $X_{(3)}\in(0.9,1)$. (Answer: $0.060162$.)
Four iid observations are uniform on $(0,8)$. Find the probability that the four ordered observations fall, respectively, in $(0,2)$, $(2,4)$, $(4,6)$, and $(6,8)$. (Answer: $3/32=0.09375$.)

What You'll Learn