Joint Laws of Consecutive Order Statistics

Why Consecutive Order Statistics Have a Distinct Joint Law

For an IID sample, \(X_{(k)}\) is the \(k\)th smallest observation and \(X_{(k+1)}\) is the next one. Their joint behavior is governed by a simple but important constraint: the first value must be smaller than the second, and there are no sample observations between them. The locations of all the other observations are therefore determined by how many fall below the first value and above the second.

This tutorial derives that joint density and uses it to calculate probabilities that the two consecutive order statistics fall in specified intervals. The sample is assumed to be IID from a continuous distribution with density \(f\) and cumulative distribution function \(F\). Continuity here means absolute continuity with respect to Lebesgue measure, so ties have probability zero.

Definition: For a sample \(X_1,\ldots,X_n\), write \(X_{(1)}\leq\cdots\leq X_{(n)}\) for the observations arranged in increasing order. The pair \(X_{(k)},X_{(k+1)}\), where \(1\leq k\leq n-1\), consists of consecutive order statistics: exactly \(k-1\) observations lie below \(X_{(k)}\), and exactly \(n-k-1\) lie above \(X_{(k+1)}\), with probability one.

The Joint Density

Suppose the two order statistics take values near \(x\) and \(y\), with \(x<y\). One observation must be near \(x\), another must be near \(y\), and among the remaining \(n-2\) observations, exactly \(k-1\) must fall below \(x\). The rest must fall above \(y\). No observation can fall between \(x\) and \(y\), because these order statistics are consecutive.

Theorem: Let \(X_1,\ldots,X_n\) be IID with absolutely continuous distribution function \(F\) and density \(f\). For \(1\leq k\leq n-1\), the joint density of \((X_{(k)},X_{(k+1)})\) is

f_{X_{(k)},X_{(k+1)}}(x,y) = \frac{n!}{(k-1)!(n-k-1)!} [F(x)]^{k-1} f(x) f(y) [1-F(y)]^{n-k-1}, \qquad x<y.

It is zero for \(x\geq y\). Values outside the support of the sample distribution also have density zero through the factors \(f(x)\) and \(f(y)\).

The coefficient counts the assignments of sample labels to the observations near \(x\), near \(y\), below \(x\), and above \(y\). The powers of \(F(x)\) and \(1-F(y)\) account for the probabilities of the lower and upper groups. The following argument establishes the formula directly, including its support.

Proof: Since the sample has a density, the probability of any tie is zero: for each pair \(i\ne j\), \(P(X_i=X_j)=0\), and there are only finitely many pairs. Thus, with probability one, the indices of \(X_{(k)}\) and \(X_{(k+1)}\) are unique.

Take any Borel set \(D\) contained in \(\{(x,y):x<y\}\). Choose an ordered pair of distinct sample indices for the observations at \(x\) and \(y\). There are \(n(n-1)\) choices. Among the other \(n-2\) indices, choose which \(k-1\) observations are below \(x\); the remaining \(n-k-1\) must be above \(y\). There are \(\binom{n-2}{k-1}\) such choices.

For each fixed assignment of indices, independence gives the probability contribution

\int_D F(x)^{k-1} f(x) f(y) [1-F(y)]^{n-k-1}\,dx\,dy.

Indeed, each designated lower observation has probability \(F(x)\) of being below \(x\), and each designated upper observation has probability \(1-F(y)\) of being above \(y\). The two observations assigned to the order statistics contribute the density factors \(f(x)\) and \(f(y)\). Integrating over \(D\) restricts their values to the specified region.

The assignments are disjoint except on the probability-zero tie event, and together they describe exactly the event \((X_{(k)},X_{(k+1)})\in D\). Their total coefficient is

n(n-1)\binom{n-2}{k-1} = \frac{n!}{(k-1)!(n-k-1)!}.

Therefore the probability of every such set \(D\) is the integral of the stated density over \(D\). The pair cannot have \(x\geq y\), which proves the support claim as well. \(\square\)

Turning Interval Conditions into Integrals

If \(A\) and \(B\) are intervals, the probability that \(X_{(k)}\in A\) and \(X_{(k+1)}\in B\) is the integral of the joint density over \(A\times B\), restricted to \(x<y\). If every value in \(A\) is below every value in \(B\), the restriction is automatic. If the intervals overlap, the part with \(x\geq y\) is impossible and must be excluded. A convenient way to do that is to split the integral at the points where the limits change.

For continuous distributions, whether interval endpoints are included does not change the probability: any fixed endpoint has probability zero. In calculations below, closed intervals are used for readability.

Worked Example: Uniform Sample and Separated Intervals

Let \(X_1,\ldots,X_5\) be IID Uniform\((0,1)\). Find \(P(0.2\leq X_{(2)}\leq0.4,\ 0.6\leq X_{(3)}\leq0.8)\).

Here \(n=5\), \(k=2\), \(F(t)=t\) and \(f(t)=1\) on \((0,1)\). The coefficient is \[ \frac{5!}{1!\,2!}=60, \] so for \(0<x<y<1\), \[ f_{X_{(2)},X_{(3)}}(x,y)=60x(1-y)^2. \] Every \(x\) in the first interval is smaller than every \(y\) in the second, so the whole rectangle is in the support. Integrate:

\begin{aligned} P &=60\int_{0.2}^{0.4}\int_{0.6}^{0.8}x(1-y)^2\,dy\,dx\\ &=60\left(\frac{0.4^2-0.2^2}{2}\right) \left(\frac{(1-0.6)^3-(1-0.8)^3}{3}\right)\\ &=60(0.06)\left(\frac{0.064-0.008}{3}\right)\\ &=60(0.06)\left(\frac{7}{375}\right) =\frac{42}{625}=0.0672. \end{aligned}

As a check on the integral, the first factor is \(\int_{0.2}^{0.4}x\,dx=0.06\), and the second is \(\int_{0.6}^{0.8}(1-y)^2\,dy=7/375\). The probability is therefore about \(6.72\%\).

Worked Example: Overlapping Intervals Require a Split

Let \(X_1,X_2,X_3\) be IID Uniform\((0,1)\). Find \(P(0.2\leq X_{(1)}\leq0.7,\ 0.4\leq X_{(2)}\leq0.8)\).

For \(n=3\) and \(k=1\), the joint density is \(6(1-y)\) on \(0<x<y<1\). The intervals overlap, so the lower limit for \(x\) depends on \(y\). For \(0.4\leq y\leq0.7\), \(x\) ranges from \(0.2\) to \(y\). For \(0.7<y\leq0.8\), it ranges from \(0.2\) to \(0.7\). Thus

\begin{aligned} P &=6\int_{0.4}^{0.7}\int_{0.2}^{y}(1-y)\,dx\,dy +6\int_{0.7}^{0.8}\int_{0.2}^{0.7}(1-y)\,dx\,dy\\ &=6\int_{0.4}^{0.7}(1-y)(y-0.2)\,dy +3\int_{0.7}^{0.8}(1-y)\,dy. \end{aligned}

The first integrand expands to \(-y^2+1.2y-0.2\), whose antiderivative is \(-y^3/3+0.6y^2-0.2y\). Its integral from \(0.4\) to \(0.7\) is \(0.045\). The second integral before its factor of \(3\) is \[ \int_{0.7}^{0.8}(1-y)\,dy =\left[y-\frac{y^2}{2}\right]_{0.7}^{0.8} =0.025. \] Therefore \[ P=6(0.045)+3(0.025)=0.27+0.075=0.345. \] The split is essential: integrating over the entire rectangle would include points with \(x\geq y\), which cannot represent the minimum and second-smallest observation in that order.

Worked Example: Consecutive Statistics from an Exponential Sample

Let \(X_1,X_2,X_3\) be IID Exponential\((1)\), so \(f(t)=e^{-t}\) and \(F(t)=1-e^{-t}\) for \(t\geq0\). Find \(P(0\leq X_{(2)}\leq\log 2,\ \log 2\leq X_{(3)}\leq\log 4)\).

For \(k=2\), the joint density is \[ 6F(x)f(x)f(y)=6(1-e^{-x})e^{-x}e^{-y}, \qquad 0\leq x<y. \] The specified intervals are ordered, so \(x<y\) throughout except possibly at the shared boundary, which has probability zero. The integral factors:

\begin{aligned} P &=6\left(\int_0^{\log 2}(1-e^{-x})e^{-x}\,dx\right) \left(\int_{\log 2}^{\log 4}e^{-y}\,dy\right)\\ &=6\left(\frac18\right)\left(\frac12-\frac14\right) =\frac{3}{16}. \end{aligned}

For the first integral, an antiderivative is \(-e^{-x}+\tfrac12e^{-2x}\), giving \(1/8\) at the stated limits. The second equals \(e^{-\log 2}-e^{-\log 4}=1/2-1/4=1/4\). Thus the probability is \(3/16=0.1875\).

Interpretation and a Common Pitfall

The formula makes the role of adjacency visible. There are \(k-1\) observations below \(x\), \(n-k-1\) above \(y\), and none between \(x\) and \(y\). This is why the density has powers of \(F(x)\) and \(1-F(y)\), but no additional factor for observations in the gap. The factorial coefficient counts all label assignments consistent with those ranks.

A common mistake is to use the joint density for two selected order statistics while forgetting that the ranks here are consecutive. For order statistics with ranks farther apart, observations may lie between the selected values, and the density must account for them. For consecutive ranks, inserting a factor for observations in \((x,y)\) would count configurations that contradict the event: \(X_{(k+1)}=y\) would no longer be the next observation after \(x\).

Another tempting error is to multiply the marginal densities of \(X_{(k)}\) and \(X_{(k+1)}\), as though they were independent. The support restriction \(x<y\) already shows why that cannot be right: the joint density vanishes on the entire region \(x\geq y\). In interval calculations, first identify the part of the rectangle allowed by this ordering, then integrate the joint density over precisely that region.

Key takeaway: For IID continuous observations, the joint density of consecutive order statistics is obtained by assigning \(k-1\) observations below the lower value, none in the gap, and \(n-k-1\) above the higher value. Interval probabilities are integrals of this density over the part of the specified rectangle satisfying \(x<y\).

Check Your Understanding

Use the support condition and the joint density to answer these questions.

For an IID continuous sample of size \(n\), how many observations must lie above \(X_{(k+1)}\)?
Write the joint density of \(X_{(3)}\) and \(X_{(4)}\) for an IID sample of size \(7\) with density \(f\) and distribution function \(F\).
Why must a rectangle of interval constraints be restricted to \(x<y\) when calculating a probability for \((X_{(k)},X_{(k+1)})\)?
For \(X_{(1)}\) and \(X_{(2)}\) from a Uniform\((0,1)\) sample of size \(4\), what is the joint-density coefficient?
When the intervals for the two order statistics overlap, what feature determines where the double integral should be split?

Joint Laws of Consecutive Order Statistics

What You'll Learn