Why Two Order Statistics Need a Joint Density
A single order statistic describes one rank in a sample: for example, the second-smallest observation. A pair of order statistics describes two ranks at once. Their joint density must account not only for the observations at the two selected values, but also for how many sample observations lie below, between, and above those values.
This tutorial derives that density for an iid continuous sample. The derivation is a direct application of counting and the parent distribution’s cumulative distribution function. The central task is to match each rank to the correct region; using the wrong probability for the observations between the selected values changes the answer.
The Joint Density Formula
To have \(X_{(r)}\) near \(x\) and \(X_{(s)}\) near \(y\), where \(x<y\), the sample must have \(r-1\) observations below \(x\), \(s-r-1\) observations between \(x\) and \(y\), and \(n-s\) observations above \(y\). One observation lies near each selected value. The probabilities of the three larger regions are, respectively, \(F(x)\), \(F(y)-F(x)\), and \(1-F(y)\).
The density is zero when \(x\geq y\), or when either selected value lies outside the support of the parent distribution. A factor with exponent zero is interpreted as 1. The coefficient counts ways to assign the \(n\) labelled observations to the required groups: one near \(x\), one near \(y\), and the specified numbers in each of the three remaining regions. The factors \(f(x)\) and \(f(y)\) account for the two observations falling in small intervals around those values.
Why the Formula Holds
The formula can be justified by examining two small intervals, one around \(x\) and one around \(y\). For a continuous density, the probability that a particular observation falls in \((x,x+\Delta x)\) is approximately \(f(x)\Delta x\), and the probability it falls in \((y,y+\Delta y)\) is approximately \(f(y)\Delta y\). Meanwhile, the probabilities of falling below \(x\), between \(x\) and \(y\), or above \(y\) are given by the corresponding CDF differences.
Choose which labelled observations fill the groups of sizes \(r-1\), \(1\), \(s-r-1\), \(1\), and \(n-s\). The number of allocations is \(n!/[(r-1)!(s-r-1)!(n-s)!]\).
For one allocation, the probabilities for the observations below \(x\), between \(x\) and \(y\), and above \(y\) contribute \([F(x)]^{r-1}[F(y)-F(x)]^{s-r-1}[1-F(y)]^{n-s}\).
The two interval probabilities contribute approximately \(f(x)\Delta x f(y)\Delta y\). Divide the resulting joint probability by \(\Delta x\Delta y\), then let both interval widths approach zero.
More formally, the probability that the two selected order statistics fall in their respective small intervals is the displayed density expression multiplied by \(\Delta x\Delta y\), plus a remainder negligible relative to \(\Delta x\Delta y\). Dividing by the area of the small rectangle and taking the limit gives the joint density. Extra observations in either shrinking interval have probabilities of smaller order and do not affect this limit. The argument uses independence to multiply region probabilities and continuity to rule out ties.
The middle factor is essential: exactly \(s-r-1\) observations must fall between the selected ranks. When \(s=r+1\), there are no ranks between them, so that factor is 1. Likewise, if \(r=1\), there are no observations below \(x\); if \(s=n\), there are none above \(y\).
Worked Applications
Worked Example: Two Ranks From a Uniform Sample
Let \(X_1,\ldots,X_5\) be iid uniform on \((0,1)\). Find the joint density of \(X_{(2)}\) and \(X_{(4)}\) at \(x=0.2\), \(y=0.7\). Give the density to four decimal places.
Method. Here \(n=5\), \(r=2\), and \(s=4\). Since the parent distribution is uniform on \((0,1)\), \(F(t)=t\) and \(f(t)=1\) for \(0<t<1\). The rank condition holds because \(0.2<0.7\).
There is one observation below \(x\), one between \(x\) and \(y\), and one above \(y\). Their region probabilities are \(F(x)=0.2\), \(F(y)-F(x)=0.7-0.2=0.5\), and \(1-F(y)=1-0.7=0.3\). The coefficient is \(5!/(1!1!1!)=120\), and the density contributions at both selected values are \(f(x)f(y)=1\cdot1=1\).
Substitute each value:
The joint density is \(3.6000\) per squared unit. Check: the three region probabilities multiply to \(0.03\), and the coefficient accounts for all assignments to the three remaining positions. This is a density, not the probability of the pair taking exactly these two values; a continuous pair has probability zero at a single point. The common mistake avoided here is treating the density value as a point probability.
Worked Example: Exponential Sample With the First and Last Ranks
Let \(X_1,X_2,X_3\) be iid exponential random variables with rate 1 per time unit, so \(F(t)=1-e^{-t}\) and \(f(t)=e^{-t}\) for \(t\geq0\). Find the joint density of \(X_{(1)}\) and \(X_{(3)}\) at \(x=\ln 2\), \(y=\ln 4\).
Method. Here \(n=3\), \(r=1\), and \(s=3\). The values satisfy \(0<\ln 2<\ln 4\). The specified times give \(e^{-x}=1/2\) and \(e^{-y}=1/4\), so \(F(x)=1/2\), \(F(y)=3/4\), and \(F(y)-F(x)=1/4\). Also, \(f(x)=1/2\) and \(f(y)=1/4\).
The coefficient is \(3!/(0!1!0!)=6\). There are no observations below the minimum or above the maximum, while the remaining observation must lie between them. Therefore,
The joint density is \(0.1875\) per squared time unit. Check: the middle-region probability is \(3/4-1/2=1/4\), and the two density values multiply to \(1/8\); thus \(6(1/4)(1/8)=0.1875\). The calculation uses the exponential rate specified in the problem; confusing a rate with a mean would produce incorrect CDF and density values.
Worked Example: Ranks With Two Observations Between Them
Suppose \(X_1,\ldots,X_6\) are iid uniform on \((0,10)\). Find the joint density of \(X_{(2)}\) and \(X_{(5)}\) at \(x=2\), \(y=7\).
Method. Here \(n=6\), \(r=2\), \(s=5\), so the required counts are one below \(x\), two between \(x\) and \(y\), and one above \(y\). For a uniform distribution on \((0,10)\), \(F(t)=t/10\) and \(f(t)=1/10\) within the interval.
The relevant values are \(F(2)=0.2\), \(F(7)-F(2)=0.7-0.2=0.5\), and \(1-F(7)=0.3\). Also, \(f(2)f(7)=0.1\cdot0.1=0.01\). The coefficient is \(6!/(1!2!1!)=720/2=360\).
The joint density is \(0.0540\) per squared unit. Check: \((0.5)^2=0.25\), so the product before multiplying by 360 is \(0.2(0.25)(0.3)(0.01)=0.00015\). This example highlights that the exponent for the between-values probability is \(s-r-1=2\), not \(s-r=3\). The two selected observations themselves are already represented by \(f(2)f(7)\).
Common Exam Trap: Using the Wrong Middle Probability
The interval between \(x\) and \(y\) has probability \(F(y)-F(x)\), not \(F(y)\). The latter is the probability of being below \(y\), which includes observations below \(x\) as well as those in the middle. This distinction matters whenever \(s-r-1>0\).
For instance, in the first worked example, incorrectly using \(F(y)=0.7\) for the one observation between the selected ranks would give \(120(0.2)(0.7)(0.3)=5.04\), instead of the correct \(120(0.2)(0.5)(0.3)=3.60\). The correction is to use \(P(x<X_i<y)=F(y)-F(x)=0.7-0.2=0.5\).
A second check is the support: the joint density must be zero when \(x\geq y\), because the \(r\)th order statistic cannot exceed the \(s\)th when \(r<s\). Do not apply the formula as though the two order statistics were independent. Their ranks impose shared constraints on the sample, and the three region counts encode those constraints.
Check Your Understanding
For each problem, identify the region counts before substituting into the joint-density formula.
- Let four observations be iid uniform on \((0,1)\). Find the joint density of \(X_{(1)}\) and \(X_{(2)}\) at \(x=0.25\), \(y=0.5\). (Answer: \(3.0000\))
- Let five observations be iid uniform on \((0,2)\). Find the joint density of \(X_{(2)}\) and \(X_{(5)}\) at \(x=0.5\), \(y=1.5\). (Answer: \(0.9375\) per squared unit)
- Let four observations be iid exponential with rate 2 per time unit. Find the joint density of \(X_{(1)}\) and \(X_{(3)}\) at \(x=(\ln 2)/2\), \(y=(\ln 4)/2\). (Answer: \(0.7500\) per squared time unit)
- Let six observations be iid uniform on \((0,1)\). Find the joint density of \(X_{(3)}\) and \(X_{(5)}\) at \(x=0.4\), \(y=0.8\). (Answer: \(4.6080\))
- For an iid continuous sample of size 7, suppose a parent distribution satisfies \(F(a)=0.3\), \(F(b)=0.8\), \(f(a)=0.4\), and \(f(b)=0.2\), where \(a<b\). Find the joint density of \(X_{(3)}\) and \(X_{(6)}\) at \((a,b)\). (Answer: \(0.4536\) per squared unit)