Expected Order Statistics Through Quantiles

Why Quantiles Give Expected Ranks

An order statistic is a sample value selected by its rank: \(X_{(1)}\) is the smallest observation, \(X_{(n)}\) is the largest, and \(X_{(k)}\) is the \(k\)th smallest. Its expected value depends both on the parent distribution and on how the rank \(k\) is positioned within the sample. Quantiles provide a direct way to separate those two ingredients: the parent distribution determines the quantile function, while the sample size and rank determine a beta density.

The resulting formula is useful when a parent quantile is easier to work with than the order statistic’s density. It also applies to distributions with different supports and shapes. We will derive it and then use it for uniform, exponential, and Pareto samples.

The Quantile Function and the Rank Distribution

Definition: For a distribution function \(F\), its generalized quantile function is \(Q(u)=\inf\{x\in\mathbb{R}:F(x)\geq u\}\), for \(0<u<1\). For an iid sample \(X_1,\ldots,X_n\), the order statistic \(X_{(k)}\) is the \(k\)th smallest sample value, with repeated values retained when ties occur.

The generalized inverse convention matters when \(F\) is not strictly increasing: it still specifies a quantile even across flat portions of the distribution function. The key rank fact is that if \(U_1,\ldots,U_n\) are independent Uniform\((0,1)\) variables, then their \(k\)th order statistic has density

f_{U_{(k)}}(u)=\frac{n!}{(k-1)!(n-k)!}u^{k-1}(1-u)^{n-k},\qquad 0<u<1.

This is a beta density with parameters \(k\) and \(n+1-k\). To see why, the event \(U_{(k)}\leq u\) occurs exactly when at least \(k\) of the \(n\) uniforms are no larger than \(u\). The number of such uniforms has a binomial distribution with parameters \(n\) and \(u\), so

P(U_{(k)}\leq u)=\sum_{j=k}^{n}\binom{n}{j}u^j(1-u)^{n-j}.

Differentiating this finite sum gives the density above. In detail, the positive terms from differentiating \(u^j\) cancel the negative terms from differentiating \((1-u)^{n-j}\), except at the boundary between the two sums; the remaining term is \(\frac{n!}{(k-1)!(n-k)!}u^{k-1}(1-u)^{n-k}\). Its integral is one, since the beta integral satisfies \(B(k,n+1-k)=(k-1)!(n-k)!/n!\).

The Quantile Formula

Theorem: Let \(X_1,\ldots,X_n\) be iid with distribution function \(F\) and generalized quantile function \(Q\). If \(E|X_{(k)}|<\infty\), then

E[X_{(k)}] = \frac{n!}{(k-1)!(n-k)!} \int_0^1 Q(u)u^{k-1}(1-u)^{n-k}\,du.

A sufficient condition for this integrability is \(E|X_1|<\infty\).

Here is a derivation that also explains why the formula does not require the parent distribution to have a density. If \(U\) is Uniform\((0,1)\), then \(Q(U)\) has distribution function \(F\). Thus we can construct an iid sample with the parent distribution by taking \(X_i=Q(U_i)\) for independent uniform variables \(U_i\). Because \(Q\) is nondecreasing, sorting the \(X_i\)'s in this construction gives \(X_{(k)}=Q(U_{(k)})\); this remains true if \(Q\) is flat on an interval. Taking expectations and using the density of \(U_{(k)}\) yields the formula.

The integrability condition ensures that this expectation can be evaluated as an ordinary signed integral. To verify the stated sufficient condition, in the construction just given, \(|X_{(k)}|\leq\max_i|X_i|\leq\sum_{i=1}^n|X_i|\). If \(E|X_1|<\infty\), then the sum on the right has finite expectation, so \(E|X_{(k)}|<\infty\) as well.

Equivalently, if \(U_{(k)}\) is viewed as a beta variable, the formula says \(E[X_{(k)}]=E[Q(U_{(k)})]\). The beta weighting is essential: ranks near \(k/(n+1)\) are more likely, but the full range of ranks contributes to the expectation.

Worked Applications

Worked Example: A Uniform Order Statistic

Let \(X_1,\ldots,X_5\) be iid Uniform\((2,8)\). Find \(E[X_{(2)}]\).

The parent quantile is \(Q(u)=2+6u\), because \(F(x)=(x-2)/6\) on \(2\leq x\leq8\). The rank variable \(U_{(2)}\) has beta parameters \(2\) and \(4\), and its density is \(20u(1-u)^3\). Therefore,

E[X_{(2)}] =\int_0^1(2+6u)20u(1-u)^3\,du =2+6E[U_{(2)}].

For a Beta\((a,b)\) variable, the mean is \(a/(a+b)\), as follows from \(B(a+1,b)/B(a,b)=a/(a+b)\). Hence \(E[U_{(2)}]=2/6=1/3\), and

E[X_{(2)}]=2+6\left(\frac13\right)=4.

The result lies in the support \([2,8]\), as it must. In this affine-quantile example, the expectation can be obtained just by transforming the mean rank.

Worked Example: An Exponential Order Statistic

Let \(X_1,\ldots,X_4\) be iid exponential with rate \(\lambda>0\), so \(F(x)=1-e^{-\lambda x}\) for \(x\geq0\). Find \(E[X_{(3)}]\).

Solving \(u=1-e^{-\lambda x}\) for \(x\) gives \(Q(u)=-\ln(1-u)/\lambda\). The rank \(U_{(3)}\) has beta parameters \(3\) and \(2\), so

E[X_{(3)}] =\frac{1}{\lambda B(3,2)} \int_0^1[-\ln(1-u)]u^2(1-u)\,du.

To evaluate the logarithmic integral, define \(B(a,b)=\int_0^1u^{a-1}(1-u)^{b-1}\,du\) and \(\psi(x)=\frac{d}{dx}\ln\Gamma(x)\), where \(\Gamma\) is the gamma function. Differentiating the beta integral with respect to \(b\) gives \[ \frac{\partial}{\partial b}B(a,b) =\int_0^1u^{a-1}(1-u)^{b-1}\ln(1-u)\,du =B(a,b)\{\psi(b)-\psi(a+b)\}. \] The differentiation is valid for \(a,b>0\): on any compact neighborhood of such parameters, the integrand with the additional logarithm is bounded by an integrable function near both endpoints. Consequently, the integral with \(-\ln(1-u)\) is \(B(a,b)\{\psi(a+b)-\psi(b)\}\).

Here \(a=3\), \(b=2\), and the normalization is \(B(3,2)\), so the expectation equals \(\{\psi(5)-\psi(2)\}/\lambda\). The recurrence \(\psi(x+1)=\psi(x)+1/x\), obtained by differentiating \(\Gamma(x+1)=x\Gamma(x)\), gives \(\psi(5)-\psi(2)=1/2+1/3+1/4=13/12\). Thus

E[X_{(3)}]=\frac{13}{12\lambda}.

This is finite for every \(\lambda>0\), consistent with the finite mean of the exponential distribution.

Worked Example: A Pareto Order Statistic and Its Existence Condition

Suppose \(X_1,\ldots,X_3\) are iid Pareto Type I with scale \(x_m>0\) and shape \(\alpha>0\), so \(F(x)=1-(x_m/x)^\alpha\) for \(x\geq x_m\). Find \(E[X_{(2)}]\) when \(\alpha=2\).

Solving \(u=1-(x_m/x)^\alpha\) gives \(Q(u)=x_m(1-u)^{-1/\alpha}\). Since \(U_{(2)}\) has beta parameters \(2\) and \(2\), the quantile formula yields, for any \(\alpha\) for which the integral is finite,

E[X_{(2)}] =x_m\frac{B(2,\,2-1/\alpha)}{B(2,2)}.

The second beta parameter must be positive, so this expectation is finite precisely when \(2-1/\alpha>0\), or \(\alpha>1/2\). For the requested shape \(\alpha=2\), the condition holds, and

E[X_{(2)}] =x_m\frac{B(2,3/2)}{B(2,2)} =x_m\frac{4/15}{1/6} =\frac85x_m.

For the beta values, \(B(2,3/2)=\Gamma(2)\Gamma(3/2)/\Gamma(7/2)=4/15\), while \(B(2,2)=1/6\). The finite result is possible even when the parent mean is infinite: for example, any \(\alpha\) with \(1/2<\alpha\leq1\) gives an infinite parent mean but a finite expected second-smallest value in a sample of three.

Interpreting the Formula and Avoiding a Common Error

The quantile formula separates a rank distribution from a parent distribution. The beta density is concentrated around the typical location of the \(k\)th rank; the quantile function maps each possible rank location to the corresponding value on the parent scale. This makes the formula applicable even when the quantile integral is simpler than integrating the order-statistic density.

A common mistake is to replace the expectation of a transformed rank with the transformation of its expectation: \(E[Q(U_{(k)})]\) is generally not \(Q(E[U_{(k)}])\). The two agree for an affine quantile function, as in the uniform example, but not generally. For the exponential example, \(E[U_{(3)}]=3/5\), so evaluating the quantile at that mean gives \(-\ln(2/5)/\lambda\), approximately \(0.9163/\lambda\). The actual expected order statistic is \(13/(12\lambda)\), approximately \(1.0833/\lambda\). The nonlinear logarithm makes the full beta-weighted integral necessary.

Existence also depends on the combination of the tail of \(Q(u)\) near \(u=1\) and the factor \((1-u)^{n-k}\). In the Pareto example, that factor determines the condition \(\alpha(n-k+1)>1\) for a finite mean of \(X_{(k)}\). Thus an order statistic can have a finite expectation even when the parent distribution does not, because its rank may keep it sufficiently far from the extreme upper tail.

Key takeaway: For an iid sample, \(E[X_{(k)}]\) is the beta-weighted average of the parent quantile \(Q(u)\). Check integrability at the endpoints, and do not replace the integral by \(Q(k/(n+1))\) unless the quantile has a form that makes this valid.

Check Your Understanding

Use the beta-weighted quantile formula to reason through these questions.

What are the beta parameters for \(U_{(k)}\) in a sample of size \(n\), and how do they depend on \(k\)?
For a Uniform\((a,b)\) sample, why does the expected \(k\)th order statistic equal \(a+(b-a)k/(n+1)\)?
For an exponential parent with rate \(\lambda\), write the quantile function and identify the feature that makes \(Q(E[U_{(k)}])\) differ from \(E[Q(U_{(k)})]\).
For a Pareto parent with shape \(\alpha\), what condition makes \(E[X_{(k)}]\) finite in a sample of size \(n\)?
Why can the expected second-smallest value in a sample have a finite mean even when the parent Pareto distribution has an infinite mean?