Expected Minima And Maxima From Tail Integrals

Why Tail Integrals Simplify Sample Extrema

A sample minimum or maximum can have a more complicated distribution than any one observation. Its expectation, however, can often be found directly from a tail integral, without first deriving a density. This is especially useful when the probability that the whole sample lies above or below a threshold has a simple form.

Throughout, let $X_1,\ldots,X_n$ be independent and identically distributed nonnegative random variables, with cdf $F_X(t)=P(X_i\le t)$ and survival function $S_X(t)=P(X_i>t)=1-F_X(t)$. Write $X_{(1)}=\min(X_1,\ldots,X_n)$ and $X_{(n)}=\max(X_1,\ldots,X_n)$. The independence assumption is what allows the probabilities for all sample members to be multiplied. If the observations are dependent, these product formulas generally do not apply.

Key result: For any nonnegative random variable $Z$, its expectation, including the possibility of an infinite expectation, is $$ E[Z]=\int_0^\infty P(Z>t)\,dt. $$ For an independent and identically distributed sample of size $n$, $$ E[X_{(1)}]=\int_0^\infty S_X(t)^n\,dt, \qquad E[X_{(n)}]=\int_0^\infty \left[1-F_X(t)^n\right]\,dt. $$ These expectations are finite when the corresponding integrals are finite.

Why the Tail-Integral Identity Holds

For a fixed nonnegative value $z$, the indicator $1_{\{z>t\}}$ equals 1 for thresholds from 0 up to $z$, and equals 0 above $z$. Therefore its area over all nonnegative thresholds is $z$:

z=\int_0^\infty 1_{\{z>t\}}\,dt.

Now substitute the random variable $Z$ for $z$. The result holds for each realized value of $Z$. Taking expectations and interchanging expectation and integration is valid for this nonnegative integrand (by Tonelli’s theorem), even if the result is infinite:

E[Z] =E\left[\int_0^\infty 1_{\{Z>t\}}\,dt\right] =\int_0^\infty E[1_{\{Z>t\}}]\,dt =\int_0^\infty P(Z>t)\,dt.

To obtain the sample formulas, use the events defining the extrema. The minimum exceeds $t$ exactly when every sample value exceeds $t$. The maximum is at most $t$ exactly when every sample value is at most $t$. Independence then gives

P(X_{(1)}>t)=P(X_1>t,\ldots,X_n>t)=S_X(t)^n, \qquad P(X_{(n)}\le t)=P(X_1\le t,\ldots,X_n\le t)=F_X(t)^n.

Thus $P(X_{(n)}>t)=1-F_X(t)^n$, and substituting these two tail probabilities into the identity gives the stated expectations. The identity itself does not require a continuous distribution; atoms are allowed. The formulas for the sample extrema also hold for any independent sample if the individual cdfs or survival functions are used separately.

Uniform Samples

Suppose each observation is uniform on $[0,b]$, where $b>0$. For $0\le t\le b$, $F_X(t)=t/b$ and $S_X(t)=1-t/b$. Below 0 or above $b$, the corresponding cdf and survival function are 0 or 1 as appropriate. Both the minimum and maximum lie in $[0,b]$, so their tail integrals only need to be evaluated over that interval.

Worked Example: Expected Minimum of Four Uniform Measurements

A hypothetical sensor records four independent measurement errors. Each error $X_i$ is uniformly distributed between 0 and 12 units. Find the expected smallest error. The measurements are independent and identically distributed, and the model is $X_i\sim \operatorname{Uniform}(0,12)$.

Method. Use the survival function of the minimum. For $0\le t\le12$, one measurement has survival function $S_X(t)=1-t/12$. The minimum exceeds $t$ only if all four measurements exceed $t$, so

P(X_{(1)}>t)=\left(1-\frac{t}{12}\right)^4.

The minimum cannot exceed 12, so the expectation is

\begin{aligned} E[X_{(1)}] &=\int_0^{12}\left(1-\frac{t}{12}\right)^4\,dt\\ &=12\int_0^1(1-u)^4\,du \qquad (u=t/12,\;dt=12\,du)\\ &=12\left[-\frac{(1-u)^5}{5}\right]_0^1\\ &=\frac{12}{5}=2.4\text{ units}. \end{aligned}

Check. The result is positive and below the individual mean of 6 units, as a sample minimum should be. The common mistake is to integrate the survival function of one measurement, which would give 6 units, rather than raising it to the fourth power for the minimum of four independent measurements.

Worked Example: Expected Maximum of Four Uniform Measurements

Use the same hypothetical measurement model: four independent observations, each distributed as $\operatorname{Uniform}(0,12)$. Find the expected largest measurement.

Method. A maximum exceeds $t$ if at least one observation exceeds $t$. Equivalently, its cdf is the probability that all four observations are at most $t$:

P(X_{(4)}\le t)=\left(\frac{t}{12}\right)^4, \qquad P(X_{(4)}>t)=1-\left(\frac{t}{12}\right)^4 \quad (0\le t\le12).

Integrate the maximum’s survival probability over its possible range:

\begin{aligned} E[X_{(4)}] &=\int_0^{12}\left[1-\left(\frac{t}{12}\right)^4\right]\,dt\\ &=\left[t-\frac{t^5}{5(12^4)}\right]_0^{12}\\ &=12-\frac{12}{5} =\frac{48}{5}=9.6\text{ units}. \end{aligned}

Check. The maximum’s expected value is above the individual mean of 6 units and below the upper endpoint 12. The trap is to use $(1-t/12)^4$ as the maximum’s survival probability. That is the probability that all four observations exceed $t$, which describes the minimum instead. For the maximum, use one minus the probability that all observations are at most $t$.

Exponential Samples

Let each observation have an exponential distribution with rate $\lambda>0$, measured in inverse time units. Its cdf and survival function, for $t\ge0$, are

F_X(t)=1-e^{-\lambda t}, \qquad S_X(t)=e^{-\lambda t}.

The rate’s units matter: if time is measured in hours, $\lambda$ is per hour, and an expectation obtained by integrating over $t$ is in hours. For the minimum of $n$ independent observations, the survival function is $e^{-n\lambda t}$, so its expectation is $1/(n\lambda)$. For the maximum, expand $1-(1-e^{-\lambda t})^n$, or integrate that expression directly.

Worked Example: Expected Minimum and Maximum of Three Exponential Waiting Times

Three independent service requests have exponential waiting times with rate $\lambda=2$ per hour. Find the expected shortest and longest waiting times. Each waiting time has mean $1/\lambda=1/2$ hour.

Minimum. All three waiting times must exceed $t$ for the minimum to exceed $t$. Therefore

\begin{aligned} E[X_{(1)}] &=\int_0^\infty (e^{-2t})^3\,dt\\ &=\int_0^\infty e^{-6t}\,dt\\ &=\left[-\frac{1}{6}e^{-6t}\right]_0^\infty =\frac{1}{6}\text{ hour}. \end{aligned}

Maximum. The maximum’s survival function is one minus the probability that all three requests have finished by time $t$:

\begin{aligned} P(X_{(3)}>t) &=1-(1-e^{-2t})^3\\ &=3e^{-2t}-3e^{-4t}+e^{-6t}. \end{aligned}

Integrate each term. Since $\int_0^\infty e^{-ct}\,dt=1/c$ for $c>0$,

\begin{aligned} E[X_{(3)}] &=\int_0^\infty\left(3e^{-2t}-3e^{-4t}+e^{-6t}\right)\,dt\\ &=\frac{3}{2}-\frac{3}{4}+\frac{1}{6}\\ &=\frac{18-9+2}{12} =\frac{11}{12} \approx0.9167\text{ hour}. \end{aligned}

The minimum is exactly $1/6$ hour, and the maximum is approximately 0.9167 hour (rounded to four decimal places). The minimum is below the individual mean of 0.5 hour, while the maximum is above it, as expected. A common error is to use the minimum’s survival function $e^{-6t}$ for the maximum; the maximum requires $1-(1-e^{-2t})^3$.

Using the Right Tail

The formulas are short, but the event behind each tail probability must be read carefully. For a minimum, the event “minimum exceeds $t$” requires every observation to exceed $t$. For a maximum, the event “maximum exceeds $t$” requires at least one observation to exceed $t$. The complement of that second event is often the easiest way to calculate it: all observations are at most $t$.

Identify the support.
For nonnegative observations, begin the tail integral at 0. Use the actual upper endpoint if the sample is bounded; otherwise integrate to infinity.

Translate the event.
For a minimum, write $P(X_{(1)}>t)$. For a maximum, write $P(X_{(n)}>t)=1-P(X_{(n)}\le t)$.

Use independence where justified.
For iid observations, the minimum tail is $S_X(t)^n$, and the maximum cdf is $F_X(t)^n$. Do not use these powers if the observations are dependent.

Integrate and check.
Confirm the answer is nonnegative and lies within the sample’s support when that support is bounded. A minimum should not exceed the individual mean, and a maximum should not be below it, when the expectations exist.

These expectation comparisons follow from the pointwise inequalities $X_{(1)}\le X_i\le X_{(n)}$ for every observation $i$. Taking expectations gives $E[X_{(1)}]\le E[X_i]\le E[X_{(n)}]$, provided the expectations are finite. They are useful checks, but not substitutes for identifying and integrating the correct tail.

Key takeaway: For nonnegative variables, expectation is the area under the survival function. Independence turns the minimum’s survival into a product, while the maximum’s survival is one minus the product of the individual cdfs. Correctly identifying those events is the central step.

Check Your Understanding

For each independent sample, use a tail integral to find the requested expected extrema. Include the stated time or measurement units.

Three iid measurements are uniformly distributed on $[0,8]$. Find the expected minimum and maximum. (Answer: minimum $=2$ units; maximum $=6$ units.)
Two iid waiting times are exponential with rate 4 per week. Find the expected minimum and maximum. (Answer: minimum $=1/8$ week; maximum $=3/8$ week.)
Two iid values are uniformly distributed on $[2,14]$. Find the expected minimum and maximum. (Answer: minimum $=6$ units; maximum $=10$ units.)
Two independent waiting times are exponential, with rates 1 per day and 3 per day. Find the expected minimum and maximum. (Answer: minimum $=1/4$ day; maximum $=13/12$ days.)
Three iid waiting times are exponential with rate 0.5 per hour. Find the expected minimum and maximum. (Answer: minimum $=2/3$ hour; maximum $=11/3$ hours.)

What You'll Learn