Why Sample Extremes Have Simple Laws
The smallest and largest observations summarize the ends of a sample. Their distributions are useful whenever a question concerns whether all observations lie below a threshold, or whether at least one observation crosses it. Independence turns these events into powers of a single-observation probability, yielding direct formulas for the distributions of the sample minimum and maximum.
Let \(X_1,\ldots,X_n\) be independent and identically distributed random variables with common CDF \(F\). Write the ordered observations as \(X_{(1)}\leq \cdots\leq X_{(n)}\), so the sample minimum is \(X_{(1)}\) and the sample maximum is \(X_{(n)}\). The CDF formulas below require only independence and a common distribution. The density formulas additionally require that the parent law be absolutely continuous.
The CDFs From Independence
For the maximum to be at most \(x\), every observation must be at most \(x\). For the minimum to exceed \(x\), every observation must exceed \(x\). Those two event descriptions are the key: independence lets the probability of each intersection factor into a product.
To prove the CDF formulas, first observe that $$ \{X_{(n)}\leq x\}=\bigcap_{i=1}^n\{X_i\leq x\}. $$ Independence and the common distribution therefore give $$ P(X_{(n)}\leq x)=\prod_{i=1}^nP(X_i\leq x)=F(x)^n. $$ Similarly, $$ \{X_{(1)}>x\}=\bigcap_{i=1}^n\{X_i>x\}, \qquad P(X_{(1)}>x)=[1-F(x)]^n. $$ Taking the complement yields \(P(X_{(1)}\leq x)=1-[1-F(x)]^n\). These event identities and probabilities hold whether or not the parent distribution is continuous.
Now assume the common law is absolutely continuous with density \(f\), so \(F\) is absolutely continuous and \(F'(x)=f(x)\) almost everywhere. The chain rule for absolutely continuous functions gives, almost everywhere, $$ \frac{d}{dx}F(x)^n=nF(x)^{n-1}f(x), \qquad \frac{d}{dx}\{1-[1-F(x)]^n\} =n[1-F(x)]^{n-1}f(x). $$ These derivatives are the respective densities. The chain rule applies because the outer power functions have bounded derivatives on the range \([0,1]\), and composition with the absolutely continuous \(F\) remains absolutely continuous. Thus the CDFs are integrals of the displayed derivatives, with no unaccounted point masses. \(\square\)
The minimum density emphasizes the parent distribution’s upper-tail probability \(1-F(x)\): all the other \(n-1\) observations must be above a candidate minimum value. The maximum density instead uses \(F(x)\): the other observations must be below a candidate maximum value. The factor \(n\) accounts for which of the \(n\) observations occupies the extreme position.
Worked Examples
Worked Example: Minimum and Maximum of a Uniform Sample
Let \(X_1,\ldots,X_4\) be IID Uniform\((0,1)\). The parent CDF and density on the support are \(F(x)=x\) and \(f(x)=1\) for \(0<x<1\). Substituting into the CDF formulas gives $$ F_{X_{(1)}}(x)=1-(1-x)^4,\qquad F_{X_{(4)}}(x)=x^4,\qquad 0<x<1. $$
Differentiating, the densities on \(0<x<1\) are $$ f_{X_{(1)}}(x)=4(1-x)^3,\qquad f_{X_{(4)}}(x)=4x^3. $$ Each density is zero outside \((0,1)\). As a check, their integrals over the support are $$ \int_0^1 4(1-x)^3\,dx=[-(1-x)^4]_0^1=1,\qquad \int_0^1 4x^3\,dx=[x^4]_0^1=1. $$
For example, the probability that the maximum is at most \(0.8\) is \(0.8^4=0.4096\). The probability that the minimum exceeds \(0.2\) is \((1-0.2)^4=0.8^4=0.4096\). These probabilities are equal here because the thresholds are complementary in the symmetric unit interval; they come from different events.
Worked Example: Extremes of an Exponential Sample
Suppose \(X_1,X_2,X_3\) are IID exponential with rate \(\lambda=2\), so for \(x\geq0\), $$ F(x)=1-e^{-2x},\qquad f(x)=2e^{-2x}. $$ For \(x<0\), the parent CDF is zero. For \(x\geq0\), the minimum CDF is $$ F_{X_{(1)}}(x)=1-[e^{-2x}]^3=1-e^{-6x}. $$ Its density is \(6e^{-6x}\) for \(x\geq0\), which is the exponential density with rate \(6\).
For the maximum, the CDF and density on \(x\geq0\) are $$ F_{X_{(3)}}(x)=(1-e^{-2x})^3, $$ $$ f_{X_{(3)}}(x)=3(1-e^{-2x})^2(2e^{-2x}) =6e^{-2x}(1-e^{-2x})^2. $$ The maximum density integrates to one: with \(u=1-e^{-2x}\), \(du=2e^{-2x}dx\), and \(u\) increases from 0 to 1 as \(x\) increases from 0 to infinity, $$ \int_0^\infty 6e^{-2x}(1-e^{-2x})^2\,dx =3\int_0^1u^2\,du=1. $$
At \(x=\tfrac12\), the probability that all three observations are at most \(\tfrac12\) is $$ P(X_{(3)}\leq\tfrac12)=(1-e^{-1})^3. $$ The probability that at least one exceeds \(\tfrac12\) is the complement, \(1-(1-e^{-1})^3\). This illustrates why the maximum CDF describes an “all observations are below” event, not an “at least one is above” event.
Worked Example: Maximum of a Normal Sample
Let \(X_1,X_2,X_3\) be IID \(N(\mu,\sigma^2)\), where \(\sigma>0\). Write \(\Phi\) and \(\phi\) for the standard normal CDF and density. The parent CDF and density are $$ F(x)=\Phi\!\left(\frac{x-\mu}{\sigma}\right),\qquad f(x)=\frac{1}{\sigma}\phi\!\left(\frac{x-\mu}{\sigma}\right). $$ Consequently, for every real \(x\), $$ F_{X_{(3)}}(x)=\left[\Phi\!\left(\frac{x-\mu}{\sigma}\right)\right]^3, $$ $$ f_{X_{(3)}}(x)=\frac{3}{\sigma} \left[\Phi\!\left(\frac{x-\mu}{\sigma}\right)\right]^2 \phi\!\left(\frac{x-\mu}{\sigma}\right). $$ For comparison, the minimum has CDF \(1-[1-\Phi((x-\mu)/\sigma)]^3\) and density $$ f_{X_{(1)}}(x)=\frac{3}{\sigma} \left[1-\Phi\!\left(\frac{x-\mu}{\sigma}\right)\right]^2 \phi\!\left(\frac{x-\mu}{\sigma}\right). $$
To evaluate a probability, standardize the threshold. At \(x=\mu+\sigma\), the standardized value is \((x-\mu)/\sigma=1\), so $$ P(X_{(3)}\leq\mu+\sigma)=\Phi(1)^3\approx(0.8413)^3\approx0.595. $$ Thus the chance that at least one observation exceeds \(\mu+\sigma\) is \(1-\Phi(1)^3\approx0.405\). The factor \(1/\sigma\) in each density is required by the change of scale from the standard normal variable.
Conditions, Interpretation, and a Common Pitfall
The CDF identities rely on the sample being independent and identically distributed. If the observations are dependent, the probability of an intersection generally does not factor into a power. If they are independent but have different CDFs, the product is instead \(\prod_{i=1}^nF_i(x)\) for the maximum, and \(1-\prod_{i=1}^n[1-F_i(x)]\) for the minimum. The density formulas stated above specifically assume a common absolutely continuous parent law; they should not be used as complete descriptions when the parent law has atoms.
For instance, if the parent distribution puts positive probability at a point \(a\), the sample minimum or maximum may also have positive probability at \(a\). Differentiating a CDF captures its continuous density component, not a jump in the CDF. In that setting the CDF event formulas remain correct, but the distribution must also account for its jumps.
Common mistake: reversing “all” and “at least one.” A tempting but incorrect argument says that the maximum is at most \(x\) if at least one observation is at most \(x\), leading to \(1-[1-F(x)]^n\). That event actually describes the minimum being at most \(x\). For the maximum to be at most \(x\), all observations must be at most \(x\), so independence gives \(F(x)^n\). Conversely, the minimum is above \(x\) only when all observations are above \(x\), giving the complement formula \(1-[1-F(x)]^n\) for its CDF.
Check Your Understanding
Use the event descriptions as well as the formulas when you work through these questions.
- For IID observations with CDF \(F\), explain in words why the maximum CDF is \(F(x)^n\).
- If the parent law is continuous with density \(f\), write the densities of the sample minimum and maximum.
- For \(n=2\) IID Uniform\((0,1)\) observations, find the CDF and density of the sample minimum for \(0<x<1\).
- For an IID exponential sample with rate \(\lambda\), what is the distribution of the sample minimum? State the rate and justify it from the CDF.
- Why can the CDF formulas remain valid for a discrete parent distribution even though the density formulas may not describe the full laws?