Gumbel Limits for Exponential-Tail Models

Why Tail Behavior Determines the Maximum

For a sample of independent observations, the largest value is governed by the probability of seeing an observation far out in the tail. If that tail decreases exponentially, the maximum typically grows on a logarithmic scale with sample size. The right centering and scaling turn that growing maximum into a nondegenerate limit with a simple distribution.

Here $X_1,\ldots,X_n$ are independent and identically distributed with cumulative distribution function $F$, and $M_n=\max(X_1,\ldots,X_n)$. The target limit is the standard Gumbel distribution, whose cumulative distribution function at $x\in\mathbb{R}$ is $\exp(-e^{-x})$. The central calculation is to choose $b_n$ and $a_n>0$ so that the probability an individual exceeds $b_n+a_nx$ is approximately $e^{-x}/n$.

Definition: A sequence of normalized maxima has a standard Gumbel limit if, for every real $x$, $$ P\left(\frac{M_n-b_n}{a_n}\le x\right)\longrightarrow \exp(-e^{-x}), $$ where $b_n$ is a centering sequence and $a_n>0$ is a scaling sequence.

A Tail Condition and Its Normalizations

A useful class of exponential tails has a polynomial factor multiplying the exponential decay. Suppose the common survival function $\overline F(t)=1-F(t)$ satisfies $$ \overline F(t)\sim C t^r e^{-\lambda t}\qquad (t\to\infty), $$ where $C>0$, $\lambda>0$, and $r$ is a fixed real number. The notation $\sim$ means that the ratio of the two sides tends to one. For all sufficiently large $n$, set $$ b_n=\frac{\log n+r\log\log n+\log C-r\log\lambda}{\lambda}, \qquad a_n=\frac{1}{\lambda}. $$ The logarithmic correction involving $\log\log n$ accounts for the factor $t^r$. If $r=0$, the tail is asymptotically a constant times a pure exponential and no such correction is needed.

Theorem: If $X_1,\ldots,X_n$ are independent and identically distributed and their survival function satisfies $\overline F(t)\sim C t^r e^{-\lambda t}$ with $C,\lambda>0$ and $r\in\mathbb{R}$, then the sequences above satisfy $$ P\left(\frac{M_n-b_n}{a_n}\le x\right)\longrightarrow \exp(-e^{-x}) $$ for every $x\in\mathbb{R}$.

Proof. Write $L_n=\log n+r\log\log n+\log C-r\log\lambda$, so $b_n=L_n/\lambda$. Since $L_n/\log n\to1$, we have $\log L_n-\log\log n\to0$, and $b_n\to\infty$. Direct calculation gives $$ \log\{nC b_n^r e^{-\lambda b_n}\} =\log n+\log C+r(\log L_n-\log\lambda)-L_n =r(\log L_n-\log\log n)\longrightarrow0. $$ Thus $nC b_n^r e^{-\lambda b_n}\to1$.

For fixed $x$, $b_n+a_nx=b_n+x/\lambda$, which tends to infinity and is positive eventually. The assumed tail equivalence, applied at both $b_n+x/\lambda$ and $b_n$, yields $$ \frac{\overline F(b_n+x/\lambda)}{\overline F(b_n)} \longrightarrow \lim_{n\to\infty}\left(1+\frac{x}{\lambda b_n}\right)^r e^{-x} =e^{-x}. $$ Together with $n\overline F(b_n)\to1$, this implies $n\overline F(b_n+a_nx)\to e^{-x}$.

Independence gives $P(M_n\le t)=F(t)^n$. Put $u_n=\overline F(b_n+a_nx)$. Then $u_n\to0$ and $nu_n\to e^{-x}$. For $0\le u<1$, the inequality $$ -\frac{u}{1-u}\le \log(1-u)\le -u $$ shows that $n\log(1-u_n)\to-e^{-x}$: both bounding expressions tend to that value because $nu_n\to e^{-x}$ and $u_n\to0$. Therefore $$ P\left(\frac{M_n-b_n}{a_n}\le x\right) = (1-u_n)^n =\exp\{n\log(1-u_n)\} \longrightarrow \exp(-e^{-x}). $$ This proves the result. $\square$

Worked Examples

Worked Example: Exponential Lifetimes

Suppose $X_1,\ldots,X_n$ are independent exponential lifetimes with rate $\lambda=2$. Their survival function is $\overline F(t)=e^{-2t}$ for $t\ge0$, so $C=1$, $r=0$, and $\lambda=2$. The theorem gives $b_n=(\log n)/2$ and $a_n=1/2$.

For any fixed $x$, the threshold is $b_n+a_nx=(\log n+x)/2$. For sufficiently large $n$, this threshold is nonnegative, and its survival probability is exactly $$ \overline F(b_n+a_nx) =e^{-2(\log n+x)/2} =\frac{e^{-x}}{n}. $$ Consequently the finite-sample probability is $$ P\left(\frac{M_n-b_n}{a_n}\le x\right) =\left(1-\frac{e^{-x}}{n}\right)^n \longrightarrow \exp(-e^{-x}). $$ At $x=0$ and $n=1000$, the exact probability is $(0.999)^{1000}$, approximately $0.3677$; the limiting value is $e^{-1}$, approximately $0.3679$. The small difference is the finite-sample approximation error.

Worked Example: Gamma Lifetimes and the Log-Log Correction

Let each lifetime have a gamma distribution with shape $3$ and rate $2$. Its survival function is $$ \overline F(t)=e^{-2t}\left(1+2t+\frac{(2t)^2}{2}\right),\qquad t\ge0. $$ As $t\to\infty$, the quadratic term dominates the polynomial, so $\overline F(t)\sim 2t^2e^{-2t}$. Thus $C=2$, $r=2$, and $\lambda=2$. The resulting normalizations are $$ a_n=\frac12,\qquad b_n=\frac{\log n+2\log\log n-\log 2}{2}. $$

For $n=10^6$, $\log n\approx13.8155$ and $\log\log n\approx2.6258$, giving $b_n\approx9.1870$. At a normalized threshold $x=0$, the leading tail approximation gives $$ n\cdot 2b_n^2e^{-2b_n} =n\cdot 2\left(\frac{L_n}{2}\right)^2e^{-L_n}, \qquad L_n=\log n+2\log\log n-\log 2. $$ Since $e^{-L_n}=2/[n(\log n)^2]$, this expression equals $$ \frac{L_n^2}{(\log n)^2}\longrightarrow1. $$ The theorem therefore gives $P(M_n\le b_n)\to e^{-1}$. The $\log\log n$ term is essential: using only $(\log n)/2$ would leave the polynomial factor in the tail unaccounted for.

Normal Maxima: A Quadratic Exponential Tail

The normal survival tail is not of the form $Ct^r e^{-\lambda t}$ with a fixed $\lambda$, because its exponent is quadratic rather than linear. Its maxima still have a Gumbel limit, but the normalizations follow from a closely related tail calculation.

Lemma (Mills’ ratio): For the standard normal density $\phi(t)=(2\pi)^{-1/2}e^{-t^2/2}$ and survival function $\overline\Phi(t)=\int_t^\infty\phi(u)\,du$, $$ \overline\Phi(t)\sim\frac{\phi(t)}{t}\qquad(t\to\infty). $$

Proof. For $t>0$, since $u/t\ge1$ when $u\ge t$, $$ \overline\Phi(t)\le\int_t^\infty \frac{u}{t}\phi(u)\,du=\frac{\phi(t)}{t}. $$ Integration by parts, using $\phi'(u)=-u\phi(u)$, gives $$ \overline\Phi(t)=\frac{\phi(t)}{t}-\int_t^\infty\frac{\phi(u)}{u^2}\,du \ge \frac{\phi(t)}{t}-\frac{\overline\Phi(t)}{t^2}. $$ Rearranging, $\overline\Phi(t)\ge [\phi(t)/t]/(1+1/t^2)$. The upper and lower bounds have ratio tending to one, proving the lemma. $\square$

Worked Example: Maxima of Standard Normal Observations

Let $X_1,\ldots,X_n$ be independent standard normal variables. Write $L_n=\sqrt{2\log n}$ and define $$ b_n=L_n-\frac{\log\log n+\log(4\pi)}{2L_n}, \qquad a_n=\frac{1}{L_n}. $$ To check the normalization, put $c_n=[\log\log n+\log(4\pi)]/2$, so $b_n=L_n-c_n/L_n$. Since $c_n^2/L_n^2\to0$, $$ \frac{b_n^2}{2} =\log n-c_n+\frac{c_n^2}{2L_n^2} =\log n-c_n+o(1). $$ Also $b_n/L_n\to1$. It follows that $$ \frac{n\phi(b_n)}{b_n} \sim \frac{e^{c_n}}{\sqrt{2\pi}L_n} =1, $$ because $e^{c_n}=\sqrt{4\pi\log n}=\sqrt{2\pi}L_n$. Mills’ ratio now gives $n\overline\Phi(b_n)\to1$.

For fixed $x$, let $t_n=b_n+x/L_n$. Then $t_n/b_n\to1$, and $$ \frac{t_n^2-b_n^2}{2} =\frac{b_nx}{L_n}+\frac{x^2}{2L_n^2} \longrightarrow x. $$ Using Mills’ ratio at $t_n$ and $b_n$ shows $\overline\Phi(t_n)/\overline\Phi(b_n)\to e^{-x}$. Hence $n\overline\Phi(t_n)\to e^{-x}$, and independence gives the standard Gumbel limit for $(M_n-b_n)/a_n$. For $n=10^6$, $L_n\approx5.2565$, $b_n\approx4.766$, and $a_n\approx0.1902$; at $x=0$, the limiting probability that $M_n\le b_n$ is $e^{-1}\approx0.3679$.

Interpreting the Result and Avoiding a Common Error

These calculations show how the tail controls both the typical location of a maximum and the scale of its fluctuations. For an exponential tail with rate $\lambda$, the maximum is centered near $(\log n)/\lambda$ and has scale $1/\lambda$. A polynomial tail factor changes the centering by a term of order $\log\log n$, as in the gamma example. For normal observations, the leading location is $\sqrt{2\log n}$, with a smaller correction needed to obtain the stated standard Gumbel normalization.

A common mistake is to use $P(M_n\le t)=1-n\overline F(t)$. That expression is generally not the exact probability: it subtracts the expected number of exceedances from one. Under independence the exact formula is $P(M_n\le t)=[1-\overline F(t)]^n$. The approximation $[1-u_n]^n\approx e^{-nu_n}$ is justified when $u_n\to0$ and $nu_n$ has a finite limit; it does not justify replacing the exact probability by $1-nu_n$.

Key takeaway: To find a Gumbel limit, choose the threshold so that a single observation exceeds it with probability approximately $e^{-x}/n$. Then independence turns the probability of no exceedances into the Gumbel cumulative distribution function.

Check Your Understanding

Use the tail probability at the proposed threshold as the starting point for each question.

For an exponential distribution with rate $\lambda$, what centering and scaling give a standard Gumbel limit for the maximum?
If $\overline F(t)\sim Ct^r e^{-\lambda t}$, which part of the centering accounts for the factor $t^r$?
For the gamma example with shape $3$ and rate $2$, identify $C$, $r$, and $\lambda$ in the tail approximation.
Why does the exact probability $P(M_n\le t)$ involve a power of $F(t)$?
For standard normal maxima, what is the role of the $\log\log n$ term in $b_n$?