Extreme-Value Limits for Normalized Maxima

Why maxima have their own limit theory

A sample maximum is determined by the observations in the far upper tail, rather than by the typical observations near the center of a distribution. As the sample grows, the maximum usually moves, so a fixed distributional limit is not expected without changing its location or scale. The central question is whether suitable centering and scaling produce a nondegenerate limiting distribution.

The possible limits are highly constrained. Under the conditions of the Fisher–Tippett–Gnedenko theorem, every nondegenerate limit for normalized maxima belongs, up to location and positive scale, to one of three types: Gumbel, Fréchet, or reverse-Weibull. The tail behavior of the underlying distribution determines which type can arise.

Definition: Let \(X_1,X_2,\ldots\) be independent and identically distributed real-valued random variables with common distribution function \(F\). Their sample maximum is \(M_n=\max(X_1,\ldots,X_n)\). A sequence of normalized maxima is \((M_n-b_n)/a_n\), where \(b_n\in\mathbb{R}\) and \(a_n>0\). A limit is nondegenerate if its distribution is not concentrated at a single point.

The exact distribution of a maximum

Independence gives the fundamental starting point. For any real \(u\), the event \(M_n\leq u\) occurs exactly when every observation is at most \(u\). Thus

P(M_n\leq u) =P(X_1\leq u,\ldots,X_n\leq u) =\prod_{i=1}^n P(X_i\leq u) =F(u)^n.

Consequently, for \(a_n>0\), the distribution function of the normalized maximum is

P\!\left(\frac{M_n-b_n}{a_n}\leq x\right) =F(b_n+a_nx)^n.

This identity applies to continuous and discrete distributions alike. The limit problem is therefore a question about how close \(F(b_n+a_nx)\) is to one: even a small probability of exceeding the threshold can have a substantial effect when it is repeated \(n\) times.

Theorem (Fisher–Tippett–Gnedenko): Suppose \(X_1,X_2,\ldots\) are iid real-valued random variables with distribution function \(F\). If there are constants \(a_n>0\) and \(b_n\in\mathbb{R}\) such that \((M_n-b_n)/a_n\) converges in distribution to a nondegenerate distribution, then the limit is, after a location and positive-scale transformation, one of the following three canonical distribution functions:

\begin{aligned} \Lambda(x)&=\exp(-e^{-x}), &&x\in\mathbb{R} &&\text{(Gumbel)},\\ \Phi_\alpha(x)&= \begin{cases} 0,&x\leq 0,\\ \exp(-x^{-\alpha}),&x>0, \end{cases} &&\alpha>0 &&\text{(Fréchet)},\\ \Psi_\alpha(x)&= \begin{cases} \exp(-(-x)^\alpha),&x\leq 0,\\ 1,&x>0, \end{cases} &&\alpha>0 &&\text{(reverse-Weibull)}. \end{aligned}

The theorem classifies possible nondegenerate limits; it does not say that every distribution \(F\) has such a limit. When a suitable normalization exists, the type depends on the upper-tail behavior of \(F\). The canonical forms place the Gumbel limit on the whole real line, the Fréchet limit on the positive half-line, and the reverse-Weibull limit on the nonpositive half-line.

From small exceedance probabilities to a limit

A useful calculation explains why these three distribution functions appear. At a threshold \(u_n(x)=b_n+a_nx\), define the exceedance probability \(q_n(x)=1-F(u_n(x))\). Then the maximum identity becomes

P\!\left(\frac{M_n-b_n}{a_n}\leq x\right) =(1-q_n(x))^n.

If \(nq_n(x)\) has a finite limit, the probability of exceeding the threshold is small, but there are \(n\) opportunities to exceed it. The next result makes this balance precise.

Limit criterion: Suppose that for a fixed \(x\), \(0\leq q_n(x)<1\) and \(nq_n(x)\to\tau(x)\) for some finite \(\tau(x)\geq 0\). Then \((1-q_n(x))^n\to\exp(-\tau(x))\).

Proof. Since \(nq_n(x)\) converges to a finite number, \(q_n(x)\to0\). For \(0\leq q<1\), the logarithm satisfies

-\frac{q}{1-q}\leq \log(1-q)\leq -q.

Multiplying by \(n\), the upper bound tends to \(-\tau(x)\), while the lower bound also tends to \(-\tau(x)\), because \(nq_n(x)/(1-q_n(x))\to\tau(x)\). Hence \(n\log(1-q_n(x))\to-\tau(x)\). Exponentiating proves the claim. \(\square\)

Thus, to obtain a candidate limiting distribution \(H\), it is enough to choose thresholds for which \(n[1-F(b_n+a_nx)]\to-\log H(x)\) at the relevant \(x\). The criterion is a pointwise calculation; a full convergence-in-distribution argument also checks convergence at the continuity points of the proposed limit.

Worked examples across the three types

Worked Example: Exponential maxima and the Gumbel limit

Let \(X_1,\ldots,X_n\) be iid exponential random variables with rate \(1\), so \(F(u)=1-e^{-u}\) for \(u\geq0\). Find a centering and scale that give a nondegenerate limit for \(M_n\).

Take \(b_n=\log n\) and \(a_n=1\). For any fixed real \(x\), once \(n\) is large enough that \(\log n+x\geq0\),

\begin{aligned} P(M_n-\log n\leq x) &=F(\log n+x)^n\\ &=\left(1-e^{-(\log n+x)}\right)^n\\ &=\left(1-\frac{e^{-x}}{n}\right)^n\\ &\longrightarrow \exp(-e^{-x}). \end{aligned}

The limit is \(\Lambda(x)\), the Gumbel distribution function. The maximum grows at the rate \(\log n\), while fluctuations around that growing location remain on a scale of one.

Worked Example: Pareto maxima and the Fréchet limit

Suppose \(X_1,\ldots,X_n\) have a Pareto distribution with \(F(u)=1-u^{-\alpha}\) for \(u\geq1\), where \(\alpha>0\), and \(F(u)=0\) for \(u<1\). Use \(a_n=n^{1/\alpha}\) and \(b_n=0\).

For fixed \(x>0\), \(n^{1/\alpha}x\geq1\) for all sufficiently large \(n\). Therefore

\begin{aligned} P\!\left(\frac{M_n}{n^{1/\alpha}}\leq x\right) &=F(n^{1/\alpha}x)^n\\ &=\left(1-(n^{1/\alpha}x)^{-\alpha}\right)^n\\ &=\left(1-\frac{x^{-\alpha}}{n}\right)^n\\ &\longrightarrow \exp(-x^{-\alpha}). \end{aligned}

For \(x\leq0\), the normalized maximum is positive, so its distribution function is zero. These limits together give \(\Phi_\alpha\). The positive support and power-law tail form distinguish the Fréchet type.

Worked Example: Uniform maxima and the reverse-Weibull limit

Let \(X_1,\ldots,X_n\) be iid uniform on \((0,1)\). The upper endpoint is finite, so consider the centered and scaled maximum \(n(M_n-1)\), which is nonpositive. For fixed \(x\leq0\), \(1+x/n\in[0,1]\) for all sufficiently large \(n\), and

\begin{aligned} P\!\left(n(M_n-1)\leq x\right) &=P(M_n\leq 1+x/n)\\ &=F(1+x/n)^n\\ &=(1+x/n)^n\\ &\longrightarrow e^x. \end{aligned}

For \(x>0\), \(n(M_n-1)\leq0<x\) always, so the probability is one. Since \(e^x=\exp(-(-x)^1)\) when \(x\leq0\), the limit is \(\Psi_1\), a reverse-Weibull distribution. The negative sign in \(n(M_n-1)\) records that the maximum approaches the endpoint from below.

What the three types reveal

The limit type is a concise description of how the upper tail or upper endpoint behaves under increasingly extreme sampling. Gumbel limits arise in models where maxima, after suitable centering, have a limiting distribution on the full real line. Fréchet limits have a positive power-law form and are associated with sufficiently heavy upper tails. Reverse-Weibull limits are bounded above and arise when the distribution has a finite upper endpoint. These are broad associations, not substitutes for checking the assumptions and normalization for a particular model.

The normalizing constants matter. A maximum that diverges should generally be centered, as in the exponential example; a heavy-tailed maximum may instead be rescaled without centering, as in the Pareto example. With a finite endpoint, centering at that endpoint and using a positive scale produces a limit supported on the nonpositive half-line. Different valid choices of constants can produce location-scale versions of the same canonical limit.

Common pitfall: It is incorrect to treat \(P(M_n\leq u)\) as \(nF(u)\), or to apply the one-observation distribution directly to the maximum. Independence gives \(P(M_n\leq u)=F(u)^n\). The useful approximation is instead \((1-q_n)^n\to e^{-\tau}\) when \(nq_n\to\tau\), where \(q_n\) is the one-observation probability of exceeding the threshold.

A second error is to assume every maximum has a Gumbel limit because the maximum is an extreme observation. The Pareto and uniform calculations show why that is not true: tail heaviness and a finite endpoint lead to different limit types. The general theorem restricts the possible nondegenerate limits, but identifying the applicable type requires examining the distribution's upper tail and selecting normalizing constants that make exceedances occur at order \(1/n\).

Check Your Understanding

Use the maximum identity and the three canonical forms to answer these questions.

For iid observations with distribution function \(F\), derive the distribution function of \((M_n-b_n)/a_n\).
In the exponential example, why does centering by \(\log n\) give a nondegenerate limit?
What feature of the Pareto limit distinguishes it from the Gumbel limit?
For a uniform distribution on \((0,1)\), why is \(n(M_n-1)\) nonpositive?
If \(nq_n\to\tau\), what is the limit of \((1-q_n)^n\), and why is this useful for maxima?