Normalizing Maxima in Standard Distribution Families

Why Normalize a Maximum?

For independent observations, the maximum usually grows with the sample size or moves closer to a finite upper endpoint. A useful limit therefore requires more than taking the maximum itself: we must choose a centering constant \(b_n\) and a positive scale \(a_n\), then study \((M_n-b_n)/a_n\). The choices depend on the tail of the individual observations.

This tutorial works out those choices for exponential, Pareto, and uniform samples. In each case, the verification comes from the exact CDF of the maximum, not from an informal argument about what large observations are likely to do.

The Maximum CDF and a Useful Limit

For any real \(t\), the event \(M_n\leq t\) occurs exactly when every observation is at most \(t\). Independence therefore gives the basic identity.

Thus, to study a normalized maximum at a fixed value \(x\), evaluate \(F(b_n+a_nx)^n\). The following tail calculation will be used repeatedly.

Proof. If \(\tau=0\), then \(0\leq (1-u_n)^n\leq 1\), and Bernoulli’s inequality gives \((1-u_n)^n\geq 1-nu_n\) for \(u_n\in[0,1]\). Both bounds tend to 1. If \(\tau>0\), then \(u_n\to0\), and for all sufficiently large \(n\), \(u_n<1\). Taking logarithms,

because \(\log(1-u)/u\to-1\) as \(u\downarrow0\). Exponentiating proves the result. \(\square\)

In an extreme-value calculation, \(u_n\) is often the probability that one observation exceeds the threshold \(b_n+a_nx\). If that probability is approximately \(\tau(x)/n\), the maximum CDF is approximately \((1-\tau(x)/n)^n\), which tends to \(e^{-\tau(x)}\).

Exponential Samples: Center by the Logarithmic Growth

Let each observation have an exponential distribution with rate \(\lambda>0\), so that \(F(t)=1-e^{-\lambda t}\) for \(t\geq0\). The upper-tail probability at \(t\) is \(e^{-\lambda t}\). To make the expected number of observations above a threshold roughly one, set \(n e^{-\lambda t}\) near 1. Solving gives the centering \(b_n=(\log n)/\lambda\). A shift of size \(1/\lambda\) changes the tail by a fixed multiplicative factor, suggesting \(a_n=1/\lambda\).

The resulting limit is the standard Gumbel CDF \(G(x)=\exp(-e^{-x})\). The rate \(\lambda\) affects the centering and scale, but not this limiting CDF after normalization.

Pareto Samples: Scale by the Tail Index

Consider a Pareto distribution with minimum value \(x_m>0\) and shape parameter \(\alpha>0\). Its CDF is \(F(t)=1-(x_m/t)^\alpha\) for \(t\geq x_m\). The tail probability decreases as a power rather than exponentially. At a threshold \(t=x_m n^{1/\alpha}x\), for fixed \(x>0\) and sufficiently large \(n\), the exceedance probability is \(x^{-\alpha}/n\). This suggests scaling the maximum by \(x_m n^{1/\alpha}\), with no centering.

More generally, for any \(x_m>0\) and \(\alpha>0\), the same calculation gives the limit \(\exp(-x^{-\alpha})\) for \(x>0\). The power \(n^{1/\alpha}\) is essential: it balances the Pareto tail so that the probability of exceeding the normalized threshold is of order \(1/n\).

Uniform Samples: Center at the Upper Endpoint

If \(X_i\sim\operatorname{Uniform}(0,1)\), the maximum cannot grow without bound. Instead, it approaches the upper endpoint 1. The natural scale is the distance from that endpoint: thresholds of the form \(1+x/n\) have CDF \(1+x/n\) when \(x\leq0\) and \(1+x/n\geq0\). This leads to centering \(b_n=1\) and scale \(a_n=1/n\) for \((M_n-b_n)/a_n\).

The endpoint gap has a complementary normalization. Let \(Y_n=n(1-M_n)=-Z_n\). For \(y\geq0\) and \(n\geq y\),

For \(y<0\), this probability is zero. Therefore the endpoint gap converges to an exponential distribution with rate 1. This is the same limit described with a sign reversal; it does not change the centering and scale for the conventional normalized maximum.

A General Verification Pattern

The three calculations share one structure: choose a threshold \(b_n+a_nx\), find the probability \(u_n(x)=1-F(b_n+a_nx)\) that one observation exceeds it, and then use

When \(n u_n(x)\to\tau(x)\), the proved limit gives a maximum CDF of \(\exp(-\tau(x))\). For the exponential model, \(\tau(x)=e^{-x}\); for the Pareto model, \(\tau(x)=x^{-\alpha}\) on \(x>0\); and for the uniform model below the endpoint, \(\tau(x)=-x\) on \(x<0\). The resulting CDFs are evaluated on their full support, with zero or one assigned outside the region where the threshold is inside the distribution’s support.

Common Pitfalls

• Using the scale in the wrong direction. Writing \(a_n(M_n-b_n)\) instead of \((M_n-b_n)/a_n\) changes the normalization. State the normalized variable explicitly before calculating its CDF.
• Centering a Pareto maximum by a logarithm. Exponential tails produce logarithmic growth, but the Pareto tail requires the power scale \(n^{1/\alpha}\).
• Forgetting the finite endpoint. For a uniform sample, thresholds above 1 have CDF 1. Extending the expression \(1+x/n\) as though it were a valid uniform CDF for \(x>0\) is incorrect.
• Dropping the \(n\)th power. A single observation’s exceedance probability tends to zero, but the maximum depends on all \(n\) observations. The factor \(n\) in \(n u_n(x)\) determines the nondegenerate limit.