Why Two Tail Regimes Appear
A sub-Gaussian MGF bound controls deviations with an exponent proportional to the square of the deviation. Some variables have tails that eventually decay more slowly: their MGF is controlled near zero, but only on a finite interval. The resulting tail bound has two regimes. Moderate deviations retain a quadratic exponent, while sufficiently large deviations have a linear exponent. The transition between these regimes is determined by the parameters in the MGF bound.
This distinction is useful when a model is not naturally bounded but still has exponentially decaying tails. The definition below concerns one variable at a time. Bounds for sums of independent sub-exponential variables require additional steps and are not derived here.
Definition Through the MGF
The centering is essential. The MGF in this definition is that of \(X-E[X]\), not of \(X\) itself; a nonzero mean would otherwise contribute a linear term in \(\lambda\). The parameter \(\nu\) controls the quadratic part of the bound, while \(b\) limits how far from zero the MGF inequality is assumed to hold. A finite interval of control is enough to obtain a tail bound, but it forces a change in the tail exponent for large deviations.
Worked Example: A Normal Variable
Let \(X\sim N(3,4)\). Its mean is \(3\), its variance is \(4\), and the MGF of \(X-E[X]\) is $$ E[\exp\{\lambda(X-3)\}]=\exp(2\lambda^2) $$ for every real \(\lambda\). With \(\nu=2\), the right side in the sub-exponential definition is \(\exp(\nu^2\lambda^2/2)=\exp(2\lambda^2)\). Thus the required inequality holds with equality for all \(\lambda\), and in particular for \(|\lambda|<1/2\). One valid choice of parameters is \((\nu,b)=(2,2)\).
For these parameters, the transition scale \(\nu^2/b\) is \(4/2=2\). The general two-sided bound derived below is Gaussian-shaped up to deviations of size \(2\), and has a linear exponent beyond that size. The normal variable itself has a sharper Gaussian tail for every deviation; its membership in the sub-exponential class does not mean that its actual tail must exhibit the weaker large-deviation behavior.
Worked Example: A Centered Exponential Variable
Let \(Y\) have the exponential distribution with rate \(1\), so \(E[Y]=1\), and set \(Z=Y-1\). For \(\lambda<1\), $$ E[\exp(\lambda Z)]=\frac{\exp(-\lambda)}{1-\lambda}, \qquad \log E[\exp(\lambda Z)]=-\lambda-\log(1-\lambda). $$ For \(|\lambda|<1/2\), expand the logarithm in its convergent power series and bound the absolute values of its terms: $$ -\lambda-\log(1-\lambda) =\sum_{k=2}^{\infty}\frac{\lambda^k}{k} \leq \sum_{k=2}^{\infty}\frac{|\lambda|^k}{2} =\frac{\lambda^2}{2(1-|\lambda|)} \leq \lambda^2. $$ The last inequality follows because \(1-|\lambda|>1/2\). Exponentiating gives \(E[\exp(\lambda Z)]\leq \exp(\lambda^2)\). This is the defining bound with \(\nu^2=2\), and the interval \(|\lambda|<1/2\) corresponds to \(b=2\). Thus \(Z\) is sub-exponential with parameters \((\sqrt{2},2)\).
Here the transition scale is \(\nu^2/b=2/2=1\). For example, at \(t=8\), the large-deviation part of the two-sided bound below gives $$ P(|Z|\geq 8)\leq 2\exp\!\left(-\frac{3(8)}{8(2)}\right) =2\exp(-3/2). $$ This bound is valid but conservative. In fact, \(Z=Y-1\geq -1\), so the event \(|Z|\geq 8\) is just \(Y\geq 9\), whose probability is \(\exp(-9)\).
Deriving the Two-Regime Tail Bound
The MGF condition leads to a tail inequality by applying Markov's inequality to an exponential transform. The restriction \(|\lambda|<1/b\) matters: the value of \(\lambda\) that would optimize the quadratic expression is not available for every deviation size.
To prove the result, first consider the upper tail and put \(W=X-E[X]\). For any \(0<\lambda<1/b\), Markov's inequality and the MGF assumption give $$ P(W\geq t) =P\bigl(\exp(\lambda W)\geq \exp(\lambda t)\bigr) \leq \exp(-\lambda t)E[\exp(\lambda W)] \leq \exp\!\left(-\lambda t+\frac{\nu^2\lambda^2}{2}\right). $$ The proof splits according to whether the unconstrained choice \(\lambda=t/\nu^2\) lies inside the permitted interval.
If \(0<t<\nu^2/b\), choose \(\lambda=t/\nu^2\), which satisfies \(0<\lambda<1/b\). Substitution gives \(P(W\geq t)\leq \exp(-t^2/(2\nu^2))\). At \(t=\nu^2/b\), take a sequence of permitted \(\lambda\) values increasing to \(1/b\); the same bound follows by continuity of the displayed numerical expression.
If \(t>\nu^2/b\), choose \(\lambda=1/(2b)\), which is strictly inside the permitted interval. Then \(P(W\geq t)\leq \exp(-t/(2b)+\nu^2/(8b^2))\). Since \(t>\nu^2/b\), we have \(\nu^2/(8b^2)<t/(8b)\), so this is at most \(\exp(-3t/(8b))\).
Apply the same argument to \(-W\). Its MGF satisfies \(E[\exp(\lambda(-W))]=E[\exp((-\lambda)W)]\leq\exp(\nu^2\lambda^2/2)\) for \(|\lambda|<1/b\). Add the upper- and lower-tail probabilities to obtain the stated two-sided inequality.
For the simpler bound, when \(t\leq \nu^2/b\), the minimum is \(t^2/\nu^2\), and the quadratic bound has exponent \(-t^2/(2\nu^2)\), which is no larger than \(-t^2/(4\nu^2)\). When \(t>\nu^2/b\), the minimum is \(t/b\), and the large-deviation exponent \(-3t/(8b)\) is no larger than \(-t/(4b)\). These comparisons prove the simpler inequality in both ranges. \(\square\)
Worked Example: A Laplace Variable
Let \(X\) have the Laplace density \(f(x)=\tfrac12\exp(-|x|)\) for real \(x\). This distribution is symmetric about zero, so \(E[X]=0\). Direct integration, valid for \(|\lambda|<1\), gives $$ E[\exp(\lambda X)] =\frac12\int_{-\infty}^{0}\exp((\lambda+1)x)\,dx +\frac12\int_{0}^{\infty}\exp((\lambda-1)x)\,dx =\frac{1}{1-\lambda^2}. $$ For \(|\lambda|<1/2\), $$ \log E[\exp(\lambda X)] =-\log(1-\lambda^2) \leq \frac{\lambda^2}{1-\lambda^2} \leq \frac{4}{3}\lambda^2 \leq \frac{3}{2}\lambda^2. $$ The first inequality uses \(-\log(1-u)\leq u/(1-u)\) for \(0\leq u<1\); it follows, for example, by integrating \(1/(1-s)\leq 1/(1-u)\) over \(0\leq s\leq u\). The second uses \(\lambda^2<1/4\). Therefore the MGF is at most \(\exp(3\lambda^2/2)\) on this interval. The definition holds with \(\nu^2=3\) and \(b=2\), so \((\sqrt{3},2)\) is a valid parameter pair.
The transition scale is \(\nu^2/b=3/2\). At \(t=3\), which is beyond the transition, the theorem gives $$ P(|X|\geq 3)\leq 2\exp\!\left(-\frac{3(3)}{8(2)}\right) =2\exp(-9/16). $$ The exact probability is \(P(|X|\geq 3)=\exp(-3)\), obtained by integrating the two tails of the density. Again, the MGF-based bound captures the exponential form of the large-deviation behavior but need not be numerically tight.
How Rescaling Changes the Parameters
Sub-exponential parameters also behave predictably under multiplication by a constant. This matters when a variable is expressed in different units or when deviations are rescaled.
Indeed, the centered version of \(aX\) is \(a(X-E[X])\). For \(|\lambda|<1/(|a|b)\), we have \(|a\lambda|<1/b\). Hence $$ E[\exp\{\lambda(aX-E[aX])\}] =E[\exp\{(a\lambda)(X-E[X])\}] \leq \exp\!\left(\frac{\nu^2a^2\lambda^2}{2}\right) =\exp\!\left(\frac{(|a|\nu)^2\lambda^2}{2}\right). $$ This is exactly the required MGF bound with parameters \((|a|\nu,|a|b)\). If \(a=0\), the variable \(aX\) is constant and its centered MGF equals \(1\), so it is sub-exponential with any positive parameters. \(\square\)
Interpreting the Transition and Avoiding a Common Error
The scale \(\nu^2/b\) is where the two candidate choices for the exponential parameter meet: the quadratic choice \(t/\nu^2\) reaches the edge \(1/b\) of the MGF control. Below this scale, optimizing the quadratic exponent yields a deviation exponent proportional to \(t^2\). Above it, a fixed admissible value of \(\lambda\) yields an exponent proportional to \(t\). In the simpler bound, this change is visible in the minimum of \(t^2/\nu^2\) and \(t/b\).
Common mistake: applying the quadratic choice \(\lambda=t/\nu^2\) for every \(t\) and reporting a Gaussian tail for all deviations. That choice is valid only when \(t/\nu^2<1/b\), or \(t<\nu^2/b\). For larger \(t\), the MGF assumption gives no bound at that value of \(\lambda\). The correction is to use an admissible value, such as \(1/(2b)\), and accept the resulting linear exponent.
A second practical caution is that a valid bound need not be informative at every \(t\). For small deviations, the right side may exceed \(1\), in which case the probability bound is true but weaker than the elementary bound \(P(A)\leq 1\). Parameter choices can also be conservative, as the exponential and Laplace examples illustrate. The useful conclusion is the form of the two regimes, not a promise that every numerical upper bound is sharp.
Check Your Understanding
Use the definition and derivation above to answer these questions.
- In the definition, why is the MGF bound imposed on \(X-E[X]\) rather than directly on \(X\)?
- For parameters \((\nu,b)=(3,2)\), what is the transition scale, and which exponent applies at \(t=1\) and at \(t=6\)?
- Why is the choice \(\lambda=t/\nu^2\) not available for every deviation size?
- If \(X\) is sub-exponential with parameters \((\nu,b)\), what parameters does the rescaled variable \(-4X\) have according to the proposition?
- A derived upper bound for a probability is \(1.3\). Is the bound invalid? What can be said about its usefulness?