Sub-Gaussian Variables: MGF and Tail Definitions

Two Ways to Describe Gaussian-Scale Concentration

A Gaussian random variable has tails that decrease exponentially in the square of the threshold. Many non-Gaussian variables share this feature, and their moment-generating functions (MGFs) can be bounded by a Gaussian-shaped expression as well. These bounds are useful because they turn information about a single variable into explicit probability bounds.

The parameter in the MGF bound is called a proxy variance. It acts like a variance in the exponential bound, but it need not equal the actual variance. In particular, a variable can have a proxy larger than its variance, and a proxy is not automatically the smallest possible one.

Definition: A random variable $X$ with finite mean is sub-Gaussian with proxy variance $v$, where $v\geq 0$, if $$ E\!\left[\exp\{\lambda(X-E[X])\}\right]\leq \exp\!\left(\frac{v\lambda^2}{2}\right) \quad\text{for every }\lambda\in\mathbb{R}. $$ Equivalently, the centered variable $Z=X-E[X]$ has an MGF bounded by $\exp(v\lambda^2/2)$ for every real $\lambda$. This condition in particular requires the MGF to be finite for every real $\lambda$.

The centering matters: the bound concerns deviations from $E[X]$, not from zero unless $X$ is already mean zero. A proxy variance $v$ is any value for which the displayed inequality holds. If several values work, the smallest admissible one is the sharpest proxy.

From an MGF Bound to a Tail Bound

The MGF definition directly yields a two-sided tail bound. The argument uses the exponential form of Markov's inequality and chooses the exponential parameter to make the resulting bound as small as possible.

Theorem: If $X$ is sub-Gaussian with proxy variance $v>0$, then for every $t\geq 0$, $$ P\bigl(|X-E[X]|\geq t\bigr)\leq 2\exp\!\left(-\frac{t^2}{2v}\right). $$ For $v=0$, the defining MGF condition implies $X=E[X]$ almost surely, so the probability is zero for every $t>0$.

Proof. Write $Z=X-E[X]$. For any $\lambda>0$, Markov's inequality and the MGF bound give

P(Z\geq t) =P(e^{\lambda Z}\geq e^{\lambda t}) \leq e^{-\lambda t}E[e^{\lambda Z}] \leq \exp\!\left(-\lambda t+\frac{v\lambda^2}{2}\right).

For $t>0$, the exponent is minimized over $\lambda>0$ at $\lambda=t/v$. Substitution gives $P(Z\geq t)\leq \exp(-t^2/(2v))$. Apply the same argument to $-Z$, whose MGF at $\lambda$ is the MGF of $Z$ at $-\lambda$, to get $P(Z\leq -t)\leq \exp(-t^2/(2v))$. The events $\{Z\geq t\}$ and $\{Z\leq -t\}$ cover $\{|Z|\geq t\}$, so adding their probability bounds proves the result. If $v=0$, the MGF bound and Jensen's inequality imply $E[e^{\lambda Z}]=1$ for every $\lambda$; alternatively, applying the tail argument with any positive number in place of $v$ and then letting that number decrease to zero shows $P(|Z|\geq t)=0$ for $t>0$. $\square$

A common tail-based formulation says that a centered variable satisfies a Gaussian tail bound with scale $s>0$ if $P(|X-E[X]|\geq t)\leq 2\exp(-t^2/(2s^2))$ for all $t\geq0$. The theorem shows that an MGF proxy $v$ supplies this tail bound with $s^2=v$. A tail bound stated on its own is not the MGF definition with the same parameter: constants can change when translating from tail information back to an MGF bound. Keep the stated convention and parameter clear when using either form.

What the Proxy Says About Variance

A proxy variance is at least the actual variance, when that variance exists. This comparison follows from the local behavior of the MGF, but the regularity condition behind that behavior must be stated: finite variance alone does not ensure that the MGF is finite near zero.

Theorem: If $X$ is sub-Gaussian with proxy variance $v$, then $X$ has finite variance and $$ \operatorname{Var}(X)\leq v. $$

Proof. Let $Z=X-E[X]$. The MGF definition ensures that $E[e^{\lambda Z}]$ is finite for every real $\lambda$, so in particular it is finite in an open neighborhood of zero. This regularity permits the second-order expansion

E[e^{\lambda Z}] =1+\frac{\lambda^2}{2}E[Z^2]+o(\lambda^2) \quad\text{as }\lambda\to 0,

because $E[Z]=0$. One justification for the expansion is to choose $\delta>0$ such that the MGF is finite at both $2\delta$ and $-2\delta$. Then $Z^2e^{\delta|Z|}$ is integrable: for sufficiently large $|Z|$, its value is bounded by a constant times $e^{2\delta Z}+e^{-2\delta Z}$. Taylor's formula and dominated convergence therefore justify taking the expectation of the second-order expansion. Taking logarithms gives

\log E[e^{\lambda Z}] =\frac{\lambda^2}{2}\operatorname{Var}(X)+o(\lambda^2).

The sub-Gaussian inequality gives $\log E[e^{\lambda Z}]\leq v\lambda^2/2$. Divide by $\lambda^2/2$ for $\lambda\ne0$, then let $\lambda\to0$. The left side tends to $\operatorname{Var}(X)$, proving the claim. $\square$

Worked Proxies in Familiar Distributions

Worked Example: A Gaussian Variable Has Proxy Equal to Its Variance

Let $X\sim N(\mu,\sigma^2)$, with $\sigma^2>0$. The MGF of a normal variable is

E[e^{\lambda X}]=\exp\!\left(\mu\lambda+\frac{\sigma^2\lambda^2}{2}\right).

Centering removes the linear term:

E[e^{\lambda(X-\mu)}] =e^{-\lambda\mu}E[e^{\lambda X}] =\exp\!\left(\frac{\sigma^2\lambda^2}{2}\right).

Thus $X$ is sub-Gaussian with proxy $v=\sigma^2$, and its two-sided tail bound is $P(|X-\mu|\geq t)\leq2\exp(-t^2/(2\sigma^2))$. Here the proxy matches the variance, and the MGF bound holds with equality for every $\lambda$.

Worked Example: A Symmetric Two-Point Variable

Let $X$ equal $a$ and $-a$, each with probability $1/2$, where $a>0$. Its mean is zero, and

E[e^{\lambda X}] =\frac{e^{a\lambda}+e^{-a\lambda}}{2} =\cosh(a\lambda).

For every real $u$, $\cosh(u)\leq e^{u^2/2}$. To verify the inequality, both sides are even, so it is enough to consider $u\geq0$. The derivative of $u^2/2-\log\cosh(u)$ is $u-\tanh(u)$, which is nonnegative because $\tanh(u)\leq u$. The difference is zero at $u=0$, so it is nonnegative for all $u$. Therefore

E[e^{\lambda X}] \leq \exp\!\left(\frac{a^2\lambda^2}{2}\right).

A valid proxy is $v=a^2$, which equals $\operatorname{Var}(X)$. This example also checks the tail scale: $P(|X|\geq t)=1$ for $0\leq t\leq a$ and is zero for $t>a$; the general bound $2\exp(-t^2/(2a^2))$ is valid, although it need not be sharp at every threshold.

Worked Example: A Bounded Uniform Variable

Let $X$ be uniform on $[-3,5]$. Its mean is $(-3+5)/2=1$, and its range has width $5-(-3)=8$. Hoeffding's lemma states that if a random variable $W$ lies in an interval of width $b-a$, then $E[e^{\lambda(W-E[W])}]\leq\exp(\lambda^2(b-a)^2/8)$ for every real $\lambda$. Applying it with $a=-3$ and $b=5$ gives

E[e^{\lambda(X-1)}] \leq \exp\!\left(\frac{64\lambda^2}{8}\right) =\exp(8\lambda^2) =\exp\!\left(\frac{16\lambda^2}{2}\right).

Hence $v=16$ is a valid proxy, and the corresponding tail bound is $P(|X-1|\geq t)\leq2\exp(-t^2/32)$. The actual variance of a uniform distribution on an interval of width 8 is $8^2/12=16/3$, which is less than the proxy. This comparison illustrates why a convenient valid proxy need not be the exact variance.

Boundedness and the Meaning of a Tail Definition

Every bounded variable is sub-Gaussian after centering. If $X\in[a,b]$ almost surely, Hoeffding's lemma gives proxy $v=(b-a)^2/4$, since $(b-a)^2/8=v/2$. This bound depends only on the interval width, not on the detailed distribution within the interval. For a particular bounded distribution it may be conservative, but it provides a usable proxy without calculating the MGF exactly.

The proxy also gives a direct threshold calculation. If a centered variable has proxy $v=4$, then at $t=6$ the two-sided bound is $2\exp(-36/8)=2e^{-4.5}$, approximately $0.0222$. This is an upper bound, not an assertion that the actual probability equals that value.

Common Pitfalls

Using the wrong center. The MGF bound concerns $X-E[X]$. Unless $E[X]=0$, applying it directly to $X$ introduces an unnecessary linear term.
Treating the proxy as the variance. The variance cannot exceed a valid proxy, but equality is not guaranteed. In the uniform example, the proxy 16 exceeds the variance $16/3$.
Confusing tail and MGF parameters. An MGF proxy $v$ implies a tail scale with $s^2=v$ in the stated bound. A tail-only condition should not be substituted into an MGF calculation as though it were the same inequality.
Expanding an MGF from finite variance alone. Finite variance does not ensure MGF finiteness at nonzero arguments. A lognormal variable has finite moments of every positive integer order but an infinite MGF for every positive argument. The centered MGF expansion used above is justified because the sub-Gaussian definition supplies MGF finiteness in a neighborhood of zero.

Key takeaway: The MGF condition $E[e^{\lambda(X-E[X])}]\leq e^{v\lambda^2/2}$ is a precise definition with a directly interpretable proxy. It yields a Gaussian two-sided tail bound, forces $\operatorname{Var}(X)\leq v$, and is easy to verify exactly for Gaussians or to bound for variables with known finite ranges.

Check Your Understanding

Use the definitions and results above to answer the following.

If a centered variable has MGF bounded by $\exp(3\lambda^2/2)$, what two-sided tail bound follows at threshold $t$?
A variable is uniform on an interval of width 6. What proxy does Hoeffding's lemma provide, and what tail bound follows?
Why does a valid proxy have to be at least the variance when the variance exists?
For the symmetric variable taking values $-a$ and $a$, identify its mean, variance, and proxy from the MGF calculation.
Why is finite variance by itself not enough to justify a Taylor expansion of the MGF around zero?