Why Mean And Variance Do Not Specify A Distribution

Why Two Moments Are Not a Probability Law

A mean and a variance summarize important features of a response distribution. In a generalized linear model, the variance may even be expressed as a function of the mean, with a dispersion parameter scaling that function. But knowing these quantities does not, by itself, tell us which values the response can take or how probability is distributed among them. Different probability laws can have identical first and second moments.

This distinction matters whenever a model requires more than a description of average level and spread. Probabilities of particular outcomes, tail probabilities, and likelihoods depend on the full distribution, not just its mean and variance. The examples below make the distinction explicit.

Definition: The first two moments of a random variable \(X\) with finite second moment are its mean \(E[X]\) and variance \(\operatorname{Var}(X)=E[(X-E[X])^2]\). Two distributions match in their first two moments if these means are equal and these variances are equal. This condition does not require the distributions themselves to be equal.

The support describes the values a random variable can take, or, for a continuous distribution, the region where its density is positive. The tail behavior describes how quickly probabilities of increasingly extreme values decrease. Neither is generally recoverable from only the mean and variance.

A Family of Distributions With the Same Moments

The following result constructs many different distributions with the same mean and variance. It gives a direct demonstration that the first two moments do not identify a probability law.

Theorem: For every \(p\) with \(0<p<1\), there is a two-point distribution with mean zero and variance one. In particular, varying \(p\) gives distinct distributions with the same first two moments.

Proof. Define \(X_p\) to take the values \(-\sqrt{(1-p)/p}\) with probability \(p\) and \(\sqrt{p/(1-p)}\) with probability \(1-p\). Its mean is

E[X_p] =p\left(-\sqrt{\frac{1-p}{p}}\right) +(1-p)\sqrt{\frac{p}{1-p}} =-\sqrt{p(1-p)}+\sqrt{p(1-p)} =0.

Because the mean is zero, its variance is its second moment:

\operatorname{Var}(X_p) =E[X_p^2] =p\frac{1-p}{p}+(1-p)\frac{p}{1-p} =(1-p)+p =1.

For example, when \(p=1/2\), the values are \(-1\) and \(1\), each with probability \(1/2\). When \(p=1/4\), the values are \(-\sqrt{3}\) and \(1/\sqrt{3}\), with probabilities \(1/4\) and \(3/4\). These are distinct distributions: their supports and probabilities at the support points differ, although both have mean zero and variance one. \(\square\)

Worked Example: Matching Mean and Variance, Different Skewness

Let \(X\) take the values \(-3\) and \(3\), each with probability \(1/2\). Let \(W\) take the value \(-3\sqrt{3}\) with probability \(1/4\) and the value \(\sqrt{3}\) with probability \(3/4\). Verify that their means and variances match, then compare their third central moments.

For \(X\),

E[X]=\frac12(-3)+\frac12(3)=0,\qquad \operatorname{Var}(X)=E[X^2] =\frac12(9)+\frac12(9)=9.

For \(W\),

E[W]=\frac14(-3\sqrt{3})+\frac34(\sqrt{3}) =-\frac{3\sqrt{3}}4+\frac{3\sqrt{3}}4=0,

and, since its mean is zero,

\operatorname{Var}(W)=E[W^2] =\frac14(27)+\frac34(3) =\frac{27}{4}+\frac{9}{4}=9.

Both distributions therefore have mean zero and variance nine. Their third central moments differ. For \(X\), symmetry gives \(E[(X-E[X])^3]=0\). For \(W\), the mean is zero, so

E[(W-E[W])^3]=E[W^3] =\frac14(-3\sqrt{3})^3+\frac34(\sqrt{3})^3 =-\frac{81\sqrt{3}}4+\frac{9\sqrt{3}}4 =-18\sqrt{3}.

Thus matching the first two moments does not even require matching the third central moment. The two laws also have different support points and allocate probability differently.

What Matching Moments Does Guarantee

Although the first two moments do not determine a distribution, they do determine the expectation of every quadratic function of the response. This is a precise statement of what information those moments contain.

Result: If \(X\) and \(Y\) have finite second moments, \(E[X]=E[Y]\), and \(\operatorname{Var}(X)=\operatorname{Var}(Y)\), then \(E[q(X)]=E[q(Y)]\) for every quadratic polynomial \(q(x)=a+bx+cx^2\), where \(a,b,c\) are fixed real numbers.

Derivation. Write the common mean as \(\mu\) and the common variance as \(\sigma^2\). By the definition of variance,

E[X^2]=\operatorname{Var}(X)+(E[X])^2=\sigma^2+\mu^2 =E[Y^2].

Linearity of expectation now gives

E[q(X)] =a+bE[X]+cE[X^2] =a+b\mu+c(\sigma^2+\mu^2) =E[q(Y)].

The finite second-moment condition ensures each expectation in this calculation exists. The result does not extend to arbitrary functions of the response: equal expectations of all quadratic polynomials do not imply equal probabilities for every event or equal expectations of all other functions. \(\square\)

A useful special case is the squared error around a fixed number \(a\). For any random variable with finite variance,

E[(X-a)^2] =E[((X-E[X])+(E[X]-a))^2] =\operatorname{Var}(X)+(E[X]-a)^2.

In the expansion, the cross term has expectation zero because \(E[X-E[X]]=0\). Thus two distributions with matching first two moments have the same expected squared error around every fixed \(a\), even though their probabilities of particular outcomes can differ.

Support and Tail Behavior Can Also Differ

Worked Example: Discrete and Continuous Responses

Compare a random variable \(R\) that equals \(-1\) or \(1\), each with probability \(1/2\), with \(Z\sim N(0,1)\). Both have mean zero and variance one, but one is discrete and the other is continuous.

For \(R\), direct calculation gives \(E[R]=(-1+1)/2=0\) and \(\operatorname{Var}(R)=E[R^2]=1\). By the definition of the standard normal distribution, \(E[Z]=0\) and \(\operatorname{Var}(Z)=1\). However, \(R\) has support \(\{-1,1\}\), whereas \(Z\) has a continuous density over the real line.

For instance, \(P(R=1)=1/2\), while \(P(Z=1)=0\), since a continuous distribution assigns probability zero to any single point. Equal means and variances do not identify even the basic distinction between a discrete and continuous response law.

Worked Example: Equal Variance, Different Tail Decay

Compare \(Z\sim N(0,1)\) with a Laplace random variable \(L\) having density \(f_L(y)=(1/\sqrt{2})\exp(-\sqrt{2}|y|)\). Verify that both have mean zero and variance one, then compare their tail probabilities.

The Laplace density is symmetric, so \(E[L]=0\). Its second moment is

E[L^2] =2\int_0^\infty y^2\frac{1}{\sqrt{2}}e^{-\sqrt{2}y}\,dy =\sqrt{2}\frac{2}{(\sqrt{2})^3} =1.

Here \(\int_0^\infty y^2 e^{-ay}\,dy=2/a^3\) for \(a>0\), as follows by two integrations by parts. Thus \(\operatorname{Var}(L)=1\), matching the standard normal variance.

For \(t\geq 0\), symmetry and integration of the density give the Laplace tail exactly:

P(|L|>t) =2\int_t^\infty \frac{1}{\sqrt{2}}e^{-\sqrt{2}y}\,dy =e^{-\sqrt{2}t}.

For \(t>0\), the standard normal tail satisfies

P(|Z|>t) =\frac{2}{\sqrt{2\pi}}\int_t^\infty e^{-u^2/2}\,du \leq \frac{2}{t\sqrt{2\pi}}e^{-t^2/2}.

To obtain the inequality, use \(u/t\geq 1\) in the integral and then integrate \(u e^{-u^2/2}\). Dividing this upper bound by \(P(|L|>t)\) gives a quantity bounded above by \(\frac{2}{t\sqrt{2\pi}}\exp(-t^2/2+\sqrt{2}t)\), which tends to zero as \(t\) increases. The normal and Laplace responses therefore have different tail decay despite having the same mean and variance.

What This Means for Mean-Variance Relationships

A variance function describes how variance is related to the mean within a specified model framework. For example, a relationship of the form \(\operatorname{Var}(Y)=\phi V(\mu)\) states a constraint on the first two moments, where \(\mu=E[Y]\), \(V\) is the variance function, and \(\phi\) is a dispersion scale. It does not, considered alone, specify the support, probabilities or density, or tail behavior of \(Y\).

A full response distribution supplies additional information needed to calculate quantities such as \(P(Y\in A)\) for an event \(A\), or the likelihood of observed data. Two response laws can agree on the mean and variance while giving different answers to those questions. Consequently, a mean-variance relationship is useful but should not be mistaken for a complete probability model.

Common Mistake

Wrong reasoning: “The response mean and variance match those of a particular named distribution, so the response must follow that distribution.”
Correction: Matching two moments establishes only those two equalities. Check the model’s distributional assumptions separately; other laws can have the same moments but different support, skewness, or tails.
Another caution: A variance function is a relationship across possible means in a model family. It is not, by itself, a statement that every response with that variance has the family’s full probability law.

Key takeaway: The mean and variance determine expectations of quadratic functions, but they do not generally determine the probability distribution. To specify support, tail probabilities, or the likelihood, more than a mean-variance relationship is needed.

Check Your Understanding

Use the definitions and calculations in this tutorial to answer the following.

Give one reason two distributions with the same mean and variance can still be different.
For the two-point family \(X_p\), what are the mean and variance when \(p=1/2\)?
Why do the variables \(R\) and \(Z\) in the discrete-versus-continuous example have different probabilities at the point \(1\)?
What additional feature, besides mean and variance, differs between the normal and Laplace responses in the final worked example?
Does equality of the first two moments imply equal expectations of every function of the response? Explain briefly.