Higher Moments of Independent Sums by Expansion

Why Expand Central Moments?

The mean and variance of a sum of independent random variables are often simple to calculate. Higher central moments require more care: expanding a power produces mixed terms, and some—but not all—of those terms disappear. The third central moment has no surviving mixed terms. The fourth has a contribution from pairs of variances. Knowing exactly why is useful both for calculating a sum’s shape and for avoiding the mistaken assumption that every central moment simply adds.

We will center each summand first, then expand the relevant powers and use independence. The method applies to summands that need not have the same distribution.

Central Moments of the Summands

Definition: Let $X_i$ have a finite fourth absolute moment, and write $m_i=E[X_i]$. Its centered version is $Y_i=X_i-m_i$, so $E[Y_i]=0$. For $r=2,3,4$, define its $r$th central moment by $\mu_{r,i}=E[Y_i^r]$, and write $v_i=\operatorname{Var}(X_i)=\mu_{2,i}$. For a finite collection, the sum $S=\sum_{i=1}^n X_i$ has mean $E[S]=\sum_{i=1}^n m_i$, and its centered version is $S-E[S]=\sum_{i=1}^n Y_i$.

The finite-fourth-moment assumption ensures that all the moments in the expansions below exist. In particular, it gives finite absolute third moments as well: $E[|X_i|^3]\leq (E[|X_i|^4])^{3/4}$. Centering preserves finiteness of the fourth absolute moment. Independence of the $X_i$ implies independence of their centered versions $Y_i$.

The Third and Fourth Moment Formulas

Theorem: Let $X_1,\ldots,X_n$ be independent random variables with finite fourth absolute moments, and define $S$, $Y_i$, $v_i$, and $\mu_{r,i}$ as above. Then $$ E[(S-E[S])^3]=\sum_{i=1}^n \mu_{3,i} $$ and $$ E[(S-E[S])^4]=\sum_{i=1}^n \mu_{4,i} +6\sum_{1\leq i<j\leq n}v_i v_j. $$

The coefficient 6 comes from the number of ways to select two factors of $Y_i$ and two of $Y_j$ in the fourth-power expansion. The proof also explains why terms involving a lone centered factor vanish.

Proof. Put $T=\sum_{i=1}^n Y_i=S-E[S]$. Expanding the cube gives terms indexed by three choices of summand. The terms with all three indices equal are $Y_i^3$. Every other term has at least one index that occurs exactly once. For example, a term with two factors $Y_i$ and one factor $Y_j$, for $i\ne j$, has expectation $$ E[Y_i^2Y_j]=E[Y_i^2]E[Y_j]=v_i\cdot 0=0, $$ where independence factors the expectation and centering gives $E[Y_j]=0$. Thus only the all-equal terms remain: $$ E[T^3]=\sum_{i=1}^n E[Y_i^3]=\sum_{i=1}^n\mu_{3,i}. $$

For the fourth power, classify terms by how often each index occurs. A term with all four factors from one index contributes $E[Y_i^4]=\mu_{4,i}$. A term with three factors from one index and one from another has expectation $E[Y_i^3]E[Y_j]=0$. A term with two factors from one index and one each from two other indices has expectation $E[Y_i^2]E[Y_j]E[Y_k]=0$. A term with four distinct indices also has expectation zero, since its expectation factors into four zero means.

The remaining possibility is two factors from index $i$ and two from index $j$, where $i<j$. For each pair $i,j$, there are $\binom{4}{2}=6$ arrangements of those factors. Each has expectation $$ E[Y_i^2Y_j^2]=E[Y_i^2]E[Y_j^2]=v_i v_j. $$ Adding the all-equal terms and these pair terms proves the fourth-moment formula: $$ E[T^4]=\sum_{i=1}^n\mu_{4,i}+6\sum_{1\leq i<j\leq n}v_i v_j. $$ All expectations and factorizations used here are valid because the fourth absolute moments are finite and the variables are independent. $\square$

Identically Distributed Summands

If the independent summands have a common distribution, let their common variance be $v$ and their common third and fourth central moments be $\mu_3$ and $\mu_4$. There are $\binom{n}{2}=n(n-1)/2$ distinct pairs in the fourth-moment formula. Therefore,

E[(S-E[S])^3]=n\mu_3,\qquad E[(S-E[S])^4]=n\mu_4+3n(n-1)v^2.

The pair contribution is $6\binom{n}{2}v^2=3n(n-1)v^2$. Notice that it depends on the square of the variance, not on the fourth central moment of an individual summand.

Worked Example: A Sum of Four Bernoulli Variables

Let $X_1,\ldots,X_4$ be independent Bernoulli random variables with $P(X_i=1)=1/4$. Find the third and fourth central moments of $S=X_1+\cdots+X_4$.

For one summand $X$, its mean is $1/4$. Thus $Y=X-1/4$ equals $3/4$ with probability $1/4$, and $-1/4$ with probability $3/4$. Its variance is $$ v=E[Y^2]=\frac14\left(\frac34\right)^2+\frac34\left(-\frac14\right)^2 =\frac{9}{64}+\frac{3}{64}=\frac{3}{16}. $$

Its third central moment is $$ \mu_3=E[Y^3]=\frac14\left(\frac34\right)^3+\frac34\left(-\frac14\right)^3 =\frac{27}{256}-\frac{3}{256}=\frac{3}{32}. $$ Its fourth central moment is $$ \mu_4=E[Y^4]=\frac14\left(\frac34\right)^4+\frac34\left(-\frac14\right)^4 =\frac{81}{1024}+\frac{3}{1024}=\frac{21}{256}. $$

Since there are four identically distributed summands, $$ E[(S-E[S])^3]=4\left(\frac{3}{32}\right)=\frac{3}{8}. $$ For the fourth central moment, $$ E[(S-E[S])^4] =4\left(\frac{21}{256}\right)+3(4)(3)\left(\frac{3}{16}\right)^2 =\frac{84}{256}+\frac{324}{256} =\frac{408}{256}=\frac{51}{32}. $$ The positive pair contribution is essential; adding only the individual fourth moments would give $84/256$, not $51/32$.

Worked Example: Adding an Exponential and a Bernoulli Variable

Let $Z$ have an exponential distribution with rate 1, and let $B$ be Bernoulli with success probability $1/2$, independently of $Z$. Find the third and fourth central moments of $S=Z+B$.

For $Z$, the raw moments are $E[Z^k]=k!$ for $k=1,2,3,4$. Its mean is 1 and its variance is $2-1^2=1$. Expanding about the mean gives $$ E[(Z-1)^3]=6-3(2)+3(1)-1=2, $$ $$ E[(Z-1)^4]=24-4(6)+6(2)-4(1)+1=9. $$

For $B$, the centered variable $B-1/2$ takes values $1/2$ and $-1/2$ with equal probabilities. Its variance is $1/4$, its third central moment is 0, and its fourth central moment is $1/16$. The theorem now gives $$ E[(S-E[S])^3]=2+0=2 $$ and $$ E[(S-E[S])^4]=9+\frac{1}{16}+6(1)\left(\frac14\right) =9+\frac{1}{16}+\frac{24}{16} =\frac{169}{16}. $$ Here $E[S]=1+1/2=3/2$. The different means do not create extra terms because the calculation uses the centered summands $Z-1$ and $B-1/2$.

Worked Example: Checking the Formula for Normal Summands

Let $X_1,\ldots,X_5$ be independent and each have distribution $N(0,4)$. Find the third and fourth central moments of their sum.

Each summand has mean 0 and variance $v=4$. By symmetry its third central moment is 0, and its fourth central moment is $3v^2=3(16)=48$. Therefore, $$ E[S^3]=5(0)=0 $$ and $$ E[S^4]=5(48)+3(5)(4)(4^2)=240+960=1200. $$ As a check, the sum is normal with mean 0 and variance $5(4)=20$; its fourth central moment is $3(20)^2=1200$, agreeing with the expansion formula.

Consequences for Skewness and Kurtosis

These formulas also show how standardized shape changes when independent, identically distributed summands are added. Suppose their common variance $v$ is positive. Define the individual skewness as $\gamma_1=\mu_3/v^{3/2}$, the kurtosis as $\beta_2=\mu_4/v^2$, and the excess kurtosis as $\gamma_2=\beta_2-3$. The sum has variance $nv$. Substituting the moment formulas into the definitions gives $$ \operatorname{Skewness}(S) =\frac{n\mu_3}{(nv)^{3/2}} =\frac{\gamma_1}{\sqrt n}, $$ and $$ \operatorname{Kurtosis}(S) =\frac{n\mu_4+3n(n-1)v^2}{(nv)^2} =3+\frac{\gamma_2}{n}. $$ Thus the excess kurtosis of the sum is $\gamma_2/n$. These statements require $v>0$, since standardization divides by powers of the standard deviation.

The calculation makes precise how skewness and excess kurtosis are diluted by adding independent copies: their standardized values decrease with $n$. It does not say that the raw central moments are unchanged, nor does it apply to dependent summands without additional assumptions.

Common Mistake: Treating the Fourth Moment Like the Third

A tempting but incorrect argument is: “The summands are independent and centered, so every mixed term has expectation zero; therefore the fourth central moment of the sum is just the sum of the fourth central moments.” Independence does make many mixed terms vanish, but not all of them. In particular, $E[Y_i^2Y_j^2]=E[Y_i^2]E[Y_j^2]=v_i v_j$, which is generally positive. The fourth-power expansion includes six copies of this term for each pair of indices.

Key takeaway: In the third-power expansion, every mixed term contains a centered factor appearing just once, so its expectation is zero. In the fourth-power expansion, terms with two squared centered factors survive, giving the contribution $6\sum_{i<j}v_i v_j$.

Check Your Understanding

Use the expansions and their assumptions to answer these questions.

Why does $E[Y_i^2Y_j]$ equal zero when $i\ne j$ and $Y_i,Y_j$ are independent and centered?
For two independent centered variables with variances $v_1,v_2$, write the fourth central moment of their sum in terms of their individual central moments and variances.
If $n$ independent, identically distributed variables have variance $v$, what is the coefficient of $v^2$ in the fourth central moment of their sum?
For independent, identically distributed summands with positive variance, how does the sum’s skewness relate to the individual skewness?
What moment condition in the theorem ensures that the third and fourth central moments used in the expansions are finite?