Variance-Sum Criteria for Nonidentical Weak Laws

Why a Weak Law Needs a Scale Condition

For independent, identically distributed observations with finite variance, the variance of the sample average shrinks as the sample size grows. With nonidentical observations, the variances may change from one observation to the next. Independence still lets us add their variances, but whether the average concentrates now depends on the entire sequence of variances. The relevant comparison is between their sum and $n^2$, the square of the denominator in the sample average.

The resulting criterion does not require the observations to share a distribution, a mean, or a variance. It controls the random fluctuation around the average of the observations’ expectations. That distinction matters: if the average expectation itself approaches a constant, the sample average approaches that constant; if it does not, the weak law still describes concentration around a moving, deterministic center.

Definition: Let $X_1,X_2,\ldots$ be independent random variables with $E[|X_i|]<\infty$ and $\operatorname{Var}(X_i)<\infty$ for every $i$. Define the sample average and its average expectation by $$ \bar X_n=\frac{1}{n}\sum_{i=1}^n X_i, \qquad m_n=\frac{1}{n}\sum_{i=1}^n E[X_i]. $$ The sequence $\bar X_n$ satisfies a weak law around $m_n$ if, for every $\varepsilon>0$, $$ P\bigl(|\bar X_n-m_n|>\varepsilon\bigr)\longrightarrow 0. $$ Equivalently, $\bar X_n-m_n\xrightarrow{p}0$.

The center $m_n$ is deterministic, though it can depend on $n$. The variance-sum criterion is a convenient sufficient condition for this weak law.

The Variance-Sum Criterion

Theorem (Variance-Sum Criterion): Suppose $X_1,X_2,\ldots$ are independent random variables and each has finite mean and finite variance. If $$ \frac{1}{n^2}\sum_{i=1}^n \operatorname{Var}(X_i)\longrightarrow 0, $$ then $$ \bar X_n-m_n\xrightarrow{p}0. $$

The condition compares the total variance of the sum with $n^2$. Independence is what permits the variance of the sum to be expressed as the sum of the individual variances. To see the bound directly, let $Y_n=\bar X_n-m_n$. It has mean zero and finite variance. For any $\varepsilon>0$, on the event $\{|Y_n|>\varepsilon\}$ we have $Y_n^2>\varepsilon^2$. Therefore $$ \varepsilon^2 P(|Y_n|>\varepsilon) \leq E[Y_n^2] =\operatorname{Var}(Y_n). $$ Independence gives $\operatorname{Cov}(X_i,X_j)=0$ for $i\ne j$, so expanding the variance of the sum yields $$ \operatorname{Var}(Y_n) =\operatorname{Var}\left(\frac{1}{n}\sum_{i=1}^n X_i\right) =\frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}(X_i). $$ Combining the inequalities proves the explicit probability bound $$ P(|\bar X_n-m_n|>\varepsilon) \leq \frac{1}{\varepsilon^2 n^2} \sum_{i=1}^n\operatorname{Var}(X_i). $$ By the assumed variance-sum condition, the right-hand side tends to zero for every fixed $\varepsilon>0$. This is convergence in probability, proving the theorem. $\square$

Worked Example: Bernoulli Observations With Alternating Success Probabilities

Let $X_1,X_2,\ldots$ be independent Bernoulli random variables. For odd $i$, let $P(X_i=1)=1/3$, and for even $i$, let $P(X_i=1)=2/3$. Find the limiting behavior of their sample average.

For a Bernoulli variable with success probability $p$, the mean is $p$ and the variance is $p(1-p)$. Thus each variance here is $2/9$, and $$ \frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}(X_i) =\frac{2n/9}{n^2} =\frac{2}{9n}\longrightarrow 0. $$ The variance-sum criterion applies.

The average expectation is $$ m_n=\frac{1}{n}\left(\left\lceil\frac n2\right\rceil\frac13+ \left\lfloor\frac n2\right\rfloor\frac23\right). $$ If $n$ is even, the two counts are equal and $m_n=1/2$. If $n$ is odd, there is one more odd index, so $m_n=1/2-1/(6n)$. In either case $m_n\to1/2$.

The theorem gives $\bar X_n-m_n\xrightarrow{p}0$. For any $\varepsilon>0$, once $|m_n-1/2|<\varepsilon/2$, the triangle inequality gives $$ P(|\bar X_n-1/2|>\varepsilon) \leq P(|\bar X_n-m_n|>\varepsilon/2)\longrightarrow 0. $$ Therefore $\bar X_n\xrightarrow{p}1/2$, even though the observations do not all have the same distribution.

Changing Variances and Moving Centers

The criterion permits individual variances to grow, provided their accumulated growth is small relative to $n^2$. It also permits the average expectation to move with $n$. These features are useful when observations have different scales or when their expected values do not share a common constant.

Worked Example: Poisson Means and Variances That Increase

Let $X_i$ be independent with $X_i\sim\operatorname{Poisson}(\sqrt{i})$. A Poisson random variable with parameter $\lambda$ has mean and variance both equal to $\lambda$. Hence $$ E[X_i]=\operatorname{Var}(X_i)=\sqrt{i}. $$ Since $\sqrt{i}\leq\sqrt{n}$ for $1\leq i\leq n$, $$ \frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}(X_i) =\frac{1}{n^2}\sum_{i=1}^n\sqrt{i} \leq\frac{n\sqrt{n}}{n^2} =\frac{1}{\sqrt{n}}\longrightarrow 0. $$ Thus $$ \bar X_n-\frac{1}{n}\sum_{i=1}^n\sqrt{i}\xrightarrow{p}0. $$

Here the center is not a fixed finite limit. In fact, the terms with $i\geq\lceil n/2\rceil$ are at least $\sqrt{n/2}$, and there are at least $n/2$ such terms. Consequently $$ \frac{1}{n}\sum_{i=1}^n\sqrt{i} \geq \frac{1}{2}\sqrt{\frac n2}\longrightarrow\infty. $$ The conclusion is concentration around a growing deterministic center, not convergence of $\bar X_n$ to a finite constant.

The fluctuations are small even relative to this growing center. Write $m_n=n^{-1}\sum_{i=1}^n\sqrt{i}$. Since $m_n\to\infty$, for any $\varepsilon>0$, eventually $\varepsilon m_n\geq 1$, and $$ P\left(\left|\frac{\bar X_n}{m_n}-1\right|>\varepsilon\right) =P(|\bar X_n-m_n|>\varepsilon m_n) \leq P(|\bar X_n-m_n|>1)\longrightarrow 0. $$ Thus $\bar X_n/m_n\xrightarrow{p}1$ as well.

Worked Example: Normal Observations With Alternating Means

Let $X_i$ be independent with $X_i\sim N((-1)^i,\sqrt{i})$, where the second argument denotes the variance. The variances satisfy $$ \frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}(X_i) =\frac{1}{n^2}\sum_{i=1}^n\sqrt{i} \leq\frac{1}{\sqrt n}\longrightarrow 0. $$ The criterion therefore shows that $\bar X_n-m_n\xrightarrow{p}0$.

For odd $n$, the sum of the alternating means is $-1$; for even $n$, it is $0$. Hence $m_n=-1/n$ for odd $n$ and $m_n=0$ for even $n$, so $m_n\to0$. To verify convergence to this fixed limit, for any $\varepsilon>0$ take $n$ large enough that $|m_n|<\varepsilon/2$. Then $$ P(|\bar X_n|>\varepsilon) \leq P(|\bar X_n-m_n|>\varepsilon/2)\longrightarrow 0. $$ It follows that $\bar X_n\xrightarrow{p}0$. Both the means and the variances vary with $i$; the variance-sum condition and convergence of the average means together give the conclusion.

From the Moving Center to a Fixed Limit

The examples illustrate a general consequence. Convergence of the average expectations is an additional condition, not part of the variance-sum criterion itself.

Corollary: Under the hypotheses of the variance-sum criterion, if $m_n\to\mu$ for a finite constant $\mu$, then $\bar X_n\xrightarrow{p}\mu$.

Proof. Fix $\varepsilon>0$. Since $m_n\to\mu$, for all sufficiently large $n$, $|m_n-\mu|\leq\varepsilon/2$. The triangle inequality then implies $$ \{|\bar X_n-\mu|>\varepsilon\} \subseteq \{|\bar X_n-m_n|>\varepsilon/2\}. $$ Taking probabilities and applying the theorem gives $$ P(|\bar X_n-\mu|>\varepsilon) \leq P(|\bar X_n-m_n|>\varepsilon/2)\longrightarrow 0. $$ This is exactly $\bar X_n\xrightarrow{p}\mu$. $\square$

Common Pitfall: Finite Variances Are Not Enough

A tempting but incorrect argument is: “Each $X_i$ has finite variance, so the average variance must tend to zero.” Individual finiteness says only that each variance is finite; it places no adequate bound on how quickly the variances can grow. The scale condition concerns the sum of the first $n$ variances divided by $n^2$, not whether each term is finite.

For a concrete check, let the $X_i$ be independent with $X_i\sim N(0,i)$. Every variable has a finite mean and finite variance, but $$ \operatorname{Var}(\bar X_n) =\frac{1}{n^2}\sum_{i=1}^n i =\frac{n(n+1)}{2n^2} =\frac{n+1}{2n}\longrightarrow\frac12, $$ not zero. Indeed, $\bar X_n$ is normal with mean zero and variance $(n+1)/(2n)$, so for any fixed $\varepsilon>0$, $$ P(|\bar X_n|>\varepsilon) =2\Phi\left(-\frac{\varepsilon}{\sqrt{(n+1)/(2n)}}\right) \longrightarrow 2\Phi(-\sqrt{2}\,\varepsilon)>0, $$ where $\Phi$ is the standard normal cumulative distribution function. Thus the average does not converge in probability to zero. The variance-sum condition is sufficient, not a claim that every sequence failing it must fail a weak law.

Key takeaway: For independent observations with finite variances, check the scale of the total variance: if $n^{-2}\sum_{i=1}^n\operatorname{Var}(X_i)\to0$, then the sample average differs from its average expectation by a quantity that vanishes in probability. To conclude convergence to a fixed constant, also check that the average expectations converge.

Check Your Understanding

Use the criterion and its moving-center interpretation to answer these questions.

For independent Bernoulli observations with success probabilities $p_i$, what simple upper bound on their variances guarantees the variance-sum condition?
Suppose $n^{-2}\sum_{i=1}^n\operatorname{Var}(X_i)\to0$, but $m_n$ does not converge to a finite constant. What does the theorem still establish?
Let independent $X_i$ have variance $i^{1/3}$. Bound $n^{-2}\sum_{i=1}^n\operatorname{Var}(X_i)$ and determine whether it tends to zero.
Why does the conclusion $\bar X_n-m_n\xrightarrow{p}0$ not by itself imply $\bar X_n\xrightarrow{p}\mu$ for some fixed $\mu$?
For independent $X_i\sim N(0,i)$, calculate $\operatorname{Var}(\bar X_n)$. What does this show about relying only on each variance being finite?