Start With the Question, Not the Theorem
A sample statistic can be studied in two different ways. One question is whether it gets close to a fixed population quantity as the sample grows. Another is how its random error behaves after it is centered and rescaled. The law of large numbers (LLN) addresses the first question; the central limit theorem (CLT) addresses the second under stronger assumptions. Choosing between them begins by identifying what the question asks for.
For example, a statement that a sample average approaches a population mean is a convergence statement. A statement that the sample average’s error, multiplied by \(\sqrt{n}\), is approximately normal is a distributional statement about its fluctuations. These are related but are not interchangeable. The CLT can help describe a finite-sample error, while the LLN alone generally does not give a normal approximation or a rate.
The LLN’s finite-mean condition is weaker than the CLT’s finite, positive variance condition. Thus the CLT cannot be invoked merely because a sample average is involved. Conversely, when the CLT’s assumptions hold, it gives more than an unscaled convergence claim: it describes the error at the \(\sqrt{n}\) scale.
A Practical Decision Process
Before applying either result, define the observation being averaged and identify its mean and variance, if they exist. For a statistic that is an average of transformed observations, such as \(n^{-1}\sum_{i=1}^n g(X_i)\), apply the same logic to \(Y_i=g(X_i)\): check the moments of \(Y_i\), not just those of \(X_i\).
Is the question about convergence to a fixed value, such as \(E[X_i]\), or about the distribution of a centered and scaled error?
For a sample mean use \(X_i\); for an empirical proportion use Bernoulli indicators; for an average of transformed data use \(Y_i=g(X_i)\).
For the iid LLN stated here, check independence, a common distribution, and a finite absolute mean. For the iid CLT, also check finite, positive variance.
Use the LLN for convergence in probability to the target. Use the CLT for the limiting distribution of the centered statistic multiplied by \(\sqrt{n}\), then make a normal approximation only when that is what the question requires.
These are standard iid results, not universal rules for every data-generating process. Dependence, changing distributions across observations, or a statistic that is not an average can require different arguments. Do not assume the usual iid conclusion applies before checking the setup.
Why the Scales Differ
The distinction is visible directly in the algebra for a sample mean. Under the iid finite-variance assumptions, the CLT applies to the centered sum. Dividing that sum by \(\sqrt{n}\) gives the centered, scaled mean:
The unscaled error is therefore the scaled error divided by \(\sqrt{n}\). In the finite-variance iid setting, the LLN’s conclusion can also be seen from Chebyshev’s inequality:
The inequality shows why the mean gets close to \(\mu\); the CLT tells us how its error behaves after multiplication by \(\sqrt{n}\). The LLN itself does not supply the normal distribution in the CLT statement. Nor does a finite-variance Chebyshev argument establish that normal approximation: that requires the CLT’s distributional conclusion.
Worked Examples
Worked Example: A Finite Mean but Infinite Variance
Let \(X_1,\ldots,X_n\) be iid observations from a hypothetical Pareto distribution with \(P(X_i>x)=x^{-1.5}\) for \(x\geq 1\). Decide which result justifies a limit for the sample mean and whether the usual finite-variance CLT applies.
For this Pareto distribution, the density is \(f(x)=1.5x^{-2.5}\) for \(x\geq 1\). Its mean is finite:
But its second moment diverges, since \(E[X_i^2]=1.5\int_1^\infty x^{-0.5}\,dx=\infty\); hence its variance is infinite. The iid finite-mean LLN applies, so \(\bar{X}_n\) converges in probability to \(3\). For example, \(P(|\bar{X}_n-3|>0.1)\to 0\) as \(n\to\infty\).
The finite-variance iid CLT does not apply because the variance is infinite. This does not establish that no possible limit theorem could describe this distribution; it means the usual CLT given above does not justify a normal limit. The LLN answers the convergence question without providing a finite-sample probability or a normal approximation.
Worked Example: A Sample Proportion and a Normal Approximation
In a hypothetical population, each independently sampled person supports a proposal with probability \(p=0.30\). Let \(X_i=1\) for a supporter and \(X_i=0\) otherwise, and let \(\hat{p}=n^{-1}\sum_{i=1}^n X_i\). For \(n=400\), approximate the probability that \(\hat{p}\) is within \(0.04\) of \(0.30\).
The indicators are iid Bernoulli with \(E[X_i]=p=0.30\) and \(\operatorname{Var}(X_i)=p(1-p)=0.30(0.70)=0.21\). Thus the LLN gives \(\hat{p}\to_p 0.30\), while the CLT gives the approximate sampling distribution at the \(\sqrt{n}\) scale. The standard error for the normal approximation is
Standardizing the two boundaries gives
where \(Z\sim N(0,1)\); the unrounded positive bound is approximately \(1.74574\), which rounds to \(1.746\). The result is an asymptotic normal approximation, not an exact binomial probability. In context, it estimates that about 91.9% of samples of 400 would have a support proportion within 4 percentage points of \(0.30\). The independence and common Bernoulli probability are part of the stated hypothetical sampling setup; \(p\) and \(1-p\) are both positive, and the expected supporter and non-supporter counts are 120 and 280.
Worked Example: Applying Both Results to an Average of Squares
Suppose \(X_1,\ldots,X_n\) are iid \(N(0,1)\) measurements, and define \(Q_n=n^{-1}\sum_{i=1}^n X_i^2\). Find its probability limit and use the CLT to approximate \(P(|Q_{200}-1|<0.10)\).
Set \(Y_i=X_i^2\), so \(Q_n\) is the sample mean of the iid \(Y_i\). For a standard normal variable, \(E[X_i^2]=1\) and \(E[X_i^4]=3\). Therefore \(E[Y_i]=1\), and
The finite-mean LLN gives \(Q_n\to_p 1\). Since \(Y_i\) also has finite, positive variance, the iid CLT gives \(\sqrt{n}(Q_n-1)\xrightarrow{d}N(0,2)\). For \(n=200\), the standard deviation in the normal approximation to \(Q_n\) is \(\sqrt{2/200}=0.10\). Thus
This is a large-sample approximation to the probability that the average squared measurement is between \(0.90\) and \(1.10\). Both results apply to the transformed observations \(Y_i\): their iid structure and moments have been checked, rather than inferred solely from the fact that the original \(X_i\) are normal.
What Each Result Does, and Does Not, Say
| Question | Result to consider | Conclusion in the sample-mean setting |
|---|---|---|
| Does the statistic approach a fixed target? | LLN | \(\bar{X}_n\to_p\mu\), under the stated iid finite-mean conditions. |
| What is the limiting distribution of the centered, scaled statistic? | CLT | \(\sqrt{n}(\bar{X}_n-\mu)\xrightarrow{d}N(0,\sigma^2)\), under the stated iid finite-variance conditions. |
| What is a large-sample probability for a fixed sample size? | Often CLT, if its assumptions fit | Use a normal approximation and label it as approximate, not exact. |
An LLN conclusion can be practically important even when a CLT is unavailable: it establishes convergence in probability under its assumptions. But it does not, by itself, tell us the shape of the sampling distribution at a particular \(n\). A CLT supplies a limiting distribution under stronger conditions; it is not a guarantee that the approximation is accurate for every sample size or every event. Likewise, an approximate probability from a CLT is not an exact finite-sample probability.
Common Mistakes
- Using the CLT because the statistic is an average. An average alone is not enough. Check the sampling structure and finite variance of the observations actually being averaged.
- Using the LLN to claim approximate normality. Convergence in probability to a constant is not the same conclusion as convergence in distribution after centering and scaling.
- Forgetting the scale. The CLT concerns \(\sqrt{n}(\bar{X}_n-\mu)\), not \(\bar{X}_n\) converging to a normal distribution with a fixed, nonzero spread. As \(n\) grows, the unscaled mean concentrates around \(\mu\).
- Checking moments of the wrong variable. For an average of \(g(X_i)\), check the mean and variance of \(g(X_i)\). The distribution of the original observation alone does not replace that check.
- Treating an approximation as exact. A CLT-based probability is a large-sample approximation. Report it as such and keep its units and event tied to the original statistic.
Check Your Understanding
For each question, identify the appropriate result and state what assumption must be checked.
- For iid observations with finite mean but infinite variance, what does the stated weak LLN say about the sample mean? Does the finite-variance iid CLT apply?
- An analyst wants to describe the limiting distribution of \(\sqrt{n}(\bar{X}_n-\mu)\). Which theorem is relevant, and what moment condition is required in the iid version presented here?
- For an average of \(g(X_i)\), which random variable’s variance must be finite to use the iid CLT?
- Why does an LLN conclusion alone not justify a normal approximation for a finite-sample probability?
- If the CLT gives \(\sqrt{n}(\bar{X}_n-\mu)\xrightarrow{d}N(0,\sigma^2)\), what happens to the scale of the unscaled mean’s error as \(n\) increases?