Independence and Finite-Variance Conditions

Why Check the Conditions?

The central limit theorem gives a useful approximation: under suitable conditions, the distribution of a sample mean becomes close to normal as the sample size grows. But a large sample by itself is not enough. If observations are dependent, the mean may vary more or less than the usual formula predicts. If the population has infinite variance, the standard version of the theorem does not apply at all.

The goal here is to diagnose those two conditions before using a normal model for a sample mean. The standard CLT is not the only possible limit theorem, and failure of its assumptions does not prove that every normal approximation is wrong. It means that this particular theorem has not justified the approximation.

Theorem: In the standard independent and identically distributed (iid) central limit theorem, $X_1,\ldots,X_n$ are independent observations from the same population, with finite mean $\mu$ and finite variance $\sigma^2$. As $n$ grows, the standardized sample mean $$ \frac{\bar{x}-\mu}{\sigma/\sqrt{n}} $$ has a sampling distribution that approaches the standard normal distribution. Here, $\bar{x}=(X_1+\cdots+X_n)/n$, and “finite variance” means $E[(X_i-\mu)^2]<\infty$.

The assumptions play distinct roles. Identical distributions give each observation the same population mean and variance. Independence prevents the observations from carrying additional covariance that changes the variability of their average. Finite variance makes $\sigma/\sqrt{n}$ a finite, meaningful standard error for the sample mean. In applications, the sampling process and the population being sampled determine whether these conditions are plausible.

What Independence Contributes

For random variables with finite variances, the variance of a sum includes both individual variances and covariances:

\operatorname{Var}\!\left(\sum_{i=1}^{n}X_i\right) = \sum_{i=1}^{n}\operatorname{Var}(X_i) + 2\sum_{i<j}\operatorname{Cov}(X_i,X_j).

When observations are independent, each pair has covariance zero. If they are also identically distributed with variance $\sigma^2$, this reduces to $\operatorname{Var}(\bar{x})=\sigma^2/n$, so the standard deviation of the sample mean is $\sigma/\sqrt{n}$. This calculation is a short justification for the standard error used with the iid CLT. With dependence, the covariance terms generally remain, so the usual standard error can be incorrect.

\operatorname{Var}(\bar{x}) = \operatorname{Var}\!\left(\frac{1}{n}\sum_{i=1}^{n}X_i\right) = \frac{1}{n^2} \left( \sum_{i=1}^{n}\operatorname{Var}(X_i) + 2\sum_{i<j}\operatorname{Cov}(X_i,X_j) \right).

Independence is a feature of the data-generating or sampling process, not something established just by looking at a list of observed values. Randomly selecting people does not automatically make their outcomes independent: household members may share exposures, students may be grouped in classrooms, and measurements taken repeatedly from the same person may be related.

Worked Example: Random Sampling From a Population

Suppose a hypothetical large population contains individual measurements with mean $\mu=50$ units and standard deviation $\sigma=12$ units. A simple random sample of $n=64$ different individuals is drawn, with each selected person contributing one measurement. Assess whether the standard iid CLT is a reasonable basis for approximating the sample mean.

Independence and identical distributions. If the population can be treated as a stable source of observations and selection is effectively with replacement, or the sampling fraction is small enough for the observations to be treated as approximately independent, the measurements have the same distribution and are approximately independent. The simple random selection supports representativeness, but the sampling design and any shared structure still need consideration.

Finite variance. The scenario specifies a finite standard deviation of 12 units, so the variance is $\sigma^2=144$ square units, which is finite.

Under those conditions, the sample mean has standard error $$ SE(\bar{x})=\frac{\sigma}{\sqrt{n}} =\frac{12}{\sqrt{64}} =\frac{12}{8} =1.5\text{ units}. $$ The standard CLT supports an approximately normal sampling distribution for $\bar{x}$, centered at 50 units, with standard deviation about 1.5 units. This conclusion depends on the stated sampling conditions; the sample size alone would not establish it.

Dependence From Clustering

A common source of dependence is clustering: observations from the same clinic, school, household, or work site may share conditions. Increasing the number of people sampled within a small number of clusters does not necessarily provide as much independent information as sampling the same number of people across many unrelated clusters.

Worked Example: Patients Within Clinics

Imagine a hypothetical study that measures an outcome for 100 patients, with 10 patients from each of 10 clinics. Use the model $X_{ij}=\mu+U_j+\epsilon_{ij}$, where $U_j$ is a clinic effect and $\epsilon_{ij}$ is an individual component. Assume the clinic effects are independent across clinics, the individual components are independent across patients and clinics, and the two sets of terms are mutually independent. Let $\operatorname{Var}(U_j)=4$ and $\operatorname{Var}(\epsilon_{ij})=9$, in squared outcome units.

The study mean can be written as $\mu$ plus the average of the 10 clinic effects plus the average of all 100 individual components: $$ \bar{x} = \mu+\frac{1}{10}\sum_{j=1}^{10}U_j +\frac{1}{100}\sum_{j=1}^{10}\sum_{i=1}^{10}\epsilon_{ij}. $$

Patients from different clinics have independent clinic effects, but two patients in the same clinic share $U_j$. For two different patients in one clinic, their covariance is $\operatorname{Cov}(X_{ij},X_{kj})=\operatorname{Var}(U_j)=4$, because their individual components are independent and have mean zero.

Using the displayed representation and the assumed independence of its terms, the variance of the study mean is $$ \operatorname{Var}(\bar{x}) = \frac{1}{10^2}(10)(4) + \frac{1}{100^2}(100)(9) = 0.4+0.09 = 0.49 $$ in squared outcome units. Thus its standard deviation is $\sqrt{0.49}=0.70$ outcome units.

Each patient's marginal variance is $4+9=13$. If the 100 patient measurements were incorrectly treated as independent, the calculated variance of their mean would be $13/100=0.13$, corresponding to a standard error of about $0.361$ units. That is substantially smaller than the correct $0.70$ units under this model. The observations are not independent, so the ordinary iid CLT and its usual standard error cannot be applied as if all 100 patients were unrelated. A method that accounts for the cluster structure is needed.

Dependence can also arise when sampling without replacement from a finite population: once one unit is selected, the remaining possibilities change. The earlier material on sampling without replacement and the finite population correction addresses how this affects variability. In checking the ordinary iid CLT, do not label such a sample exactly independent. A small sampling fraction may make independence a useful approximation; otherwise the sampling design needs to be handled explicitly.

Why Finite Variance Matters

A population has finite variance when the expected squared distance from its mean is finite. A population may have a finite mean but infinite variance, so checking only whether an average exists is not enough. The standard CLT requires both a finite mean and a finite variance; its usual normalization uses $\sigma/\sqrt{n}$.

Worked Example: A Population With Infinite Variance

Consider a hypothetical positive measurement $X$ with a Pareto distribution having lower bound $1$ and shape parameter $a=1.5$. Its density is $f(x)=1.5x^{-2.5}$ for $x\geq1$. Determine whether the standard iid CLT conditions hold for independent observations from this population.

The mean is finite. Using the density, $$ E[X] = \int_1^\infty x(1.5x^{-2.5})\,dx = 1.5\int_1^\infty x^{-1.5}\,dx = 3. $$ But the second moment is $$ E[X^2] = \int_1^\infty x^2(1.5x^{-2.5})\,dx = 1.5\int_1^\infty x^{-0.5}\,dx = \infty, $$ because the integral of $x^{-0.5}$ from 1 to infinity diverges. Since the second moment is infinite, the variance is infinite as well.

Thus, even if the observations are independent and identically distributed, the finite-variance condition fails. The standard CLT using $\sigma/\sqrt{n}$ does not justify a normal approximation to the sample mean. This diagnosis does not establish that no other approximation or limit result could apply; it identifies the limit of the standard result.

A Practical Diagnostic

A useful assessment separates the conditions rather than treating “large $n$” as a complete argument. For each situation, ask how observations were generated, whether they share sources of variation, and what is known about the population's spread. A large sample can improve a normal approximation when the assumptions support the standard CLT, but it cannot by itself remove dependence or turn an infinite variance into a finite one.

Identify the sampling units.
Determine what each observation represents and whether multiple observations come from the same person, cluster, time series, or other shared source.

Assess independence.
Use the design and context. Random selection may help, but clustering, repeated measurements, or sampling without replacement can create dependence.

Check the population variance.
Establish whether the population has a finite variance. If the tail behavior is unknown, do not claim the finite-variance condition has been proved merely because a sample standard deviation can be computed.

State the conclusion with its scope.
If the observations are reasonably modeled as iid with finite variance, the standard CLT supports an approximate normal distribution for the sample mean at sufficiently large $n$. If a condition fails, use a method suited to the actual design or distribution.

Common Mistakes and Why the Checks Matter

A frequent mistake is to reason, “There are many observations, so the sample mean must be approximately normal.” The CLT is not a sample-size rule detached from assumptions. A large number of highly correlated measurements can provide much less information than the same number of independent measurements, as the clinic calculation showed. Likewise, an infinite-variance population falls outside the standard iid result no matter how confidently one writes down $n$.

Another mistake is to call a sample independent because its members were selected randomly. Random sampling describes a selection procedure, but observations can remain related because of shared clusters or repeated measurements. Sampling without replacement also creates dependence, even though the finite population correction may make the dependence modest when the sample fraction is small.

These checks matter because the normal approximation is often used to calculate probabilities about sample means. If the model understates the variability, calculated probabilities can be misleading. If a condition is only approximately met, state that clearly rather than presenting the result as exact.

Key takeaway: The standard CLT for sample means requires iid observations with finite mean and finite variance. Independence controls covariance contributions to the mean's variability; finite variance makes the standard $\sigma/\sqrt{n}$ normalization meaningful. A large sample does not substitute for either condition.

Check Your Understanding

For each question, distinguish what the standard CLT assumptions establish from what they do not establish.

A researcher measures one outcome for 80 people, with 40 people from each of two households. What feature of the design should be checked before treating all 80 measurements as independent?
In one sentence, state the independence and variance conditions in the standard iid CLT for a sample mean.
Suppose observations are iid and have a finite mean but infinite variance. Which standard CLT condition fails, and why is $\sigma/\sqrt{n}$ unavailable?
A sample is drawn without replacement from a finite population. Are its observations exactly independent? What information helps decide whether independence is a useful approximation?
Explain why a large sample from a clustered population might have a sample mean with more variability than the iid formula predicts.