VEE Mathematical Statistics · Law of Large Numbers and CLT · Tutorial 367 of 880

Convergence in Probability for Sample Means

Learn to express and assess whether sample means become concentrated near their population mean as the sample size increases.

Intermediate 10 min read

What You'll Learn

State the definition of convergence in probability using a fixed positive tolerance.
Express convergence of sample means to a population mean in probability notation.
Interpret the probability of a sample mean falling outside a specified band.
Use the exact normal sampling distribution to assess concentration at different sample sizes.
Calculate exact probabilities for a Bernoulli sample proportion and handle strict inequalities.
Recognize that a sample mean does not automatically converge to its marginal population mean when observations are dependent.

Why Convergence in Probability Matters

A sample mean is random before the observations are collected. A particular sample may produce a value close to the population mean, or it may not. The question addressed by convergence in probability is about what happens to the chance of a substantial discrepancy as the sample size increases: does that chance approach zero?

This is a precise way to express the idea that sample means become concentrated near a population mean. It does not say that every sample mean equals the population mean, or that a finite sample is guaranteed to be close. It describes the limiting behavior of probabilities across a sequence of sample sizes.

Definition: A sequence of random variables $T_n$ converges in probability to a constant $c$ if, for every fixed $\varepsilon>0$, $$ \lim_{n\to\infty}P(|T_n-c|>\varepsilon)=0. $$ The notation is $T_n\xrightarrow{P}c$. Equivalently, for every fixed $\varepsilon>0$, the probability that $T_n$ lies within $\varepsilon$ of $c$ approaches 1.

For a sequence of observations $X_1,X_2,\ldots$, define the sample mean of the first $n$ observations by $$ \bar X_n=\frac{1}{n}\sum_{i=1}^{n}X_i. $$ If the observations have a common finite population mean $\mu=E[X_i]$, the statement that their sample means converge in probability to that mean is written $$ \bar X_n\xrightarrow{P}\mu, \qquad\text{meaning}\qquad \lim_{n\to\infty}P(|\bar X_n-\mu|>\varepsilon)=0 \quad\text{for every fixed }\varepsilon>0. $$

Reading the Definition Carefully

The tolerance $\varepsilon$ is a fixed positive number while $n$ increases. For example, one may ask whether the probability that a sample mean differs from $\mu$ by more than 0.5 approaches zero. The definition requires this limiting behavior for every positive fixed tolerance, not just one convenient choice.

The complement of the event “the error exceeds $\varepsilon$” is the event “the error is at most $\varepsilon$.” Since the two events have probabilities summing to 1, $$ P(|T_n-c|\leq\varepsilon)=1-P(|T_n-c|>\varepsilon). $$ Thus, the definition also says that $P(|T_n-c|\leq\varepsilon)$ approaches 1 for every fixed $\varepsilon>0$. The distinction between $>$ and $\geq$ can matter for distributions with point masses at the boundary, so use the event stated in a problem rather than silently changing it.

Convergence in probability concerns a sequence of random variables and their probabilities. It is not a claim that each realized sequence of sample means must move steadily toward $\mu$. A sample mean can move farther from the target at one sample size and closer at another; convergence is determined by the limiting probabilities for each fixed tolerance.

Key takeaway: To check a convergence-in-probability statement, identify the target, choose any fixed positive tolerance, and examine the probability of being outside that tolerance. The required limit is zero for every such tolerance.

Worked Examples

Worked Example: Normal Observations and a Fixed Tolerance

Suppose $X_1,\ldots,X_n$ are independent observations from a hypothetical measurement process, each with distribution $N(12,9)$. Thus, each observation has mean 12 and variance 9. Find the probability that $\bar X_n$ is within 0.5 of 12 when $n=36$ and when $n=144$, then state what happens to this probability as $n$ increases without bound.

Method and assumptions. The observations are independent and normally distributed with the same mean and variance. The sum, and therefore the sample mean, is exactly normal: $$ \bar X_n\sim N\left(12,\frac{9}{n}\right). $$ The second parameter is the variance. The standard deviation of $\bar X_n$ is $3/\sqrt n$.

For $n=36$. The sample-mean variance is $9/36=0.25$, so its standard deviation is $\sqrt{0.25}=0.5$. Standardizing the two endpoints gives $$ P(|\bar X_{36}-12|\leq0.5) =P\left(\frac{-0.5}{0.5}\leq\frac{\bar X_{36}-12}{0.5}\leq\frac{0.5}{0.5}\right) =P(-1\leq Z\leq1), $$ where $Z\sim N(0,1)$. From the standard normal distribution, $\Phi(1)=0.8413$ and $\Phi(-1)=0.1587$. Therefore the probability is $0.8413-0.1587=0.6826$, or 0.6827 using the usual rounded value for the central probability.

For $n=144$. The variance is $9/144=0.0625$, and the standard deviation is $\sqrt{0.0625}=0.25$. The standardized tolerance is $0.5/0.25=2$, so $$ P(|\bar X_{144}-12|\leq0.5)=P(-2\leq Z\leq2) =\Phi(2)-\Phi(-2)=0.9772-0.0228=0.9544. $$ Using the usual rounded central probability gives 0.9545.

Limiting interpretation. For any fixed $\varepsilon>0$, the standardized tolerance is $$ \frac{\varepsilon}{3/\sqrt n}=\frac{\varepsilon\sqrt n}{3}. $$ As $n$ increases, this quantity increases without bound. The standard normal probability outside an interval whose endpoints move arbitrarily far from zero approaches zero. Hence, in this model, $P(|\bar X_n-12|>\varepsilon)\to0$ for every fixed $\varepsilon>0$, so $\bar X_n\xrightarrow{P}12$.

Check. At $n=144$, the tolerance is two sample-mean standard deviations, giving a central probability of about 0.9545, which is greater than the $n=36$ probability of about 0.6827. The common mistake to avoid is using the population standard deviation 3 as the standard deviation of the sample mean; the correct value is $3/\sqrt n$.

Worked Example: A Bernoulli Sample Proportion and a Strict Boundary

A hypothetical quality check marks each inspected component as either defective or acceptable. Let $X_i=1$ for a defective component and $X_i=0$ otherwise. Assume independent observations with $P(X_i=1)=0.4$. For $n=5$ and $n=10$, calculate the probability that the sample mean is strictly within 0.2 of the defect probability $p=0.4$.

Method and model. The sample mean $\bar X_n$ is the sample proportion defective. If $K=\sum_{i=1}^n X_i$, then $K$ has a binomial distribution with parameters $n$ and 0.4, and $\bar X_n=K/n$. The requested event is $$ |\bar X_n-0.4|<0.2. $$ The inequality is strict.

For $n=5$. The condition is $|K/5-0.4|<0.2$. The possible values $K=0,1,2,3,4,5$ give sample proportions $0,0.2,0.4,0.6,0.8,1$. Only $K=2$, with proportion 0.4, is strictly within 0.2 of 0.4. The proportions 0.2 and 0.6 are exactly 0.2 away and are excluded. Thus, $$ P(|\bar X_5-0.4|<0.2) =P(K=2) =\binom52(0.4)^2(0.6)^3 =10(0.16)(0.216) =0.3456. $$

For $n=10$. The event becomes $|K/10-0.4|<0.2$, or $2<K<6$, so the allowed integer values are $K=3,4,5$. Calculate each probability: $$ P(K=3)=\binom{10}{3}(0.4)^3(0.6)^7 =120(0.064)(0.0279936) =0.214990848, $$ $$ P(K=4)=\binom{10}{4}(0.4)^4(0.6)^6 =210(0.0256)(0.046656) =0.250822656, $$ $$ P(K=5)=\binom{10}{5}(0.4)^5(0.6)^5 =252(0.01024)(0.07776) =0.2006581248. $$ Adding the mutually exclusive cases gives $0.214990848+0.250822656+0.2006581248=0.6664716288$, or 0.6665 to four decimal places.

Interpretation and check. For these two sample sizes, the probability of being strictly within the specified band is 0.3456 and 0.6665, respectively. This finite calculation illustrates the event in the definition; by itself, it is not a proof of a limiting statement. The common mistake is to include $K=1$ or $K=3$ when $n=5$: their proportions are exactly on the excluded boundary.

Worked Example: Dependence Can Prevent Concentration

Consider a hypothetical repeated-measurement setting in which a single random environmental effect $Z$ is either $-1$ or $1$, each with probability 0.5. Every measurement in the sequence records the same effect: $X_i=Z$ for all $i$. Each observation has mean zero. Determine whether the sample means converge in probability to that mean.

Method and assumptions. The observations share the same random value, so they are dependent. For every $n$, $$ \bar X_n=\frac{1}{n}\sum_{i=1}^n Z=Z. $$ The population mean of each $X_i$ is $E[X_i]=E[Z]=(-1)(0.5)+(1)(0.5)=0$.

Choose the fixed tolerance $\varepsilon=0.5$. Since $\bar X_n=Z$, its distance from the mean is always $|Z-0|=1$, which exceeds 0.5. Consequently, $$ P(|\bar X_n-0|>0.5)=P(|Z|>0.5)=1 $$ for every $n$. This probability does not approach zero, so $\bar X_n$ does not converge in probability to 0.

Check and lesson. Each individual observation has mean zero, but averaging repeated copies of the same random value does not make the result concentrate at zero. The dependence is essential here. The common mistake is to infer convergence merely from the fact that the number of terms in the average increases; the definition requires the relevant probabilities to approach zero, which they do not in this example.

What the Statement Does—and Does Not—Say

A convergence statement must name its target and use a probability limit. For sample means with population mean $\mu$, the intended statement is $\bar X_n\xrightarrow{P}\mu$. It is not enough to say that “the sample mean gets closer” without specifying what “closer” means. The definition makes the claim testable by fixing a tolerance and considering the probability of exceeding it.

The normal example provides an exact way to evaluate that probability under its stated model. The Bernoulli example shows how the event can translate into a range of integer counts, where strict boundaries need attention. The dependent-measurement example shows why the conclusion cannot be asserted solely from the algebraic form of an average. Assumptions about how observations are generated matter.

For a fixed tolerance, a high probability of being inside the band at one sample size is not by itself convergence in probability. Convergence is about the limit as $n\to\infty$, and it must hold for every fixed positive tolerance. In particular, a tolerance that itself shrinks with $n$ poses a different question from the one in the definition.

Common exam trap: Replacing the limiting condition with a statement about one finite $n$. A calculation such as $P(|\bar X_n-\mu|\leq\varepsilon)=0.95$ for one sample size may describe useful concentration, but it does not establish convergence. To make the convergence claim, the probability of exceeding each fixed tolerance must tend to zero as $n$ increases.

Check Your Understanding

For each problem, identify the relevant event and use the model assumptions stated.

Independent observations have distribution $N(8,16)$. Find $P(|\bar X_{64}-8|\leq1)$. (Answer: 0.9545)
Independent Bernoulli observations have success probability 0.25. For $n=4$, find $P(|\bar X_4-0.25|<0.25)$. (Answer: 0.4219)
Write the definition that states $\bar X_n$ converges in probability to a finite population mean $\mu$. (Answer: For every fixed $\varepsilon>0$, $\lim_{n\to\infty}P(|\bar X_n-\mu|>\varepsilon)=0$.)
Let $Z$ equal 2 or $-2$, each with probability 0.5, and set $X_i=Z$ for every $i$. Does $\bar X_n$ converge in probability to $E[X_i]$? Use $\varepsilon=1$ to justify your answer. (Answer: No; $E[X_i]=0$ and $P(|\bar X_n|>1)=1$ for every $n$.)
Independent observations have distribution $N(5,9)$. For fixed $\varepsilon=0.75$, express $P(|\bar X_n-5|>0.75)$ using the standard normal distribution and state its limit as $n\to\infty$. (Answer: $2[1-\Phi(0.25\sqrt n)]$; the limit is 0.)

Back to Tutorials