Why These Two Estimators Become Reliable
The sample mean and sample variance are central summaries of independent observations. Their finite-sample values vary from sample to sample, but under suitable assumptions they approach the corresponding population quantities as the sample size increases. This long-run property is called consistency.
For an actuarial analysis, it matters which assumptions support that conclusion. A proof using Chebyshev’s inequality may require more moments than the consistency result itself. Keeping those two levels separate prevents an unnecessarily strong condition from being mistaken for a necessary one.
Let \(X_1,\ldots,X_n\) be independent and identically distributed (iid) observations with population mean \(\mu=E[X_i]\) and variance \(\sigma^2=\operatorname{Var}(X_i)\). The sample mean and the usual sample variance are
The denominator \(n-1\) defines the usual sample variance. It is important to retain that denominator when using its exact finite-sample properties. For consistency, however, the factor \(n/(n-1)\) that appears in a useful representation approaches 1.
Consistency of the Sample Mean
If the observations have finite variance, independence gives \(E[\bar{X}_n]=\mu\) and \(\operatorname{Var}(\bar{X}_n)=\sigma^2/n\). Chebyshev’s inequality then provides a direct bound on the probability that the sample mean is far from its target.
For every fixed \(\varepsilon>0\), the right-hand side tends to zero as \(n\) increases, so \(\bar{X}_n\) is consistent for \(\mu\). Finite variance is a convenient sufficient condition for this proof. The weak law of large numbers gives the broader result that iid observations with \(E[|X_i|]<\infty\) also have \(\bar{X}_n\to\mu\) in probability.
Worked Example: Choosing a Sample Size for a Bernoulli Mean
A hypothetical audit records whether each independently selected claim file contains a specified coding error. Let \(X_i=1\) when file \(i\) contains the error and \(X_i=0\) otherwise. Assume \(X_i\sim\operatorname{Bernoulli}(0.4)\). Find a sample size for which Chebyshev’s inequality guarantees \(P(|\bar{X}_n-0.4|\ge 0.1)\le 0.05\).
Method. Use the sample-mean variance and Chebyshev’s inequality. For a Bernoulli random variable with success probability \(p\), \(E[X_i]=p\) and \(\operatorname{Var}(X_i)=p(1-p)\). Here,
Independence gives \(\operatorname{Var}(\bar{X}_n)=0.24/n\). Therefore,
To make this upper bound no greater than \(0.05\), require \(24/n\le 0.05\), or \(n\ge 24/0.05=480\). Thus \(n=480\) suffices; for example, at \(n=500\), the bound is \(24/500=0.048\).
Check. At \(n=500\), the sample-mean variance is \(0.24/500=0.00048\), and dividing by the squared tolerance \(0.01\) gives \(0.048\), as above. The bound is a guarantee, not the exact probability. A common mistake is to use \(0.4/n\) as the variance of the mean; the Bernoulli variance is \(p(1-p)/n=0.24/n\).
Consistency of the Sample Variance
A useful way to analyze \(S_n^2\) is to rewrite it in terms of the sample mean and the average of the squared observations. Expanding the squared deviations gives
because \(\sum_{i=1}^{n}X_i=n\bar{X}_n\). Defining \(A_n=n^{-1}\sum_{i=1}^{n}X_i^2\), we obtain
Since \(E[X_i^2]<\infty\), \(E[|X_i|]<\infty\). Thus \(\bar{X}_n\to\mu\) in probability.
The variables \(X_i^2\) are iid and have finite mean \(E[X_i^2]\). Therefore \(A_n\to E[X_i^2]\) in probability.
By continuity, \(\bar{X}_n^2\to\mu^2\) in probability. Hence \(A_n-\bar{X}_n^2\to E[X_i^2]-\mu^2=\sigma^2\).
The deterministic factor \(n/(n-1)\) tends to 1. Thus \(S_n^2=\frac{n}{n-1}(A_n-\bar{X}_n^2)\to\sigma^2\) in probability.
This proof uses the weak law for \(X_i^2\), which requires \(E[X_i^2]\) to be finite. It does not require the variance of \(X_i^2\) to exist. That distinction is important: calculating \(\operatorname{Var}(A_n)\) by a Chebyshev argument would require \(E[X_i^4]<\infty\), a stronger assumption.
Worked Example: A Normal Sample Variance Bound
Suppose a hypothetical measurement process produces iid observations \(X_i\sim N(0,4)\), and a sample of size \(n=10\) is collected. Find \(E[S_{10}^2]\), calculate a Chebyshev upper bound for \(P(|S_{10}^2-4|\ge 2)\), and state how the bound behaves as the sample size grows.
Method. For iid observations with finite fourth central moment \(\mu_4=E[(X_i-\mu)^4]\), the variance of the usual sample variance is
For a normal distribution, \(\mu_4=3\sigma^4\). Here \(\sigma^2=4\), so \(\sigma^4=16\) and \(\mu_4=3(16)=48\). The usual sample variance has expectation \(E[S_{10}^2]=\sigma^2=4\). Its variance is
Chebyshev’s inequality now gives
This is a valid but loose bound. For this normal model, the same calculation at general \(n\) gives \(\operatorname{Var}(S_n^2)=2\sigma^4/(n-1)=32/(n-1)\). Consequently, for any fixed \(\varepsilon>0\), Chebyshev’s bound \(32/((n-1)\varepsilon^2)\) tends to zero.
Check. The squared tolerance is \(2^2=4\), so the bound is \((32/9)/4=8/9\). A common mistake is to treat \(S_n^2\) as having variance \(\sigma^2/n\); that is the sample mean’s variance pattern, not the sample variance’s. The stated sample-variance formula also uses the normal model’s finite fourth moment.
Why a Fourth Moment Is Not Necessary
A fourth moment makes it possible to use some convenient finite-sample variance calculations, but it is not part of the theorem’s minimum condition for consistency. The sample-variance proof needs the average of \(X_i^2\) to converge to \(E[X_i^2]\). The weak law provides this when \(E[X_i^2]\) is finite, even if \(\operatorname{Var}(X_i^2)\) is infinite.
Worked Example: Consistency with an Infinite Fourth Moment
Let \(X_1,X_2,\ldots\) be iid from a Pareto distribution with \(P(X_i>x)=x^{-3}\) for \(x\ge 1\). Determine the population mean and variance, check whether the fourth moment is finite, and decide whether the usual sample variance is consistent.
Method. The density is \(f(x)=3x^{-4}\) for \(x\ge1\). For \(r<3\), integration gives
Thus \(E[X_i]=3/(3-1)=3/2\), and \(E[X_i^2]=3/(3-2)=3\). The variance is
The fourth moment is not finite: its defining integral is \(3\int_1^\infty 1\,dx\), which diverges. However, the second moment is finite. The theorem therefore applies, and \(S_n^2\to 3/4\) in probability.
Check. The existence of the variance follows from the finite second moment, and the sample-variance consistency proof uses that same condition. A calculation requiring \(\operatorname{Var}(X_i^2)\) would fail here, but that does not invalidate the weak-law proof. Confusing the requirements for a Chebyshev bound with the requirements for consistency would incorrectly rule out this example.
Common Exam Traps and Practical Interpretation
- Confusing convergence in probability with an exact guarantee. Consistency says that the probability of a fixed-size error tends to zero as \(n\) grows. It does not state that a particular finite sample must be close to the target.
- Using the wrong moment condition. A Chebyshev proof for the average of \(X_i^2\) asks for \(E[X_i^4]<\infty\). The weak-law consistency proof for \(S_n^2\) only needs \(E[X_i^2]<\infty\).
- Dropping the sample-variance denominator. The identity is \(S_n^2=\frac{n}{n-1}(A_n-\bar{X}_n^2)\), not simply \(A_n-\bar{X}_n^2\). The factor tends to 1, but it is part of the finite-sample formula.
- Assuming iid observations without checking the model. The stated arguments rely on independent observations with a common distribution. Dependence or changing distributions require additional conditions and cannot be handled by these conclusions automatically.
Check Your Understanding
For each problem, identify the relevant moment condition and show enough work to verify the final result.
- Let \(X_i\) be iid Bernoulli with \(p=0.3\). Use Chebyshev’s inequality to bound \(P(|\bar{X}_{200}-0.3|\ge0.05)\). (Answer: the bound is \(0.4200\).)
- A sample consists of \(1,2,4,5\). Calculate its sample mean and its usual sample variance. (Answer: \(\bar{X}_4=3\) and \(S_4^2=10/3\).)
- Suppose iid observations have \(E[X_i]=3\) and \(E[X_i^2]=13\). Find the population variance and state whether the usual sample variance is consistent. (Answer: \(\sigma^2=4\); yes, because the second moment is finite.)
- For iid \(X_i\sim N(5,9)\) with \(n=20\), use the normal-model sample-variance formula and Chebyshev’s inequality to bound \(P(|S_{20}^2-9|\ge3)\). (Answer: \(\operatorname{Var}(S_{20}^2)=162/19\), so the bound is \(18/19\approx0.9474\).)
- A Pareto population has \(P(X_i>x)=x^{-2.5}\) for \(x\ge1\). Its fourth moment is infinite. Is the usual sample variance nevertheless consistent? (Answer: yes; the second moment is finite because \(2.5>2\), so the iid finite-second-moment result applies.)