Finite-Sample Behavior of Variance Estimates

Why Finite-Sample Variability Needs a Separate Check

An asymptotic variance describes how an estimator behaves as the sample size grows. It can provide a useful standard error, but it is not automatically the exact sampling variance at a particular sample size. There is another distinction to keep in view: the true sampling variance is a fixed quantity under a specified model, while a variance estimate calculated from observed data is itself random.

These distinctions matter in actuarial estimation. A plug-in standard error may look precise even when it varies substantially from sample to sample. An estimator may also have finite variance while its usual variance estimate is biased. The following hypothetical examples use complete, independent exponential losses to make each comparison explicit. The conclusions about finite-sample behavior depend on that model; they should not be transferred without checking the assumptions for another model or data structure.

A Model With Exact Finite-Sample Results

Suppose \(X_1,\ldots,X_n\) are independent and identically distributed complete losses with an exponential distribution of mean \(\theta>0\). Thus the density is \(f_X(x)=\theta^{-1}e^{-x/\theta}\) for \(x>0\), and \(E[X]=\theta\), \(\operatorname{Var}(X)=\theta^2\). The maximum likelihood estimator of the mean parameter is the sample mean \(\hat{\theta}=\bar{X}\). The corresponding rate parameter is \(\lambda=1/\theta\), whose maximum likelihood estimator is \(\hat{\lambda}=1/\bar{X}\).

The sum \(S=\sum_{j=1}^n X_j\) has a gamma distribution with shape \(n\) and scale \(\theta\). Consequently, \(\bar{X}=S/n\) has mean \(\theta\) and variance \(\theta^2/n\). In this particular model, the asymptotic variance formula for \(\hat{\theta}\), \(\theta^2/n\), is also its exact finite-sample variance. That special agreement does not mean a variance estimate computed from one sample is exact.

Why the Plug-In Variance Estimate Is Biased

The sample mean is unbiased, but squaring it does not preserve unbiasedness. Since \(\operatorname{Var}(\hat{\theta})=\theta^2/n\) and \(E[\hat{\theta}]=\theta\), the second moment is \(E[\hat{\theta}^2]=\operatorname{Var}(\hat{\theta})+(E[\hat{\theta}])^2=\theta^2/n+\theta^2\). Dividing by \(n\) gives the expectation of the plug-in variance estimate.

The target is \(\theta^2/n\), so the plug-in variance estimate has upward bias \(\theta^2/n^2\), or relative bias \(1/n\). Its square root, the usual plug-in standard error \(\hat{\theta}/\sqrt{n}\), is unbiased in this model because \(E[\hat{\theta}]=\theta\). That does not make the estimated variance unbiased: taking a square and taking an expectation are not interchangeable.

For another view of sample-to-sample instability, the gamma distribution gives \(E[(\hat{\theta}/\theta)^4]=(n+1)(n+2)(n+3)/n^3\). Combining this with the second moment yields \(\operatorname{Var}(\hat{\theta}^2/n)=2\theta^4(n+1)(2n+3)/n^5\). Thus the estimated variance is random even when the model and sample size are fixed. Its relative bias decreases with \(n\), but a single observed estimate can still differ substantially from the true variance.

Reciprocal Estimates Can Behave Quite Differently

For the rate parameter \(\lambda=1/\theta\), the MLE is \(\hat{\lambda}=n/S\). Although the mean-parameter MLE is unbiased, the reciprocal estimator generally is not. The gamma distribution of \(S\) gives, for \(n>1\), \(E[S^{-1}]=1/[\theta(n-1)]\). For \(n>2\), it also gives \(E[S^{-2}]=1/[\theta^2(n-1)(n-2)]\). These negative moments follow by integrating the gamma density and using \(\Gamma(n)=(n-1)\Gamma(n-1)\).

The first expression shows upward finite-sample bias. The second shows that the exact variance can be much larger than the asymptotic variance \(\lambda^2/n\) when \(n\) is small. If \(n\leq2\), the reciprocal estimator does not have finite variance; for \(n=2\) its second moment diverges. An observed sample still produces a numerical plug-in standard error, but that number does not make the estimator’s sampling variance finite.

Common Exam Traps and Practical Interpretation

The principal trap is to use an asymptotic variance formula as though it necessarily described exact finite-sample variability. The exponential sample mean happens to have exact variance equal to its asymptotic variance, but its plug-in variance estimate is still biased. For the reciprocal rate estimate, even the estimator’s finite-sample variance can differ sharply from the asymptotic expression.

• Confusing a parameterization. For an exponential mean \(\theta\), the MLE is \(S/n\). For the rate \(\lambda=1/\theta\), it is \(n/S\). These are reciprocals, not interchangeable estimates.
• Confusing a standard error with a variance. A standard error is the square root of a variance. Squaring an unbiased standard error estimate need not give an unbiased variance estimate.
• Ignoring existence conditions. The reciprocal rate estimate has finite mean only when \(n>1\), and finite variance only when \(n>2\). A numerical plug-in result does not remove those conditions.
• Overstating model-specific results. The calculations assume independent, identically distributed, complete exponential observations. Censoring, truncation, dependence, or a different severity model requires its own variance analysis.

In practice, compare an estimated standard error with what the assumed sampling model implies at the actual sample size. If an exact finite-sample result is available, it can reveal bias or instability hidden by an asymptotic approximation. If it is not available, the asymptotic result should be interpreted as an approximation, with its assumptions and sample-size limitations made explicit. These distinctions are aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.