Statistical Information and Precision

Why Precision Needs a Defined Scale

A result can be reported as an estimate, but its value alone does not tell us how precisely it has been estimated. If repeated samples under the same model would produce estimates clustered tightly around a target, the estimator is precise; if the estimates would vary widely, it is less precise. In group sequential work, this distinction matters because information is used to describe how much precision an estimate has at a particular analysis. The earlier material on canonical model assumptions and diagnostics emphasizes that such interpretations depend on the data-generating assumptions being reasonable.

For a scalar parameter, one useful measure of precision is the reciprocal of the estimator’s variance. This is an estimator-specific, variance-based definition: it does not by itself describe every meaning of “statistical information” used in statistics. In particular, it should not be confused with a general assessment of study quality, or with a likelihood-based measure that may be introduced separately.

Definition: Let $\hat{\theta}$ be an estimator of a scalar parameter $\theta$, and let $\operatorname{Var}_{\theta}(\hat{\theta})$ denote its variance under the model when the parameter value is $\theta$. If $0<\operatorname{Var}_{\theta}(\hat{\theta})<\infty$, its variance-based information at $\theta$ is $$ I_{\hat{\theta}}(\theta)=\frac{1}{\operatorname{Var}_{\theta}(\hat{\theta})}. $$ The restriction to a positive, finite variance is part of the definition. A zero variance has no reciprocal under this definition, and an infinite or undefined variance does not yield finite variance-based information.

The variance is measured in squared units of the estimator, so its reciprocal has inverse-squared units. If $\hat{\theta}$ measures a treatment effect in milligrams, then $I_{\hat{\theta}}(\theta)$ is measured in inverse milligrams squared. The standard error is $SE(\hat{\theta})=\sqrt{\operatorname{Var}_{\theta}(\hat{\theta})}$; therefore, when the variance is positive and finite,

I_{\hat{\theta}}(\theta)=\frac{1}{SE(\hat{\theta})^2}, \qquad SE(\hat{\theta})=\frac{1}{\sqrt{I_{\hat{\theta}}(\theta)}}.

Thus, for estimators of the same target on the same scale, a larger variance-based information corresponds exactly to a smaller variance and standard error. Information is not itself an estimate of $\theta$, nor is it a probability that an estimate is correct.

Assumptions and Scope of the Comparison

The variance in the definition is a repeated-sampling property under a specified model. To use it, we need an estimator with a well-defined sampling distribution and a variance that is positive and finite at the parameter value of interest. A numerical variance also depends on the model assumptions used to calculate it. For example, an independence assumption can be essential: if observations are correlated but treated as independent, the calculated variance may be too small and the corresponding information too large.

A comparison is interpretable when the estimators target the same scalar parameter, use the same measurement scale, and are evaluated under clearly stated models and sampling conditions. Even then, variance alone does not capture every aspect of estimator performance. An estimator can have small variance but be biased. If bias matters, mean squared error (MSE) includes both variance and squared bias:

\operatorname{MSE}_{\theta}(\hat{\theta}) = \operatorname{Var}_{\theta}(\hat{\theta}) + \left(\operatorname{E}_{\theta}[\hat{\theta}]-\theta\right)^2.

This identity follows by writing $\hat{\theta}-\theta$ as its centered random deviation plus its bias, then taking the expected square; the cross-product has expectation zero because the centered deviation has mean zero. Variance-based information describes only the first term. It is therefore most directly a precision measure, not a complete ranking of estimators.

Calculating and Comparing Information

Two familiar estimation problems show how the definition works. In each case, the model assumptions are stated before the variance formula is used. The formulas give information about an estimator’s sampling precision, not about the probability that a particular parameter value is true.

Worked Example: Information for a Sample Mean

Suppose a hypothetical sample consists of $n=36$ independent observations from a population with mean $\mu$ and known variance $\sigma^2=144$ square units. The estimator is the sample mean $\bar{x}$. The assumptions for this calculation are that the observations are independent, have the same finite variance, and each has mean $\mu$. The variance of their mean is then $\sigma^2/n$.

Substituting the given values,

\operatorname{Var}(\bar{x})=\frac{144}{36}=4\ \text{square units}, \qquad SE(\bar{x})=\sqrt{4}=2\ \text{units}.

The variance is positive and finite, so the variance-based information is defined:

I_{\bar{x}}(\mu)=\frac{1}{4}=0.25\ \text{per square unit}.

If the hypothetical sample size were instead $n=144$, with the same population variance and assumptions, the variance would be $144/144=1$ square unit and the information would be $1$ per square unit. The information is four times as large, while the standard error is half as large, $1$ unit rather than $2$. This illustrates the inverse-square relationship between information and standard error. The comparison concerns precision under the stated model; it does not establish that the observations are representative or that the model is appropriate.

Worked Example: Information for a Sample Proportion

In a hypothetical study, $n=200$ independent participants each have a binary outcome with the same probability $p$ of a positive outcome. Let $X$ be the number of positive outcomes and $\hat{p}=X/n$. Under the independent Bernoulli model, $X\sim\operatorname{Bin}(n,p)$, so $\operatorname{Var}(\hat{p})=p(1-p)/n$. Assume the true value for this calculation is $p=0.30$.

The true variance and information are

\operatorname{Var}_{p}(\hat{p}) = \frac{0.30(1-0.30)}{200} = \frac{0.21}{200} = 0.00105, \qquad I_{\hat{p}}(p)=\frac{1}{0.00105}\approx 952.4.

The variance is positive and finite because $0<p<1$ and $n$ is positive and finite. The information is $952.4$ per squared proportion unit; proportions are dimensionless, but stating the squared scale makes clear that this is a reciprocal variance.

For illustration, suppose the observed count is $X=62$, so $\hat{p}=62/200=0.31$. Since $p$ is usually unknown, an estimated standard error may substitute $\hat{p}$ for $p$. That plug-in calculation gives

\widehat{\operatorname{Var}}(\hat{p}) = \frac{0.31(1-0.31)}{200} = \frac{0.2139}{200} = 0.0010695, \qquad \widehat{I}=\frac{1}{0.0010695}\approx 935.0.

The value $935.0$ is a plug-in estimate of the variance-based information, not the true information at $p=0.30$. The distinction matters: a variance evaluated at the unknown parameter and a variance estimated from observed data are not the same quantity.

Why Reciprocal Variance Has a Precision Interpretation

The interpretation follows directly from standardization. If an estimator has positive finite variance $V$, its standard error is $\sqrt{V}$, the scale of its repeated-sampling fluctuations around its expectation. Replacing $V$ by its reciprocal gives $1/V$; consequently, decreasing $V$ increases $1/V$, and vice versa. More precisely, if two estimators of the same target have variances $V_1$ and $V_2$, both positive and finite, then

V_2<V_1 \quad\Longleftrightarrow\quad \frac{1}{V_2}>\frac{1}{V_1}.

Their standard errors change by the square root of the variance ratio, not by the variance ratio itself. For example, if one estimator has four times the variance of another, it has half the information and twice the standard error. This is why information ratios and standard-error ratios should not be interchanged.

A simple combination of independent estimates makes the precision comparison concrete. Suppose two hypothetical estimators $A$ and $B$ are unbiased for the same scalar effect $\theta$, are independent, and have known variances $4$ and $9$ square units. Independence is needed for the variance calculation below; unbiasedness ensures both estimate the same target without a systematic offset. The following derivation finds the minimum-variance linear combination $T_w=wA+(1-w)B$.

\begin{aligned} \operatorname{Var}(T_w) &=4w^2+9(1-w)^2,\\ \frac{d}{dw}\operatorname{Var}(T_w) &=8w-18(1-w)=26w-18,\\ 26w-18=0 &\quad\Longrightarrow\quad w=\frac{9}{13}. \end{aligned}

The second derivative is $26>0$, so this stationary point is a minimum. For the illustrative observed estimates $A=12$ and $B=15$, the resulting combined estimate is $T=(9A+4B)/13$. Its variance and information can be checked directly:

T=\frac{9(12)+4(15)}{13}=\frac{168}{13}\approx 12.92\ \text{units}, \qquad \operatorname{Var}(T)=\frac{36}{13}\approx 2.769\ \text{square units}, \qquad I_T=\frac{13}{36}\approx 0.3611\ \text{per square unit}.

The information for $A$ alone is $1/4=0.25$ per square unit, so the combined estimator has greater variance-based information under the stated assumptions. This conclusion depends on the estimates being unbiased, independent, and correctly assigned the stated variances. If their covariance were nonzero, the variance formula would need an additional covariance term.

Diagnostics, Alternatives, and Limitations

The reciprocal-variance calculation is only as trustworthy as its variance input. Useful checks include asking whether the estimator targets the intended parameter, whether its variance formula matches the sampling design, and whether dependence, clustering, missing observations, or model misspecification have been ignored. In practice, compare model-based and design-aware standard errors when both are available, and check whether estimated variances are stable under reasonable alternative assumptions. A variance estimate near zero can produce an enormous reciprocal; that numerical result is not automatically evidence of extraordinarily strong data.

Several quantities can be relevant, but they answer different questions:

Quantity	What it describes	Important qualification
Variance-based information	The reciprocal of an estimator’s variance	Estimator-specific; requires positive, finite variance
Standard error	The square root of an estimator’s variance	Has the same units as the estimator
Mean squared error	Variance plus squared bias	Assesses error around the target, including bias
Likelihood-based information	A model-based quantity associated with how a likelihood responds to parameter changes	Not defined here as simply the reciprocal variance of an arbitrary estimator

The distinction between variance-based and likelihood-based information is especially important. Under additional conditions, particular estimators and model-based information quantities can be closely related, but that does not make the reciprocal variance a universal definition of information. Nor does higher variance-based information guarantee better coverage, less bias, or more reliable conclusions under model failure.

In sequential settings, an estimate at each analysis can have its own variance and corresponding reciprocal, provided each variance is positive and finite under the model. Comparing these values describes marginal precision at those analyses; it does not, on its own, establish the joint dependence between estimates at different analyses or justify a sequential testing procedure. Those questions require their own assumptions and calculations.

Common Mistakes

Taking the reciprocal without checking the variance. The definition requires $0<\operatorname{Var}_{\theta}(\hat{\theta})<\infty$. A constant estimator has zero variance, for which this reciprocal is undefined.
Calling a plug-in value the true information. If an unknown parameter is replaced by an estimate in the variance formula, the result is estimated information and should be labelled accordingly.
Comparing unlike targets or scales. A reciprocal variance has units and is specific to its estimator. A larger numerical value is not automatically a fair precision comparison when estimands or scales differ.
Ignoring bias or dependence. Small variance does not ensure small total error, and independence-based variance calculations can be wrong when estimates are correlated.
Interpreting information as certainty. Information summarizes variance under a model; it is not the probability that a hypothesis is true or that an estimate equals the parameter.

Key takeaway: For a scalar estimator with positive finite variance, variance-based information is $1/\operatorname{Var}_{\theta}(\hat{\theta})$. It is an estimator-specific measure of precision under a stated model. Check the variance conditions, units, bias, and dependence before comparing information values.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

An estimator has variance $25$ square units. What are its variance-based information and standard error, including units?
Why is variance-based information not defined by the reciprocal-variance formula for an estimator that is constant under the model?
For $n=100$ independent Bernoulli observations with $p=0.20$, calculate the variance and information of the sample proportion.
Two estimators target the same parameter and have positive finite variances $V_A=2$ and $V_B=8$. Which has more variance-based information, and by what factor?

Solutions. (1) The information is $1/25=0.04$ per square unit, and the standard error is $\sqrt{25}=5$ units. (2) A constant estimator has variance zero, so $1/\operatorname{Var}(\hat{\theta})$ would require division by zero; the stated definition requires strictly positive variance and assigns no reciprocal-variance value in this case. (3) Under the independent, common-$p$ Bernoulli model, $\operatorname{Var}(\hat{p})=0.20(0.80)/100=0.0016$, so the information is $1/0.0016=625$ per squared proportion unit. (4) Estimator $A$ has information $1/2=0.5$, while $B$ has $1/8=0.125$, in reciprocal squared units of the parameter. Thus $A$ has four times the variance-based information of $B$, and its standard error is half as large.