Information Near Parameter Boundaries

Why a Boundary Changes the Information Story

Fisher information is often used to approximate estimator uncertainty: near a regular interior parameter value, the maximum-likelihood estimator is approximately normal, with covariance related to the inverse information matrix. But a variance component cannot be negative. When its true value is zero, the parameter is at the edge of its permissible range, and the usual approximation can fail even when the Fisher information is finite and well-conditioned.

This matters in model-informed design because an information-based standard error may look numerically routine while the likelihood is asymmetric, the estimate accumulates at zero, and symmetric intervals extend into impossible values. The issue is not simply numerical conditioning. It is a mismatch between the boundary-constrained likelihood and an approximation that treats the parameter as if it could vary freely in both directions.

A Scalar Variance-Component Model

Consider \(G\) independent groups, each with \(r\) measurements. Within group \(j\), let \(Y_{jk}=\mu+b_j+\epsilon_{jk}\), where \(b_j\sim N(0,\tau^2)\) is a group-level random effect and \(\epsilon_{jk}\sim N(0,\sigma^2)\) is measurement error. Assume the random effects and errors are mutually independent, \(\mu\) and \(\sigma^2\) are known, and \(r\) is fixed. These assumptions isolate the boundary issue; they are not a general recipe for estimating all components of a mixed model.

The group mean, centered at \(\mu\), is \(Z_j=\bar Y_j-\mu\). Under this model, the \(Z_j\) are independent and normally distributed with mean zero and variance \(V=V_0+\tau^2\), where \(V_0=\sigma^2/r\). Within-group contrasts do not depend on \(\tau^2\), so they contribute no information about this component. Write \(S=\sum_{j=1}^G Z_j^2\). Ignoring constants that do not depend on \(\tau^2\), the log-likelihood is

The constraint is part of the model, not an optional adjustment after estimation. Differentiating on the variance scale gives the score and its expectation:

For the last equality, under the model \(E(S)=GV\) and \(\operatorname{Var}(S)=2GV^2\), because \(S/V\) has a chi-squared distribution with \(G\) degrees of freedom. Substitution into the variance of the score gives \(I=2GV^2/(4V^4)=G/(2V^2)\). Thus the expected information at \(\tau^2=0\) is \(G/(2V_0^2)\), a finite positive number. That fact alone does not establish the ordinary normal approximation: the true value is still on the boundary.

How the Boundary Alters the Approximation

The estimate follows directly from the likelihood. If the unconstrained maximizing value \(S/G-V_0\) is nonnegative, it is also feasible and maximizes the constrained likelihood. If it is negative, then \(S/G<V_0\), so the score at zero is negative: \(U(0)=(S-GV_0)/(2V_0^2)<0\). The likelihood decreases as \(\tau^2\) increases from zero, making zero the constrained maximum. Combining these cases gives the expression above.

At a regular interior point, a local quadratic likelihood and an unconstrained estimator typically lead to an approximately normal estimate. At a single lower boundary, a useful simple-model heuristic is instead a projected normal approximation: the unconstrained local estimate fluctuates on both sides of zero, but the constrained estimate cannot go below zero. In the one-component setting, under suitable regularity conditions for nuisance parameters, this leads to a likelihood-ratio statistic with limiting distribution \(0.5\chi^2_0+0.5\chi^2_1\) under the boundary null. Here \(\chi^2_0\) denotes a point mass at zero. This result is not universal for every random-effects model; multiple or interacting boundary components can have different limiting distributions.

A short justification explains the mixture. Locally, the unconstrained estimate has an approximately symmetric normal fluctuation around the true boundary value. About half of those fluctuations point into the feasible region and yield a positive estimate; the other half point outside it and are projected onto zero. For a likelihood-ratio statistic, the projected cases contribute zero, while the positive cases contribute the usual one-degree-of-freedom chi-squared limit. This gives the stated half-at-zero, half-chi-squared distribution under the stated single-boundary conditions. The argument depends on the local approximation and should not be treated as an exact finite-sample result.

What an Information-Based Standard Error Misses

The inverse expected information supplies a local variance approximation on the variance scale. In the example, at the boundary \(I(0)=G/(2V_0^2)=4/2=2\) per fourth-power unit, so the corresponding standard error is \(1/\sqrt{2}=0.707\) squared units. A symmetric normal interval centered on the estimate zero would extend below zero. Replacing its lower endpoint by zero does not, by itself, make the interval have its advertised coverage: the shape and boundary behavior of the estimator's sampling distribution have not been addressed by that clipping operation.

Even a positive estimate can be close enough to the boundary for a symmetric approximation to be misleading. For illustration, suppose four hypothetical centered group means are \(\sqrt{2.5},-\sqrt{2.5},0,0\) units. Then \(S=5\) squared units and \(\hat{\tau}^2=5/4-1=0.25\) squared units. At this estimate, the plug-in information is \(I=4/[2(1.25)^2]=1.28\) per fourth-power unit and the plug-in standard error is \(1/\sqrt{1.28}=0.884\) squared units. The nominal symmetric 95% Wald interval is approximately \(0.25\pm1.96(0.884)=[-1.48,1.98]\) squared units. The negative endpoint is impossible, and the interval's symmetry is at odds with the constrained estimate. With only four groups, the large-sample approximation is also especially difficult to justify.

Diagnostics, Alternatives, and Design Implications

Warning signs include estimates repeatedly landing exactly at zero across simulated data sets, strongly asymmetric profile likelihoods, intervals whose lower endpoints are negative, and substantial disagreement between Wald intervals and likelihood-based or simulation-based intervals. A finite, invertible information matrix does not rule out any of these problems. Conversely, a zero estimate in one data set alone does not prove that the model is misspecified; it may be a plausible outcome under a small or weakly informative design.

Several approaches answer different questions:

For design work, Fisher information can remain a useful local summary when the anticipated parameter value is safely interior and the likelihood is regular. Near a plausible zero component, supplement it with simulations that reproduce the constraints and the proposed analysis, or examine likelihood behavior over plausible component values. This is a diagnostic for uncertainty and operating behavior, not a substitute for checking model identifiability or the adequacy of the assumed random-effects structure.

Common Mistake

Wrong reasoning: “The information at zero is finite and positive, so the maximum-likelihood estimate must be approximately normal and the usual Wald standard error is valid.” The information calculation establishes a local curvature or score-variance quantity; it does not remove the constraint \(\tau^2\geq0\), make zero an interior point, or guarantee the usual estimator distribution.

Correction: Check whether the true or fitted value is near a boundary, whether the likelihood is asymmetric, and whether the estimator is frequently constrained at zero. If so, use an inference or simulation method that respects the boundary and state its assumptions. Treat inverse Fisher information as a local approximation, not an automatic guarantee of calibrated uncertainty.