Why Zero Is a Special Value for a Variance
A random-effect variance describes between-individual variation on a specified model scale. For example, if an individual’s random effect is \(\eta_i\), its variance \(\omega^2\) measures how dispersed the \(\eta_i\) values are around their population mean. A variance cannot be negative: \(\omega^2 \geq 0\). Thus zero is not an ordinary point in the parameter space. It is a boundary, and fitting can place the estimate exactly there when the data provide little evidence of between-individual variation on that scale.
This constraint matters for profile-likelihood intervals. At an interior parameter value, the usual likelihood-ratio approximation is often a chi-square distribution with one degree of freedom. At a boundary, the likelihood can behave differently: random variation in the data can push the unconstrained estimate below zero, but the fitted estimate cannot follow it there. The resulting likelihood-ratio statistic need not have the ordinary chi-square distribution.
The nuisance parameters in the profile may include fixed effects, residual-error parameters, and other estimable model parameters. They must be reoptimized at each candidate variance; otherwise the resulting curve is not the profile likelihood. The introductory material on profile likelihood explains how this constrained optimization produces a curve. Here the key additional issue is how to interpret its cutoff when the variance can be zero.
Why the Usual Cutoff Can Be Wrong at the Boundary
For a single regular parameter at an interior value, the usual large-sample reference distribution for \(D(v)\), when testing that candidate value, is \(\chi^2_1\). Its 95th percentile is about 3.841. At a boundary, the corresponding limiting distribution for a single variance component is often instead a mixture: with probability one-half the statistic is zero, and with probability one-half it has a \(\chi^2_1\) distribution.
The mixture is not simply another name for \(\chi^2_1\). It has a point mass at zero and, for positive values, only half the tail probability of the ordinary chi-square distribution. For instance, if the boundary-mixture assumptions hold, a statistic of 2.50 has upper-tail probability \(0.5P(\chi^2_1\geq 2.50)\), not \(P(\chi^2_1\geq 2.50)\). The former is about 0.0569; the latter is about 0.1138.
A short geometric argument explains the mixture. Near the true boundary value, the unconstrained estimate behaves approximately like a centered normal quantity. If its random direction points into the allowed region, the constrained estimate can move away from zero and the likelihood-ratio statistic has the usual squared-normal form, which is \(\chi^2_1\). If it points outside the allowed region, the constrained estimate remains at zero, giving a likelihood-ratio statistic of zero. A centered normal quantity points in either direction with probability one-half, yielding the mixture.
For this mixture, a 95% quantile is about 2.706: above zero, the mixture cumulative probability is one-half plus one-half of the \(\chi^2_1\) cumulative probability, so the 95th mixture percentile is the 90th percentile of \(\chi^2_1\). This number also serves as the usual one-sided 95% likelihood-ratio cutoff for an interior scalar parameter. The numerical match does not make every 95% profile interval one-sided; the intended coverage and the candidate value being evaluated matter.
Choosing and Reading an Interval
State the inferential target before choosing a cutoff. A one-sided 95% upper confidence limit for a nonnegative variance has the form \([0,U]\) when the fitted estimate is zero. The limit \(U\) is found where the profile deviance reaches the one-sided cutoff, commonly 2.706 under the stated asymptotic conditions. It is an upper limit on the variance in the units of that variance, not on the standard deviation. The corresponding standard-deviation limit is \(\sqrt{U}\), in the original random-effect units.
A two-sided 95% profile interval usually uses the interior \(\chi^2_1\) cutoff 3.841 for positive candidate values. A boundary-aware construction can use the mixture cutoff when evaluating the boundary candidate zero, while using the appropriate interior cutoff for positive candidates. Conventions and calibration targets should be reported explicitly; the phrase “95% profile interval” alone does not say whether the intended limit is one-sided or two-sided. In particular, do not replace every cutoff on the profile curve with 2.706 merely because the parameter is a variance.
Worked Example: A One-Sided Upper Limit When the Estimate Is Zero
Suppose a hypothetical NLME fit estimates an individual random-effect variance at \(\hat v=0\), with \(v\) measured in \((\log\text{ clearance})^2\). The table gives profile deviances at selected candidate values. Assume one variance component is at the boundary, nuisance parameters are identifiable, and the one-sided asymptotic approximation is considered adequate.
| Candidate variance \(v\) | Profile deviance \(D(v)\) |
|---|---|
| 0.00 | 0.00 |
| 0.04 | 0.70 |
| 0.08 | 2.00 |
| 0.12 | 2.90 |
| 0.16 | 4.40 |
For a one-sided 95% upper limit, use the cutoff 2.706. It lies between \(D(0.08)=2.00\) and \(D(0.12)=2.90\). Linearly interpolating between these two points gives $$ U=0.08+\frac{2.706-2.00}{2.90-2.00}(0.12-0.08) =0.1114\;(\log\text{ clearance})^2. $$
Thus the approximate one-sided interval is \([0,0.1114]\;(\log\text{ clearance})^2\). The data and this method do not establish that the true variance is zero; they place an upper limit on values compatible with the specified likelihood-ratio procedure. If expressed as a standard-deviation limit, the upper endpoint is \(\sqrt{0.1114}=0.3338\) log-clearance units. This interpolation is an approximation to the profile crossing; a denser constrained-optimization grid can locate it more accurately.
Worked Example: Checking Whether Zero Is Included
In a second hypothetical fit, a random-effect variance estimate is positive, \(\hat v=0.09\;(\text{mg/L})^2\). At the boundary candidate \(v=0\), the profiled deviance is \(D(0)=2.20\). Assume the single-component boundary mixture is applicable. Its 95% cutoff is 2.706, so $$ D(0)=2.20<2.706. $$
Under this boundary-aware comparison, zero is not excluded at the 5% level. Suppose the profile deviance reaches the ordinary interior 95% cutoff, 3.841, at \(v=0.22\;(\text{mg/L})^2\), and remains below the relevant cutoff between zero and that crossing. A boundary-aware interval is then approximately \([0,0.22]\;(\text{mg/L})^2\). It includes zero even though the maximum likelihood estimate is positive.
This interval does not imply that the probability the variance equals zero is 95%, nor does it show that zero is the most likely value. It says that zero and the values up to the upper endpoint have not been ruled out by the stated likelihood-ratio construction. A different target, such as a two-sided interval with a particular coverage convention, should be labelled and constructed accordingly.
Assumptions, Diagnostics, and Alternatives
The mixture approximation is most defensible when there is one variance component at the boundary, the remaining parameters are identifiable and well behaved, and the likelihood is adequately approximated by its large-sample form. NLME data can strain these conditions: a small number of individuals, weak information about between-individual variation, complicated random-effects structures, or a poorly identified nuisance parameter can make the profile asymmetric or irregular. When a random-effect variance is zero, some associated quantities may also become weakly identified; this is a warning that the simple single-boundary reference distribution may not apply.
Inspect the fitted profile rather than relying only on the reported endpoint. Check that the optimization converged at each fixed variance, that nuisance parameters were reoptimized, and that the profile changes smoothly near the relevant crossing. A flat profile, abrupt jumps, multiple apparent optima, or limits that change substantially with optimizer settings indicate that the numerical or asymptotic interval may be unreliable. A variance estimate pinned at zero is itself a reason to scrutinize the boundary behavior, not proof that the model is adequate.
When the asymptotic assumptions are doubtful, a parametric bootstrap can provide a useful calibration check. Simulate data under a clearly specified fitted model, preserve the design, and refit using the same constraints; then assess how often the resulting variance estimates or profile statistics behave as the asymptotic method predicts. The bootstrap also has limitations: finite replicate counts create Monte Carlo error, and failed fits or a misspecified simulation model can distort calibration. Report the calibration method and its assumptions rather than presenting an adjusted endpoint as universally exact.
| Approach | Useful feature | Main caution near zero |
|---|---|---|
| Wald interval | Simple when the estimate is well inside the parameter space and approximately normal. | A symmetric interval can extend below zero and may be poor when the estimate is on the boundary. |
| Profile likelihood | Uses the fitted likelihood shape and permits asymmetric limits. | The cutoff must match the boundary situation and the intended one- or two-sided coverage. |
| Parametric bootstrap | Can reflect finite-sample behavior under a specified model and fitting procedure. | Results depend on the simulation model, successful fitting, and the number of replicates. |
A frequent mistake is to see a variance estimate of zero, apply the usual two-sided \(\chi^2_1\) cutoff automatically, and describe the result as a boundary-corrected 95% upper limit. That combines different inferential targets: 3.841 is the conventional two-sided 95% interior cutoff, whereas 2.706 is the one-sided 95% cutoff and the 95% quantile of the single-boundary mixture. State which interval is intended, verify the assumptions, and interpret its endpoint on the variance scale.
Confirm that the target is a variance constrained to be nonnegative, and note whether its estimate is exactly zero or near zero.
Decide whether the requested result is a one-sided upper limit or a two-sided profile interval, and state the coverage convention.
Reoptimize nuisance parameters, check convergence and profile shape, and assess whether the single-boundary approximation is plausible.
Give the cutoff, variance units, boundary treatment, and any bootstrap calibration or numerical limitations.
Check Your Understanding
Answer each question, then compare with the solutions.
- Why can an NLME random-effect variance be estimated exactly at zero?
- Under the single-boundary mixture approximation, what is the 95% cutoff, and why is it not 3.841?
- A one-sided profile limit uses cutoff 2.706. At \(v=0.10\), the profile deviance is 2.4; at \(v=0.14\), it is 3.0. Estimate the upper limit by linear interpolation.
- Name two signs that the asymptotic profile interval may be unreliable.
Solutions. (1) The variance is constrained to be nonnegative, so zero is an allowed boundary value; if the likelihood is maximized there, the estimate is exactly zero. (2) The cutoff is about 2.706 because the 95th percentile of the mixture \(\frac12\delta_0+\frac12\chi^2_1\) is the 90th percentile of \(\chi^2_1\); 3.841 is the usual 95th percentile for an interior, two-sided \(\chi^2_1\) procedure. (3) The crossing is \(0.10+\{(2.706-2.4)/(3.0-2.4)\}(0.14-0.10)=0.1204\), in the same variance units as \(v\). (4) Examples include failed or unstable constrained fits, a flat or discontinuous profile, and weak identification of nuisance parameters.