Why Invert a Test?
A confidence interval for a fixed-effect coefficient can be viewed as a collection of hypothesis-test results. Choose a candidate value \(b_0\) for the coefficient and test \(H_0:\beta_j=b_0\). Values that the test does not reject form the confidence set. This connection is useful because it makes the interval reflect the particular test used: inverting a Wald test gives a Wald interval, while inverting a likelihood-ratio test gives a likelihood-ratio interval.
The two intervals need not agree. Wald inference uses the estimated coefficient and its standard error at the unrestricted fit, while likelihood-ratio inference compares the maximized likelihood with the likelihood under each constrained value. The distinction matters when the likelihood is asymmetric or the estimate is imprecise.
Model and Inversion Principle
For a GLMM, responses are modelled conditionally on random effects. A typical specification is \(Y_{ij}\mid b_i\) following a response distribution from an exponential-family model, with conditional mean linked to predictors by \(g\{E(Y_{ij}\mid b_i)\}=x_{ij}^{\mathsf T}\beta+z_{ij}^{\mathsf T}b_i\). The random effects \(b_i\) have a specified distribution, commonly with mean zero and covariance parameters collected in \(\psi\). The likelihood used for fixed-effect inference is the marginal likelihood, obtained by integrating over the random effects.
The procedures below assume that the fixed-effect and random-effect distributions, link, and dependence structure are adequately specified; observations or clusters have the independence structure required by the model; and the likelihood fits converge to the intended solutions. The usual chi-square and normal calibrations are large-sample approximations, relying on regularity conditions such as an identifiable model and a well-behaved likelihood. Both intervals here concern one scalar coefficient and use a two-sided test at level \(\alpha\).
Wald Confidence Interval
Let \(\hat{\beta}_j\) be the fitted coefficient and let \(SE(\hat{\beta}_j)\) be its estimated standard error from the unrestricted GLMM fit. The conventional two-sided Wald statistic for a candidate value \(b_0\) is
At level \(\alpha\), reject when \(\lvert Z_W(b_0)\rvert>z_{1-\alpha/2}\), where \(z_{1-\alpha/2}\) is the corresponding standard-normal quantile. Inverting that rule gives
Thus the Wald interval is centered at the estimate and is symmetric on the coefficient scale. This algebra is the justification for the interval: its endpoints are exactly the candidate values at which the absolute Wald statistic reaches the critical value. This conventional version uses the unrestricted-fit standard error for every candidate \(b_0\). A procedure that instead recalculates a null-based standard error defines a different test and must be inverted according to that test's own rule.
Worked Example: Wald Interval for a Logistic GLMM Coefficient
Suppose a hypothetical logistic random-intercept GLMM estimates a treatment coefficient of \(\hat{\beta}_{\text{treat}}=0.480\), with standard error \(0.200\). Treatment is coded 1 and control 0. Assume the model converged, the random-intercept structure is the one specified for the analysis, and the normal approximation for this fixed-effect test is adequate.
For a 95% interval, \(z_{0.975}=1.96\). The margin of error is \(1.96(0.200)=0.392\). Therefore,
The interval is for the treatment coefficient on the log-odds scale. Exponentiating its endpoints gives the corresponding interval for the conditional odds ratio: \(\{\exp(0.088),\exp(0.872)\}=(1.092,\ 2.392)\), to three decimal places. Under the fitted model, this is an approximate 95% interval for the treatment-versus-control odds ratio conditional on the same random-intercept value. It is not a claim that there is a 95% probability that the fixed, unknown coefficient lies in this particular interval.
Likelihood-Ratio Confidence Interval
For likelihood-ratio inversion, define the profile log-likelihood for \(\beta_j\) by maximizing over all nuisance parameters at each candidate value. Nuisance parameters include the other fixed effects and the random-effect and dispersion parameters. Write
Here \(\ell(\hat{\beta})\) denotes the maximized log-likelihood under the unrestricted model, including maximization over nuisance parameters. Under the regular fixed-effect likelihood-ratio approximation, \(D(b_0)\) has an approximate \(\chi^2_1\) reference distribution when \(H_0:\beta_j=b_0\) holds. Inverting the level-\(\alpha\) test gives the likelihood-ratio confidence set
For a 95% interval, \(\chi^2_{1,0.95}=3.841\). The endpoints are found where the profile deviance \(D(b_0)\) crosses 3.841. Unlike the Wald interval, this set can be asymmetric around the estimate because the profile log-likelihood need not be a symmetric quadratic. The likelihood-ratio test for a fixed effect should compare the same data and random-effects structure in the constrained and unrestricted fits, with nuisance parameters re-estimated in both. For such fixed-effect comparisons, use the appropriate ordinary maximum-likelihood likelihood, not a restricted likelihood comparison that changes with the fixed-effect specification.
Worked Example: Finding Profile-Likelihood Endpoints
Consider a hypothetical GLMM with a fitted coefficient near \(0.52\). Suppose constrained fits, with nuisance parameters re-estimated at each fixed coefficient, give the following profile-deviance values. Assume the fits converge and the regular \(\chi^2_1\) approximation is suitable.
| Candidate \(b_0\) | 0.10 | 0.15 | 0.90 | 0.95 |
|---|---|---|---|---|
| \(D(b_0)\) | 4.50 | 3.20 | 3.10 | 4.40 |
The 95% acceptance cutoff is 3.841. On the lower side, the crossing lies between \(0.10\) and \(0.15\). Using linear interpolation between the two displayed profile values gives
On the upper side, the crossing lies between \(0.90\) and \(0.95\). Interpolation gives
So the interpolated 95% likelihood-ratio interval is approximately \((0.125,\ 0.929)\). These are numerical approximations to the roots of \(D(b_0)=3.841\); a final analysis should locate the roots using a sufficiently fine profile or a root-finding procedure, not rely on coarse linear interpolation. The interval is asymmetric around \(0.52\), which is allowed: its endpoints are determined by the profile likelihood, not by a fixed margin on either side of the estimate.
Why the Inversion Works
The common logic can be stated directly. A level-\(\alpha\) test has a rejection rule specifying which candidate null values are rejected. For the Wald test, non-rejection means the absolute standardized distance from the estimate does not exceed its critical value. Solving that inequality for \(b_0\) yields the Wald interval. For the likelihood-ratio test, non-rejection means the likelihood loss relative to the unrestricted fit is no greater than the chi-square cutoff. The set of all such values is the likelihood-ratio confidence set.
Set \(H_0:\beta_j=b_0\), holding the data and model structure fixed.
For a Wald test, calculate its standardized distance. For a likelihood-ratio test, fit the constrained model and calculate \(D(b_0)\).
The set of values that pass the test's level-\(\alpha\) rule is its inverted confidence set.
This is also why an interval and its corresponding two-sided test agree: a candidate null value is included exactly when that test does not reject it, subject to the same approximation and test definition. A different test can yield a different interval without either calculation being internally inconsistent.
Worked Example: Wald Inversion on a Log-Rate Scale
In a hypothetical Poisson GLMM with a random intercept, suppose a predictor's fitted coefficient is \(-0.300\) with standard error \(0.120\). Assume the model is correctly specified and the Wald approximation is reasonable. For a 95% interval, the margin is \(1.96(0.120)=0.2352\), so the coefficient interval is
The coefficient is on the log-rate scale. Exponentiating gives a rate-ratio interval of approximately \((\exp(-0.5352),\exp(-0.0648))=(0.585,\ 0.937)\). Under this model, for a one-unit increase in the predictor and conditional on the same random-effect value, the rate is estimated to be multiplied by a factor in this interval. As an inversion check, test the candidate coefficient \(b_0=-0.100\):
The Wald test does not reject this candidate at the two-sided 5% level, and \(-0.100\) lies inside the interval. This agreement is required when the same conventional Wald test and cutoff are used.
Choosing and Checking an Interval
A Wald interval is quick to calculate and easy to communicate, but it depends heavily on a local normal approximation. It is symmetric on the coefficient scale even when the likelihood is noticeably skewed. A profile-likelihood interval uses the likelihood over a broader range of candidate values and can reflect that asymmetry, at the cost of repeated constrained fits and potentially greater numerical difficulty. The introductory material on profile likelihood tests provides the test framework; here the key step is to retain every candidate coefficient whose profile deviance stays below the cutoff.
| Feature | Wald interval | Likelihood-ratio interval |
|---|---|---|
| Information used | Estimate and unrestricted-fit standard error | Unrestricted and constrained profile likelihoods |
| Typical shape | Symmetric on coefficient scale | May be asymmetric |
| Computational work | Low after fitting the model | Repeated constrained fits or a profile calculation |
| Main vulnerability | Poor normal approximation or unstable standard error | Poor likelihood calibration, unreliable optimization, or unstable profile |
Neither interval is automatically reliable merely because software returns endpoints. Check that the unrestricted and constrained fits converged, that estimates are identifiable, and that the standard error or profile curve is stable under sensible numerical settings. For a profile interval, inspect the curve around the proposed endpoints: it should cross the cutoff in a reproducible way, and the constrained fits should not show unexplained jumps. A nearly flat profile can signal weak information and very wide intervals. A profile that does not cross the cutoff on one side may produce an unbounded confidence set rather than a finite endpoint.
The usual \(\chi^2_1\) calibration is approximate and depends on regularity. Small samples, sparse outcomes, severe separation, or nuisance parameters near problematic boundaries can undermine it. In such cases, investigate the adequacy of the approximation and consider an appropriate alternative calibration, such as a parametric bootstrap. A bootstrap interval is not automatically exact; its validity also depends on a suitable fitted model and implementation.
Common Pitfalls
- Using the unrestricted fit for every likelihood-ratio candidate. The constrained likelihood must be maximized over nuisance parameters at each \(b_0\). Holding those parameters fixed generally gives the wrong profile deviance.
- Assuming every interval must be symmetric. Symmetry is a feature of the conventional Wald interval on the coefficient scale, not a requirement for a likelihood-ratio interval.
- Changing the test while claiming to invert it. The confidence set must use the same test statistic and calibration as the stated test. Different Wald variants need not invert to the same interval.
- Interpreting a 95% interval as a 95% probability statement about the fixed coefficient. The coverage statement concerns the long-run performance of the procedure over repeated samples under its assumptions.
- Ignoring the model scale. A coefficient in a logistic GLMM is on a conditional log-odds scale; exponentiating it gives a conditional odds ratio. State the scale and predictor change when interpreting the interval.
Check Your Understanding
Answer each question, then check the solutions. All numerical settings below are hypothetical.
- A fitted coefficient is \(0.300\) with standard error \(0.100\). Find its 95% conventional Wald interval using \(z_{0.975}=1.96\).
- For a likelihood-ratio test with one degree of freedom at level 0.05, a candidate value has \(D(b_0)=4.20\). Is it included in the inverted 95% confidence set?
- Why must nuisance parameters be re-estimated when calculating a profile-likelihood interval?
- A Wald interval for a log-rate coefficient is \((-0.40,-0.10)\). Give the corresponding rate-ratio interval, to three decimal places.
Solutions. (1) The margin is \(1.96(0.100)=0.196\), so the interval is \(0.300\pm0.196=(0.104,\ 0.496)\). (2) No. The 5% cutoff is \(\chi^2_{1,0.95}=3.841\), and \(4.20>3.841\), so the candidate is rejected and excluded. (3) The profile likelihood at a candidate is the largest likelihood achievable under that constraint; maximizing over nuisance parameters gives that value and makes the comparison with the unrestricted fit valid. (4) Exponentiate both endpoints: \((\exp(-0.40),\exp(-0.10))=(0.670,\ 0.905)\). This interval concerns the multiplicative rate change for a one-unit predictor increase, conditional on the random effect.