Why Profile Intervals Are Often Asymmetric

Why Compare the Two Kinds of Interval?

A Wald interval uses the estimated coefficient and its estimated standard error to place equal distances on either side of the estimate. A likelihood-ratio profile interval instead finds the coefficient values whose likelihood is not too far below its maximum. These procedures can give different answers when the log-likelihood is noticeably curved differently on its two sides.

This is especially visible with sparse binomial data. When the observed number of events is small, plausible probabilities can extend a long way toward zero, while the likelihood may fall more sharply in the other direction. On the logit scale, that unequal shape can yield unequal endpoint distances. The comparison below isolates this effect in a one-coefficient logistic regression.

The Wald formula is symmetric by construction: it uses one standard error at the estimate for both directions. A profile interval need not be symmetric because it follows the likelihood’s actual shape. For a nominal 95% likelihood-ratio interval in a regular one-parameter setting, the usual large-sample cutoff is \(c=\chi^2_{1,0.95}\approx3.8415\). This asymptotic cutoff does not guarantee exact 95% coverage in a small or sparse sample; interval shape and coverage calibration are separate issues.

A Sparse Binomial Logistic-Regression Example

Consider a binomial response \(Y\sim\operatorname{Binomial}(10,p)\) with two observed successes. Fit the logistic regression model \(\operatorname{logit}(p)=\beta x\), with the covariate \(x=1\) for this group and no intercept. Thus \(\beta\) is the group’s log odds, and the model has one coefficient. This deliberately simple setup makes the likelihood shape visible without introducing a nuisance parameter.

The probability-scale endpoints help explain the imbalance. The data estimate a success probability of 0.2, but the likelihood-ratio set reaches down to about 0.036 and up to about 0.499. The logit transformation expresses those values as coefficients, but the main point here is the shape of the likelihood around its maximum: equal changes in \(\beta\) in opposite directions do not produce equal likelihood drops.

Curvature Explains Local Symmetry; Skewness Explains Its Failure

The local curvature at the maximum determines the Wald standard error. If the profile log-likelihood were exactly quadratic, its likelihood-ratio interval would be symmetric and would agree with the Wald interval when both use the same cutoff. The following result makes that comparison precise.

Proof. Since \(\hat\beta\) is an interior differentiable maximum, \(\ell_p'(\hat\beta)=0\). Taylor’s theorem gives \(\ell_p(\hat\beta+\delta)=\ell_p(\hat\beta)+\ell_p'(\hat\beta)\delta+\tfrac12\ell_p''(\hat\beta)\delta^2+o(\delta^2)\). Substituting the zero first derivative and \(\ell_p''(\hat\beta)=-J\) yields \(2\{\ell_p(\hat\beta)-\ell_p(\hat\beta+\delta)\}=J\delta^2+o(\delta^2)\). Ignoring the smaller-order remainder gives \(J\delta^2=c\), whose two solutions are \(\delta=\pm\sqrt{c/J}\). For a Wald interval, the local standard error is \(1/\sqrt J\), so the half-width is \(z/\sqrt J\). \(\square\)

This result describes a local approximation, not a promise of symmetry at every confidence level. To see the first term that can break the symmetry, suppose additionally that \(\ell_p\) has four continuous derivatives near \(\hat\beta\), and write \(K=\ell_p'''(\hat\beta)\). Taylor expansion gives \(2\{\ell_p(\hat\beta)-\ell_p(\hat\beta+\delta)\}=J\delta^2-\tfrac{K}{3}\delta^3+O(\delta^4)\). The cubic term changes sign when \(\delta\) changes sign. If the two nearby endpoints for a small cutoff \(c\) are written as \(\delta_s=s h+a h^2+O(h^3)\), where \(s\in\{-1,1\}\) and \(h=\sqrt{c/J}\), substitution gives \(2Ja-K/3=0\). Hence \(a=K/(6J)\), and \(\delta_s=s\sqrt{c/J}+\frac{K}{6J}\frac{c}{J}+O(c^{3/2})\).

Derivation of the endpoint expansion. In the likelihood-ratio expansion, the quadratic term contributes \(J\delta_s^2=c+2Js a h^3+O(h^4)\), and the cubic term contributes \(-\tfrac K3\delta_s^3=-\tfrac K3s h^3+O(h^4)\). For the likelihood-ratio value to equal \(c\), the coefficient of \(s h^3\) must vanish: \(2Ja-K/3=0\). This gives \(a=K/(6J)\) and the stated expansion. Both endpoints receive the same second-order displacement from the maximum, which makes their distances unequal. \(\square\)

For the sparse example, \(\ell''(\hat\beta)=-10(0.2)(0.8)=-1.6\), so \(J=1.6\). Also \(\ell'''(\hat\beta)=-10(0.2)(0.8)(1-2(0.2))=-0.96\). With \(c=3.8415\), the quadratic half-width is \(\sqrt{c/1.6}\approx1.5495\). The cubic approximation shifts both endpoints by \(\frac{-0.96}{6(1.6)}(1.5495)^2\approx-0.2401\), predicting displacements of roughly \(-1.7896\) and \(1.3094\). This small-cutoff expansion is only approximate at a 95% cutoff, but it correctly predicts the direction of the asymmetry: the negative third derivative shifts both endpoints toward smaller coefficients.

Interpretation and Common Pitfalls

A profile interval uses the likelihood over the range of candidate values, whereas a Wald interval summarizes the likelihood locally through curvature at its maximum. The profile approach can therefore retain information about asymmetry that a single standard error cannot express. In a regression analysis, this can matter when a coefficient is estimated from sparse events or when plausible values span a region where the likelihood changes curvature substantially.

Asymmetry alone does not establish that a profile interval has better coverage, nor does it show that the Wald interval is always unsuitable. The cutoff’s calibration, the model assumptions, and the amount of information in the data also matter. In particular, the chi-square cutoff used here is an asymptotic approximation; with sparse data, its nominal coverage may be imperfect.

• Mistake: assuming every interval must be centered on the estimate. That is true of the usual Wald construction, but not of a likelihood-ratio interval. Check the two profile endpoints rather than forcing them into a symmetric form.
• Mistake: treating the local quadratic approximation as the full likelihood. Curvature describes behavior near the maximum. If an endpoint is far away, higher-order terms can materially change its location.
• Mistake: confusing shape with calibration. An asymmetric profile interval is not automatically an exact interval. The choice of likelihood-ratio cutoff and its finite-sample performance are separate questions.
• Mistake: rounding an endpoint before transforming it and reporting false precision. If probabilities are displayed only approximately, coefficient endpoints computed from them should be reported to a compatible precision. For example, \(\log(0.4994/0.5006)\approx-0.0024\), not \(-0.0022\); the latter is appropriate only for a probability endpoint closer to \(0.49944\).