Confidence Sets Under Nonlinear Parameter Transformations

Why the Scale of the Target Matters

A generalized linear model is often fitted on a coefficient scale, while the scientific question concerns a different quantity: an odds ratio, a probability, a risk ratio, or a ratio of coefficients. If the target is a nonlinear function of the coefficients, the way its confidence set is constructed can affect its shape, range, and interpretation. Transforming a coefficient confidence set and profiling the likelihood for the target are distinct procedures.

The first procedure carries a confidence set through the transformation. The second asks which target values remain compatible with the data according to a likelihood-ratio criterion, allowing nuisance parameters to adjust for each candidate value. Neither method is universally best: the target, the model, and the adequacy of the large-sample approximation all matter.

Transforming a Coefficient Confidence Set

Definition: Let \(\beta\) be a parameter vector and let \(C(X)\) be a random set satisfying \(P_\beta\{\beta\in C(X)\}\geq 1-\alpha\). For a function \(g\), the image confidence set is \(g(C(X))=\{g(b):b\in C(X)\}\). It is a confidence set for \(g(\beta)\).

This set-based operation is different from transforming only the point estimate and then attaching a standard error. It also differs from transforming just one coefficient interval when the target depends on several coefficients: in that case, the relevant object is the image of a joint confidence set, including the dependence among coefficient estimates.

Result: If \(C(X)\) has coverage at least \(1-\alpha\), then its image \(g(C(X))\) has coverage at least \(1-\alpha\) for \(g(\beta)\), provided the sets and function are measurable.

The justification is direct. Whenever \(\beta\in C(X)\), applying \(g\) gives \(g(\beta)\in g(C(X))\). Thus the event \(\{\beta\in C(X)\}\) is contained in \(\{g(\beta)\in g(C(X))\}\), and therefore

P_\beta\{g(\beta)\in g(C(X))\} \geq P_\beta\{\beta\in C(X)\} \geq 1-\alpha.

The image set can be conservative: different coefficient values may map to the same target value, and some points in the coefficient set may not be close to its center. But its coverage guarantee does not require \(g\) to be linear or one-to-one.

Worked Example: Transforming a Log-Odds Confidence Interval

In a logistic regression, suppose the estimated treatment coefficient is \(\hat\beta=0.20\) with standard error \(0.15\). A 95% Wald interval on the coefficient scale is

0.20\pm 1.96(0.15)=0.20\pm0.294=(-0.094,\ 0.494).

The odds ratio is \(g(\beta)=\exp(\beta)\), a strictly increasing function. Its image interval is found by transforming both endpoints:

(\exp(-0.094),\exp(0.494))\approx(0.910,\ 1.639).

This is an interval on the odds-ratio scale inherited from the coefficient interval. By contrast, a delta-method Wald interval is centered at \(\exp(0.20)\approx1.221\). Since \(g'(\beta)=\exp(\beta)\), its estimated standard error is \(1.221(0.15)\approx0.183\), giving \(1.221\pm1.96(0.183)\approx(0.862,\ 1.580)\). The two intervals differ because the exponential function is curved; the Wald calculation approximates the transformed estimator locally, while the image interval transforms the original endpoints.

Direct Likelihood Inference for a Transformed Target

Suppose the full model parameter is \(\eta\), which may include regression coefficients and nuisance parameters such as dispersion. Write \(\ell(\eta)\) for the log-likelihood and let \(\hat\eta\) be its unrestricted maximum-likelihood estimate. For a scalar target \(\psi=g(\eta)\), define the profile log-likelihood by maximizing over all parameter values that produce a specified target value.

Definition: The profile log-likelihood for \(\psi\) is \(\ell_p(\psi)=\sup_{\eta:g(\eta)=\psi}\ell(\eta)\). The likelihood-ratio confidence set with cutoff \(c\) is

\left\{\psi:2\left[\ell(\hat\eta)-\ell_p(\psi)\right]\leq c\right\}.

For a regular, identifiable model with the true parameter in the interior of the parameter space, a smooth target whose gradient has rank \(q\) for a \(q\)-dimensional target, and the usual likelihood differentiability and information conditions, Wilks' theorem gives an asymptotic \(\chi^2_q\) distribution for the likelihood-ratio statistic under the true target. For a scalar target, the usual large-sample cutoff is the \(1-\alpha\) quantile of \(\chi^2_1\). The stated conditions matter: without them, this calibration need not hold.

Profiling treats the target value as fixed and lets all remaining parameters take the values that maximize the likelihood subject to that restriction. In contrast, transforming an existing coefficient set carries along all coefficient combinations in the set. The two procedures therefore need not produce the same endpoints.

Result: The profile likelihood-ratio set consists exactly of the target values whose best-fitting constrained model has a likelihood-ratio statistic no larger than the chosen cutoff.

To see this, fix a candidate \(\psi\). Its constrained best-fitting log-likelihood is \(\ell_p(\psi)=\sup_{\eta:g(\eta)=\psi}\ell(\eta)\). The likelihood-ratio statistic comparing that constrained fit with the unrestricted fit is, by definition,

-2\log\left\{\frac{\exp(\ell_p(\psi))}{\exp(\ell(\hat\eta))}\right\} =2\left[\ell(\hat\eta)-\ell_p(\psi)\right].

Consequently, the statistic is at most \(c\) if and only if \(\psi\) belongs to the displayed profile likelihood-ratio set. This equivalence also explains why nuisance parameters must be optimized subject to the target constraint rather than simply held at their unrestricted estimates. \(\square\)

Worked Example: A Binomial Probability as a Nonlinear Target

Suppose \(Y\sim\operatorname{Binomial}(10,p)\), with a logistic parameterization \(p=\{1+\exp(-\beta)\}^{-1}\), and observe \(Y=9\). The maximum-likelihood estimates are \(\hat p=0.9\) and \(\hat\beta=\log(9)\approx2.197\). The approximate standard error of \(\hat\beta\) is

\sqrt{\frac{1}{10\hat p(1-\hat p)}}= \sqrt{\frac{1}{0.9}} \approx 1.054.

A coefficient-scale 95% Wald interval is \(2.197\pm1.96(1.054)\), or approximately \((0.131,4.263)\). Transforming its endpoints gives a probability interval of approximately \((0.533,0.986)\).

A direct delta-method interval illustrates a different issue. Since \(dp/d\beta=p(1-p)\), the estimated standard error of \(\hat p\) is \(0.9(0.1)(1.054)\approx0.0949\). The resulting interval is \(0.9\pm1.96(0.0949)\approx(0.714,1.086)\), which extends above one and so includes impossible probabilities.

For the direct likelihood-ratio interval, the binomial log-likelihood, apart from its constant term, is \(9\log(p)+\log(1-p)\). The saturated fit occurs at \(p=0.9\), so the likelihood-ratio statistic at a candidate \(p\) is

D(p)=2\left[9\log\left(\frac{0.9}{p}\right) +\log\left(\frac{0.1}{1-p}\right)\right].

Using the 95% \(\chi^2_1\) cutoff \(3.841\), solve \(D(p)=3.841\) on either side of \(0.9\). The roots are approximately \(0.627\) and \(0.994\), giving the profile likelihood interval \((0.627,0.994)\). Substitution into the displayed expression gives \(D(0.627)\approx3.87\) and \(D(0.994)\approx3.84\); the small discrepancy at the lower endpoint reflects rounding. Unlike the direct delta-method interval, this interval stays within the probability parameter space. It is still an asymptotic likelihood-ratio interval, not a guarantee of exact finite-sample coverage.

When the Image of a Joint Set Is Unbounded

A nonlinear target involving a denominator can behave especially differently from its plug-in estimate. The image of a joint coefficient set can be unbounded if that set includes denominator values arbitrarily close to zero. This is a feature of the information in the set, not a calculation error.

Worked Example: A Ratio With a Denominator Near Zero

Suppose a simultaneous coefficient confidence set is the rectangle \(C=[0.6,1.4]\times[-0.2,0.6]\) for \((\beta_1,\beta_2)\), and the target is \(g(\beta_1,\beta_2)=\beta_1/\beta_2\). The numerator is strictly positive throughout the rectangle, while the denominator ranges on both sides of zero. For positive denominators, the smallest possible ratio is \(0.6/0.6=1\), and ratios grow without bound as the denominator approaches zero. For negative denominators, the largest possible ratio is \(0.6/(-0.2)=-3\), and ratios decrease without bound toward negative infinity. Thus the image is

g(C)=(-\infty,-3]\ \cup\ [1,\infty).

By the image-set coverage argument, this disconnected and unbounded set retains at least the coverage of the original rectangle. Reporting only the ratio of the coefficient estimates, or applying a local standard-error formula without checking the denominator, could conceal this instability. Direct likelihood profiling may also yield an unbounded set when the data do not constrain the denominator away from zero.

Choosing and Interpreting the Construction

A transformed coefficient set is appealing when a suitable joint set is available and a transparent coverage argument is important. A profile likelihood set directly addresses which target values fit the model nearly as well as the unrestricted fit, and it naturally respects parameter constraints. Neither method eliminates the need to assess the approximation being used: Wald sets can be sensitive to curvature and parameter boundaries, while likelihood-ratio calibration relies on regularity and adequate large-sample behavior.

A frequent mistake is to assume that transforming a point estimate and its standard error is the same as transforming a confidence set. It is not. The correction is to identify the desired target first, then specify the procedure: take the image of a valid coefficient set, use a delta-method approximation with its assumptions understood, or profile the likelihood under each target constraint. For a target depending on several coefficients, use their joint uncertainty rather than separate intervals that ignore their dependence.

Key takeaway: Nonlinear transformations do not preserve the shape or endpoints of Wald intervals. Mapping a valid joint set preserves its coverage guarantee, while direct profile likelihood evaluates the fit at each target value and uses likelihood-ratio calibration under stated regularity conditions.

Check Your Understanding

Use these questions to test whether you can distinguish the constructions and their assumptions.

Why does the image of a coefficient confidence set have at least the coverage of that original set?
For an increasing scalar transformation, how do you obtain the image of an interval? What changes if the transformation is decreasing?
In profile likelihood inference for \(g(\eta)\), what is maximized while the candidate target value is held fixed?
Why can a delta-method interval for a probability extend outside the interval from zero to one?
What feature of a joint confidence set can make the image for a ratio unbounded?