Unbounded Profile Intervals Under Weak Identification

When a Profile Interval Has No Finite Endpoint

A profile interval is often reported by finding where the profile likelihood falls far enough below its maximum. But that procedure need not find a finite endpoint on both sides. If the profile likelihood remains close to its supremum while the target parameter moves arbitrarily far in one direction, the likelihood-ratio confidence set extends without bound in that direction.

This behavior is worth diagnosing rather than treating as a numerical inconvenience. It can signal that the data provide little information about the target, or that the likelihood increases toward a limiting value without attaining a finite maximum. Sparse logistic data can produce the latter behavior through separation. In either case, the profile curve—not the success or failure of an optimizer—is the basis for deciding whether an endpoint exists.

Definition: Let $\psi$ be a scalar target, let $\lambda$ denote nuisance parameters, and let $\ell(\psi,\lambda)$ be the log-likelihood. Define the profile log-likelihood and the full likelihood supremum by $$ \ell_p(\psi)=\sup_{\lambda}\ell(\psi,\lambda), \qquad \ell^*=\sup_{\psi,\lambda}\ell(\psi,\lambda). $$ For a cutoff $c>0$, the profile likelihood-ratio confidence set is $$ C_c=\{\psi: D(\psi)\leq c\}, \qquad D(\psi)=2\{\ell^*-\ell_p(\psi)\}. $$ The suprema need not be attained. In particular, this definition still applies when there is no finite maximum-likelihood estimate.

The cutoff is often selected using a likelihood-ratio reference distribution, but the diagnostic here is about the shape of the profile and applies for any fixed positive cutoff. If the target ranges to $+\infty$, inspect $D(\psi)$ as $\psi\to+\infty$; for a negative infinite endpoint, inspect the corresponding limit as $\psi\to-\infty$.

A Tail Criterion for Unboundedness

A finite endpoint in the positive direction requires the profile deviance eventually to exceed the cutoff. The following simple result makes the converse precise.

Theorem: Suppose the parameter space for $\psi$ contains arbitrarily large values and $D(\psi)\to d_\infty$ as $\psi\to+\infty$. If $d_\infty<c$, then $C_c$ is unbounded above. If $d_\infty>c$, then $C_c$ contains no sufficiently large values of $\psi$. When $d_\infty=c$, the limit alone does not determine whether the set is unbounded.

Proof. If $d_\infty<c$, choose $\varepsilon=(c-d_\infty)/2>0$. By the definition of the limit, there is an $M$ such that $\psi>M$ implies $|D(\psi)-d_\infty|<\varepsilon$. Therefore $D(\psi)<d_\infty+\varepsilon=(d_\infty+c)/2<c$ for every $\psi>M$. The whole tail is in $C_c$, so the set is unbounded above. If $d_\infty>c$, take $\varepsilon=(d_\infty-c)/2$. For all sufficiently large $\psi$, $D(\psi)>d_\infty-\varepsilon=(d_\infty+c)/2>c$, so none of those values is in $C_c$. If $d_\infty=c$, convergence permits values on either side of $c$, so no conclusion follows without examining how $D(\psi)$ approaches its limit. $\square$

This criterion also describes a practical profile-curve diagnostic: evaluate the likelihood far into the tail, then determine whether the curve is approaching the full supremum closely enough to stay inside the chosen cutoff. A plot over a finite range can suggest this behavior, but cannot by itself establish what happens beyond that range.

Worked Example: A Sparse Logistic Sample with an Unbounded Coefficient Set

Consider three independent binary observations in a logistic regression with an intercept and one binary predictor. At $x=0$, observe one success and one failure; at $x=1$, observe one success and no failures. Write $$ \operatorname{logit}(p_x)=\alpha+\beta x, \qquad p_x=\frac{e^{\alpha+\beta x}}{1+e^{\alpha+\beta x}}. $$ The target is the slope $\beta$, the log odds ratio comparing $x=1$ with $x=0$.

Ignoring binomial coefficients, which equal one here, the likelihood is $$ L(\alpha,\beta)=p_0(1-p_0)p_1. $$ Since $p_0(1-p_0)\leq 1/4$ and $p_1<1$ for every finite $\alpha,\beta$, $L(\alpha,\beta)<1/4$ at every finite parameter value. But setting $\alpha=0$ and letting $\beta\to+\infty$ makes $p_0=1/2$ and $p_1\to1$. Thus the full likelihood supremum is $L^*=1/4$, approached but not attained.

For each fixed $\beta$, profiling over $\alpha$ gives $L_p(\beta)=\sup_\alpha L(\alpha,\beta)$. Choosing $\alpha=0$ is a permissible candidate, so $$ \frac{L_p(\beta)}{L^*}\geq \frac{(1/2)(1/2)\operatorname{logit}^{-1}(\beta)}{1/4} =\operatorname{logit}^{-1}(\beta). $$ Also $L_p(\beta)/L^*\leq1$, since $L^*$ is the full supremum. As $\beta\to+\infty$, the lower bound tends to one, so the profile likelihood ratio tends to one as well. Consequently $D(\beta)=-2\log\{L_p(\beta)/L^*\}\to0$. For every positive cutoff, the theorem implies that the profile confidence set contains a positive tail and is unbounded above.

For illustration, use the cutoff $c=3.84$. The candidate bound already guarantees inclusion whenever $\operatorname{logit}^{-1}(\beta)\geq e^{-1.92}$, or $$ \beta\geq\log\left(\frac{e^{-1.92}}{1-e^{-1.92}}\right)\approx-1.76. $$ This is a sufficient condition from the candidate $\alpha=0$, not an exact description of the whole confidence set. It shows explicitly that arbitrarily large slopes are retained. In this dataset, the cause is separation: the sole observation at $x=1$ is a success, and increasing $\beta$ can keep improving its fitted probability without a finite maximizer.

Why Sparsity Alone Does Not Guarantee an Unbounded Set

The previous example has an all-success predictor group. A single failure in that group changes the tail behavior. This distinction matters: “sparse” describes the amount of data, but does not by itself determine whether the profile likelihood reaches its supremum in a tail.

Worked Example: A Failure in Each Predictor Group

Now observe one success and one failure at each of $x=0$ and $x=1$. The same model has likelihood $$ L(\alpha,\beta)=p_0(1-p_0)p_1(1-p_1). $$ Each factor $p_x(1-p_x)$ is at most $1/4$, and both bounds are attained together at $\alpha=0,\beta=0$. Hence $L^*=1/16$, attained at a finite parameter value.

To examine the positive slope tail, let $\eta_0=\alpha$ and $\eta_1=\alpha+\beta$. Their difference is $\beta$, so at least one of $|\eta_0|$ and $|\eta_1|$ is at least $\beta/2$. For any real $\eta$, logistic variance satisfies $$ \operatorname{logit}^{-1}(\eta)\{1-\operatorname{logit}^{-1}(\eta)\} =\frac{e^\eta}{(1+e^\eta)^2} \leq e^{-|\eta|}. $$ The other variance factor is at most $1/4$. Therefore, for every $\alpha$ and $\beta>0$, $$ L(\alpha,\beta)\leq\frac14 e^{-\beta/2}, \qquad \frac{L_p(\beta)}{L^*}\leq4e^{-\beta/2}\longrightarrow0. $$ Thus $D(\beta)\to+\infty$: the profile eventually falls below any fixed finite cutoff, so the confidence set has no positive infinite tail. Here the failure at $x=1$ penalizes predictions tending to one, while pushing the intercept down to avoid that penalty would also harm the observations at $x=0$.

This contrast is a useful check on interpretation. The first dataset has no finite full maximum and its likelihood supremum is approached along an increasingly large slope. The second dataset has a finite maximum and its profile likelihood vanishes in the positive tail. Other forms of weak identification can produce a nearly flat profile without complete separation, but the defining diagnostic for an unbounded set remains the tail behavior of $D(\psi)$ relative to the cutoff.

Reporting the Endpoint on an Odds-Ratio Scale

In the first logistic example, the odds ratio is $\exp(\beta)$. This is a strictly increasing transformation, so an unbounded positive tail for $\beta$ becomes an unbounded upper tail for the odds ratio. A finite numerical upper endpoint should not be reported merely because a program stopped searching at a large value.

Worked Example: Translating the Sparse-Data Result to an Odds Ratio

For the sparse dataset with one success and one failure at $x=0$, and one success at $x=1$, the candidate calculation above places all $\beta\geq-1.76$ inside the confidence set at cutoff $3.84$. Exponentiating this sufficient included tail gives $$ \exp(\beta)\geq\exp(-1.76)\approx0.172. $$ Thus the odds-ratio confidence set contains arbitrarily large values; it has no finite upper endpoint.

As a numerical check on the slope-to-odds-ratio conversion, an odds ratio of $(e^{0.96}-1)^2$ corresponds to $$ \beta=\log\{(e^{0.96}-1)^2\} =2\log(e^{0.96}-1) \approx0.955. $$ Indeed, $e^{0.96}\approx2.611696$, so $e^{0.96}-1\approx1.611696$, whose logarithm is approximately $0.477287$; doubling gives $0.954575$, which rounds to $0.955$. This finite value is only a scale check: it does not create a finite upper endpoint for the confidence set.

Common Mistake and Practical Checks

A common mistake is to interpret “the optimizer did not converge” as proof that the profile interval is unbounded. Nonconvergence can instead reflect poor starting values, numerical scaling, or a difficult but finite optimum. The correction is to inspect the likelihood itself: determine the full supremum, evaluate the profiled likelihood along the tail, and compare its limiting deviance with the cutoff.

Do not assume the maximum is attained. Define the profile using a supremum, especially when logistic separation is possible.
Do not infer an infinite endpoint from a wide plot alone. A finite plotting window does not show the limiting profile.
Do not treat the equality case as settled. If the limiting deviance equals the cutoff, inspect the direction and rate of approach.
Report the scale and direction. State which target is unbounded and whether the confidence set extends toward positive or negative infinity.

An unbounded profile set is a substantive warning about the information in the data, not a missing endpoint to be repaired by truncation. In sparse logistic analyses, it may reflect separation and a likelihood supremum at infinity. The cutoff still determines the set, and its calibration is a separate question from whether the profile has a finite endpoint.

Check Your Understanding

Use the profile deviance and the data pattern to answer these questions.

If $D(\psi)\to1.2$ as $\psi\to+\infty$ and the cutoff is $3.84$, what does the tail criterion imply?
Why is the full likelihood supremum in the first logistic example not attained at any finite $(\alpha,\beta)$?
In the second logistic example, which factor in the likelihood penalizes a prediction $p_1$ tending to one?
Why does failure of a numerical optimizer, by itself, not establish that a profile confidence set is unbounded?
If a confidence set for a logistic slope is unbounded above, what happens to its image on the odds-ratio scale?