Why Likelihood Curvature Matters
A score equation identifies points where the coefficient likelihood is locally stationary, but it does not by itself tell us whether a solution is a maximum, whether there are several maxima, or whether a numerical search might encounter other stationary points. Concavity supplies that missing global information: on a convex parameter space, a concave log likelihood has no isolated local maximum that is inferior to the global maximum.
In generalized linear models, curvature depends on more than the variance function. The link controls how the mean changes with the linear predictor, and a noncanonical link can introduce a response-dependent term into the second derivative. We derive that term and use it to distinguish a general sufficient condition for concavity from the simpler canonical-link case.
Setup and the Curvature Formula
Suppose \(Y_1,\ldots,Y_n\) are independent observations from a one-parameter exponential dispersion family with fixed dispersion. Write the contribution to the log likelihood, apart from terms not involving the mean, as
Here \(a_i(\phi)>0\) is fixed with respect to \(\beta\), and \(V(\mu_i)>0\) on the mean domain under consideration. The generalized linear model specifies \(\eta_i=x_i^\mathsf{T}\beta\) and \(\eta_i=g(\mu_i)\), where \(x_i\) is the covariate vector and \(g\) is a differentiable, one-to-one link on the relevant mean interval. Assume the inverse link \(\mu(\eta)=g^{-1}(\eta)\) is twice differentiable there. The coefficient parameter space is taken to be convex, and all fitted predictors are required to lie in an interval where these conditions hold.
To obtain the curvature, first differentiate with respect to \(\eta_i\). Because \(d\ell_i/d\theta_i=(y_i-\mu_i)/a_i(\phi)\), the chain rule gives
Differentiating once more, with \(y_i\) fixed, yields
Since \(\eta_i=x_i^\mathsf{T}\beta\) is affine in \(\beta\), the coefficient score and Hessian are
Concavity and the Canonical Link
The curvature formula gives a condition that can be checked for a specified link and data. A particularly important case is the canonical link, for which the linear predictor is the natural parameter: \(\eta=\theta\). Then \(d\theta/d\eta=1\), so \(q(\eta)=1\), \(q'(\eta)=0\), and \(\mu'(\eta)=V(\mu)\). Consequently,
Here is a proof of the strictness claim as well as concavity. Let \(X\) be the design matrix, and let \(W\) be the diagonal matrix with entries \(V(\mu_i)/a_i(\phi)\). The canonical-link Hessian is \(-X^\mathsf{T}WX\). For every coefficient-direction vector \(v\),
Every summand is nonnegative before the minus sign, so the Hessian is negative semidefinite and the likelihood is concave on the convex domain. If \(X\) has full column rank and \(v\ne 0\), then \(Xv\ne 0\). Since every diagonal entry of \(W\) is positive, at least one squared term is positive, making the displayed quadratic form strictly negative. Thus the Hessian is negative definite at every interior point, which gives strict concavity. If \(X\) is rank deficient, a nonzero \(v\) with \(Xv=0\) instead gives zero curvature in that direction; strict concavity is not guaranteed. \(\square\)
Worked Example: A One-Parameter Logistic Likelihood
Suppose five independent Bernoulli observations share one success probability \(p\), with three successes and two failures. Use the logit link, so \(\eta=\beta=\log\{p/(1-p)\}\) and \(p=e^\beta/(1+e^\beta)\). The log likelihood, omitting no parameter-dependent terms, is
Its first and second derivatives are
The score is zero exactly when \(p=3/5\), giving \(\hat\beta=\log\{(3/5)/(2/5)\}=\log(3/2)\). The likelihood increases when \(p<3/5\) and decreases when \(p>3/5\), as follows directly from the sign of \(3-5p\). Its strict concavity therefore makes this stationary point the unique global maximum over \(\beta\in\mathbb{R}\). At the maximum the second derivative is \(-5(3/5)(2/5)=-6/5\).
Noncanonical Links: Check the Response-Dependent Term
For a noncanonical link, \(q'\) need not vanish. The term \((y_i-\mu_i)q'(\eta_i)\) can increase or decrease curvature, so the canonical-link conclusion cannot simply be carried over. A useful sufficient condition is to verify, for every observation and every allowed predictor, that
This condition is sufficient because each \(x_i x_i^\mathsf{T}\) is positive semidefinite: for any \(v\), \(v^\mathsf{T}x_i x_i^\mathsf{T}v=(x_i^\mathsf{T}v)^2\geq 0\). If all scalar coefficients in the Hessian sum are nonpositive, its quadratic form is nonpositive in every direction. It is not necessary: positive and negative contributions in the sum can sometimes combine to give a negative semidefinite coefficient Hessian. The exact criterion on a convex open domain is that the full Hessian be negative semidefinite everywhere.
Worked Example: A Concave Poisson Likelihood With the Identity Link
For a Poisson observation, \(V(\mu)=\mu\). Use the identity link \(\eta=\mu\), with predictor restricted to \(\eta>0\). Then \(\mu'=1\), \(q=1/\eta\), and \(q'=-1/\eta^2\). The scalar curvature formula gives
because a Poisson observation satisfies \(y_i\geq 0\). Equivalently, the parameter-dependent log likelihood is \(y_i\log\eta-\eta\), whose second derivative is \(-y_i/\eta^2\). For a concrete observation \(y=4\), the score is \(4/\eta-1\), so the unique stationary point is \(\eta=4\), and the second derivative there is \(-1/4\). The likelihood is strictly concave for \(y>0\); if \(y=0\), it is linear in \(\eta\), so it is concave but not strictly concave. This example shows that a noncanonical link can still produce concavity, but the conclusion must be checked rather than assumed.
Worked Example: A Nonconcave Poisson Likelihood With the Inverse Link
Use the Poisson model with the inverse link \(\eta=1/\mu\), so the allowable predictor domain is \(\eta>0\). For one observation \(y=1\), the log likelihood, up to a constant, is
Differentiating directly gives
At the allowed predictor value \(\eta=3\), the curvature is \(1/9-2/27=1/27>0\). Thus the likelihood is not concave on the full domain \((0,\infty)\). In the general formula, \(\mu=1/\eta\), \(\mu'=-1/\eta^2\), \(q=-1/\eta\), and \(q'=1/\eta^2\), which gives the same curvature \(y/\eta^2-2/\eta^3\). The positive curvature at \(\eta=3\) is a concrete check that a noncanonical Poisson likelihood need not be concave.
When Does Concavity Give a Unique Maximum?
Concavity concerns the shape of the likelihood, while existence concerns whether a maximum is attained on the parameter space. They are distinct questions. If \(\ell\) is differentiable and concave on a convex open set, any interior point \(\hat\beta\) satisfying \(\nabla_\beta\ell(\hat\beta)=0\) is a global maximizer. To see why, take any other point \(\beta\) in the set and define \(h(t)=\ell(\hat\beta+t(\beta-\hat\beta))\) for \(0\leq t\leq1\). Concavity implies \(h(1)\leq h(0)+h'(0)\); the chain rule gives \(h'(0)=\nabla_\beta\ell(\hat\beta)^\mathsf{T}(\beta-\hat\beta)=0\). Hence \(\ell(\beta)\leq\ell(\hat\beta)\).
If the likelihood is strictly concave, there cannot be two distinct global maximizers: strict concavity would make the likelihood at any point strictly between them greater than their shared maximum. Thus an interior score solution is the unique global maximizer when strict concavity holds. By contrast, negative definiteness of the Hessian at just one point establishes local curvature there; it does not establish global concavity or rule out another maximum elsewhere. Even global strict concavity does not by itself guarantee that a maximum exists on an open or unbounded parameter space—the likelihood may approach its supremum only at a boundary or at infinity.
Worked Example: Rank Deficiency Can Prevent Uniqueness
Consider two Bernoulli observations, both successes, with design vectors \(x_1=(1,0)^\mathsf{T}\) and \(x_2=(1,0)^\mathsf{T}\), and use the canonical logit link. The predictors are both \(\beta_1\); the likelihood does not depend on \(\beta_2\). Its log likelihood is
For every finite \(\beta_1\), its derivative with respect to \(\beta_1\) is \(2-2p>0\), where \(p=e^{\beta_1}/(1+e^{\beta_1})\). The likelihood increases toward its supremum as \(\beta_1\) tends to infinity, so no finite maximum is attained. Meanwhile, every value of \(\beta_2\) gives the same likelihood. This example separates two issues: rank deficiency creates a flat coefficient direction, and boundary behavior can prevent a maximum from existing at all.
Common Mistakes and Practical Use
- Assuming every GLM likelihood is concave. A linear predictor does not guarantee concavity when the link is noncanonical. Derive the scalar second derivative and check its sign over the relevant data and predictor domain.
- Dropping the response-dependent term. For a general link, \((y_i-\mu_i)q'(\eta_i)\) contributes to curvature. It disappears for a canonical link because \(q'=0\), not as a general rule.
- Equating a stationary point with a unique maximum. A zero score alone does not imply either property. Concavity makes an interior stationary point global; strict concavity gives uniqueness.
- Inferring existence from strict concavity. Strict concavity allows at most one maximizer, but the supremum may fail to be attained. Check whether an interior solution exists and whether the parameter space includes the relevant boundary.
The curvature calculation is useful for understanding what likelihood-based coefficient estimation can guarantee before applying a numerical method. A negative definite Hessian over a convex domain gives a single peak and rules out competing interior maxima. If concavity fails, a numerical procedure may still find a useful solution, but curvature alone no longer supplies the same global assurance. Concavity is therefore a structural property to establish from the model, not a synonym for successful optimization.
Check Your Understanding
Use the curvature formula and the distinction between concavity and existence to answer these questions.
- In a canonical-link model, what are \(q(\eta)\) and \(q'(\eta)\), and why does the response-dependent curvature term vanish?
- Why is a negative semidefinite Hessian at every point on a convex domain sufficient for concavity?
- For a noncanonical link, what condition on the scalar second derivative is sufficient to make the coefficient Hessian negative semidefinite?
- How can a likelihood be concave but not strictly concave? Give a feature of the design or data that can produce this situation.
- Why does strict concavity not ensure that a maximum is attained?