Why a Candidate Estimate Needs Checking
Solving likelihood score equations produces a candidate maximum-likelihood estimate, not automatically a maximum. The solution could be a minimum or a saddle point, could violate a parameter constraint, or could be inferior to a boundary value or another candidate. A reliable check uses several pieces of evidence: the score, curvature, feasibility, and likelihood values.
This tutorial concerns complete-data likelihoods such as those used in the earlier material on severity models and categorical models. For a sample with likelihood \(L(\theta)\), it is usually convenient to work with the log-likelihood \(\ell(\theta)=\log L(\theta)\). Because the logarithm is strictly increasing, maximizing \(L\) and maximizing \(\ell\) give the same candidates and ordering.
For independent observations \(X_1,\ldots,X_n\), the log-likelihood is a sum of individual log-likelihood contributions, \(\ell(\theta)=\sum_{i=1}^n \ell_i(\theta)\). Accordingly, the total score is the sum of individual score contributions. These contributions help locate calculation or data-entry errors: large positive and negative contributions may cancel in the total, so a zero total alone does not verify that each term was formed correctly.
What Score and Curvature Can Establish
For a differentiable log-likelihood and an unconstrained interior local maximum, the first-order necessary condition is \(U(\hat\theta)=0\). In multiple dimensions, every component of the gradient must be zero. This condition does not apply automatically at a parameter-space boundary, or at a point where the likelihood is not differentiable. It is also not sufficient: a stationary point might be a minimum or a saddle point.
Second-order conclusions require second-order regularity. If the log-likelihood has a second-order Taylor expansion at an interior local maximum, its Hessian must be negative semidefinite. In one dimension this means \(\ell''(\hat\theta)\leq 0\). If the Hessian is negative definite at a stationary point and the log-likelihood has continuous second derivatives in a neighborhood, the second-order Taylor test gives a strict local maximum. A zero eigenvalue or zero second derivative makes the test inconclusive in that direction; it does not by itself establish a maximum.
Here \(H\) is the Hessian matrix of second partial derivatives. At a stationary point the linear term vanishes. If \(H\) is negative definite, then \(\mathbf h^{\mathsf T}H\mathbf h<0\) for every nonzero sufficiently small direction \(\mathbf h\), and the quadratic term decreases the log-likelihood. This explains why negative-definite curvature is sufficient for a strict local maximum under the stated regularity conditions. Negative semidefinite curvature is only a necessary condition and may leave some directions unresolved.
Worked Example: Score Contributions in a Bernoulli Model
A hypothetical insurer records whether each of eight independent, similar claims exceeds a specified threshold. Five exceed it. Model each indicator as Bernoulli with exceedance probability \(p\), where \(0<p<1\). Check the candidate \(\hat p=5/8\), including its score residual, curvature, and likelihood against an alternative.
Method. For five exceedances and three non-exceedances, \(L(p)=p^5(1-p)^3\), so \(\ell(p)=5\log p+3\log(1-p)\). The score is \(U(p)=5/p-3/(1-p)\). At \(p=5/8\), \(U(5/8)=5/(5/8)-3/(3/8)=8-8=0\).
The individual score contribution is \(1/p=8/5=1.6\) for each exceedance and \(-1/(1-p)=-8/3\) for each non-exceedance. Their total is \(5(1.6)+3(-8/3)=8-8=0\), independently checking the aggregate score calculation.
The second derivative is \(\ell''(p)=-5/p^2-3/(1-p)^2\). At the candidate it is \(-5(64/25)-3(64/9)=-64/5-64/3=-512/15\), which is negative. Further, \(\ell''(p)<0\) throughout \(0<p<1\), so the log-likelihood is strictly concave there and this stationary point is the unique interior maximum.
For a direct alternative check, \(L(5/8)=(5/8)^5(3/8)^3=84375/16777216\), while \(L(1/2)=1/256=65536/16777216\). Thus the candidate has the larger likelihood. At either endpoint \(p=0\) or \(p=1\), the likelihood is zero for these data. Check: the zero score, negative curvature, and boundary comparison agree. The common mistake is to treat a zero score as sufficient without checking curvature or feasibility.
Checking a Multidimensional Candidate
With several parameters, check each score component and assess the whole Hessian, not just its diagonal entries. A negative diagonal does not guarantee a negative-definite matrix; mixed second derivatives can change curvature along combined directions. In a two-parameter problem, negative definiteness requires, for a symmetric Hessian, a negative first leading principal minor and a positive determinant.
Worked Example: Two-Parameter Normal Likelihood
A hypothetical severity sample consists of the complete observations \(1,2,3\), modeled as independent \(N(\mu,\eta)\) variables, where \(\eta=\sigma^2>0\) is the variance. Check the candidate \((\hat\mu,\hat\eta)=(2,2/3)\) using the score, Hessian, and a likelihood comparison.
Method. The log-likelihood, including the density constant, is \(\ell(\mu,\eta)=-\frac{3}{2}\log(2\pi\eta)-\frac{1}{2\eta}\sum_{i=1}^3(x_i-\mu)^2\). At \(\mu=2\), the residuals are \(-1,0,1\), their sum is zero, and their squared sum is \(2\).
The score components are \(U_\mu=\sum_i(x_i-\mu)/\eta\) and \(U_\eta=-3/(2\eta)+\sum_i(x_i-\mu)^2/(2\eta^2)\). At the candidate, \(U_\mu=0/(2/3)=0\), and \(U_\eta=-3/(4/3)+2/[2(4/9)]=-9/4+9/4=0\).
The Hessian entries are \(H_{\mu\mu}=-3/\eta\), \(H_{\mu\eta}=-\sum_i(x_i-\mu)/\eta^2\), and \(H_{\eta\eta}=3/(2\eta^2)-\sum_i(x_i-\mu)^2/\eta^3\). At the candidate these are \(-9/2\), \(0\), and \(27/8-27/4=-27/8\), respectively. Thus the Hessian is diagonal with two negative entries, so it is negative definite. The candidate is a strict local maximum under the regularity conditions here.
Compare it with the feasible alternative \((\mu,\eta)=(2,1)\). At the candidate, \(\ell=-\frac{3}{2}\log(4\pi/3)-3/2\approx-3.6486\); at the alternative, \(\ell=-\frac{3}{2}\log(2\pi)-1\approx-3.7568\). The candidate has the larger log-likelihood. Check: both score components vanish, and both curvature directions are negative. The common mistake is to check only one score or to infer negative definiteness from negative diagonal entries without checking the off-diagonal terms.
Boundaries, Support, and Likelihood Comparisons
A constrained parameter can have a maximum at a boundary even when the score is not zero. Also, a likelihood can have a support-dependent boundary: the observed data may make the likelihood zero for some parameter values and positive for others. At such a point the usual interior score test may fail because the likelihood is not differentiable across the boundary, even if the nominal parameter space itself is open.
For an interior candidate, compare its log-likelihood with other stationary candidates and inspect the behavior at feasible parameter boundaries or limiting values. In a nonregular or constrained problem, compare the candidate directly with feasible alternatives and relevant boundary values. A local curvature test does not establish a global maximum. A strict global conclusion needs a valid global argument, such as concavity over the full feasible region, or a comparison covering all competing candidates and boundaries.
Worked Example: A Support-Induced Boundary
A hypothetical sample of three complete losses from a Uniform\((0,\theta)\) severity model is \(2,3,5\), in thousands of dollars. Check the candidate \(\hat\theta=5\) against \(\theta=6\), and explain why a zero score is not required.
Method. The density is \(1/\theta\) for an observation in \([0,\theta]\), and zero otherwise. The likelihood is therefore \(L(\theta)=\theta^{-3}\) when \(\theta\geq5\), and zero when \(0<\theta<5\). On the positive-likelihood region the log-likelihood is \(\ell(\theta)=-3\log\theta\), with derivative \(-3/\theta\).
At the feasible edge \(\theta=5\), the right-hand derivative is \(-3/5\), not zero. Nevertheless, for every \(\theta>5\), \(\theta^{-3}<5^{-3}\), so the maximum occurs at \(\theta=5\). Numerically, \(L(5)=1/125=0.008\), while \(L(6)=1/216\approx0.00463\). For \(\theta<5\) the likelihood is zero.
The boundary here is the edge of the positive-likelihood region created by the sample maximum. The likelihood is not differentiable across that edge: it is zero below 5 and positive at 5. Check: the candidate is feasible and has the largest likelihood in the stated model. The common mistake is to reject it because the derivative is nonzero; the interior first-order condition does not apply at this support-induced boundary.
Confirm the candidate satisfies parameter constraints and assigns positive likelihood to the observed complete data.
For an interior differentiable candidate, calculate every score component and, where useful, the individual contributions.
Use the second derivative or full Hessian only when second-order regularity holds; distinguish a necessary test from a sufficient strict-maximum test.
Evaluate competing stationary candidates, feasible boundaries, and relevant limits. A local check alone does not establish a global maximum.
Common Exam Traps
- Stopping at a zero score. It identifies a stationary point, not necessarily a maximum. Check curvature and other candidates.
- Applying the score condition at a boundary. A constrained or support-induced boundary can maximize likelihood with a nonzero one-sided derivative.
- Claiming curvature without regularity. A second derivative or Hessian test requires suitable second-order differentiability; a negative-definite sufficient test also needs regularity near the candidate.
- Checking only Hessian diagonal entries. In multiple dimensions, assess definiteness of the whole Hessian, including mixed partial derivatives.
- Comparing rounded likelihoods too early. Use log-likelihoods or adequate precision for comparisons, and round only after deciding the ordering.
Check Your Understanding
For each problem, state which checks apply before giving your conclusion.
- A hypothetical Bernoulli sample has 7 successes and 3 failures. Check \(p=0.7\) by calculating the score and second derivative. (Answer: \(U(0.7)=0\), \(\ell''(0.7)=-1000/21\approx-47.6190\); it is a strict interior maximum because the log-likelihood is strictly concave.)
- Complete exponential observations are \(2,2,4\), with density \(f(x;\lambda)=\lambda e^{-\lambda x}\) for \(x\geq0\), \(\lambda>0\). Check \(\lambda=3/8\) using score and curvature. (Answer: \(U(3/8)=3/(3/8)-8=0\), \(\ell''(3/8)=-3/(3/8)^2=-64/3\); it is a strict interior maximum.)
- A hypothetical Uniform\((0,\theta)\) sample is \(1.5,2.5,4\). Compare \(\theta=4\) with \(\theta=5\), and identify the relevant boundary. (Answer: \(L(4)=1/64\), \(L(5)=1/125\), and the maximum is at the support-induced boundary \(\theta=4\), without a zero-score requirement.)
- For complete normal observations \(0,2\), modeled as independent \(N(\mu,\eta)\) with unknown \(\mu\) and variance \(\eta>0\), check \((\mu,\eta)=(1,1)\) using both score components and the Hessian. (Answer: both scores are zero and the Hessian is diagonal with entries \(-2\) and \(-1\), so it is negative definite and gives a strict local maximum.)
- A one-parameter log-likelihood has a differentiable interior stationary candidate with \(\ell''=0\). What can the second-order test conclude, and what further check is needed? (Answer: the second-order test is inconclusive; examine higher-order behavior or compare feasible likelihood values and boundaries.)