Why a Stationary Point Is Not Enough
Solving a score equation identifies parameter values where the likelihood has zero slope, but it does not by itself establish that any such value maximizes the likelihood. A stationary point may be a local minimum, a local maximum that is not global, or a point where the usual second-derivative test gives no conclusion. The parameter space matters too: a maximum may occur at a boundary even though the score is not zero there.
The task is to compare the likelihood over the entire allowed parameter space. In practice, this means checking interior stationary candidates, applicable boundaries, and limiting behavior when the parameter space is open or unbounded. The examples use likelihood functions supplied for fixed observed data. A likelihood is viewed as a function of the unknown parameter; its value need not be a probability and can exceed 1.
A parameter-independent positive factor can be removed from a likelihood without changing its maximizer. When the likelihood is positive, one may also maximize its log-likelihood \(\ell(\theta)=\log L(\theta;x)\), because the logarithm is strictly increasing. If the likelihood can equal zero, the log-likelihood is not finite there; compare the likelihood itself at such values or treat them as zero-likelihood cases.
Second Derivatives: A Local Test
Suppose \(\theta_0\) is an interior point of \(\Theta\), and the log-likelihood is twice differentiable in a neighborhood of \(\theta_0\). If the score is zero, the second derivative can classify the stationary point. A negative second derivative indicates that the log-likelihood curves down locally; a positive one indicates that it curves up. A zero second derivative does not decide the question.
The connection between the two derivatives follows directly from \(L(\theta)=\exp(\ell(\theta))\) wherever \(L(\theta)>0\). Differentiating gives \(L'(\theta)=L(\theta)\ell'(\theta)\). Differentiating again gives \(L''(\theta)=L(\theta)\{[\ell'(\theta)]^2+\ell''(\theta)\}\). At a stationary point, \(\ell'(\theta_0)=0\), so \(L''(\theta_0)=L(\theta_0)\ell''(\theta_0)\). Since \(L(\theta_0)>0\), the signs of the second derivatives agree. This explains why the two local tests give the same classification at a positive-likelihood stationary point.
Worked Example: Confirming a Global Maximum by Concavity
For a fixed observed data set from a one-parameter model, suppose the likelihood kernel is \(L(\theta)=\exp[-(\theta-2)^2]\), with parameter space \(\Theta=[0,3]\). Determine whether the stationary candidate is a global maximum. No particular sampling distribution is needed here: the observed-data likelihood kernel and its parameter space are fully specified.
Step 1: Use the log-likelihood. The likelihood is positive throughout the parameter space, so its logarithm preserves its maximizer: \(\ell(\theta)=-(\theta-2)^2\).
Step 2: Find and classify stationary points. Differentiating gives \(\ell'(\theta)=-2(\theta-2)\). Setting this equal to zero gives \(\theta=2\), which is inside \([0,3]\). The second derivative is \(\ell''(\theta)=-2<0\), so \(\theta=2\) is a strict local maximum.
Step 3: Establish the global result. The second derivative is negative for every \(\theta\in[0,3]\), so the log-likelihood is strictly concave throughout the interval. More directly, \(\ell(\theta)=-(\theta-2)^2\leq 0=\ell(2)\) for every \(\theta\), with equality only at \(\theta=2\). Thus \(L(\theta)\leq L(2)=1\) throughout the parameter space.
As a numerical check, \(L(0)=e^{-4}\approx0.0183\), \(L(2)=1.0000\), and \(L(3)=e^{-1}\approx0.3679\). The candidate is the unique global maximum. The common mistake avoided here is to stop after the local second-derivative test; the inequality, or concavity over the whole parameter space, supplies the global conclusion.
Checking the Whole Parameter Space
For a closed, bounded interval and a continuous likelihood, a global maximum exists. To locate it, check every interior stationary point and each included endpoint. Compare their likelihood values. Concavity can simplify the work: if a differentiable log-likelihood is concave throughout an interval, any interior stationary point is a global maximum; if it is strictly concave, that maximum is unique. If concavity does not hold, evaluating one stationary point and the endpoints may not be enough unless all other stationary points have also been found.
The same candidate-based check needs extra care for open or unbounded parameter spaces. A boundary point that is not in \(\Theta\) cannot be selected as an estimate. A likelihood may approach its largest possible value as the parameter approaches an excluded boundary, without attaining that value. Likewise, on an unbounded space, check what happens as the parameter tends to infinity or negative infinity, as applicable. A maximum likelihood estimate exists only if the largest likelihood value is attained at an allowed parameter value.
Record all restrictions, including whether each endpoint is included. A boundary outside the parameter space is not an allowable estimate.
Differentiate the likelihood or log-likelihood and solve the score equation, retaining only solutions in the interior of the parameter space.
Use the second derivative for local information, then compare likelihood values across all relevant candidates. Use concavity only if it holds throughout the parameter space under consideration.
Evaluate included endpoints and examine limiting likelihood values at excluded boundaries or at infinity. State explicitly if the supremum is not attained.
Worked Example: A Stationary Point That Is Not a Maximum
For fixed observed data, suppose a model has likelihood kernel \(L(\theta)=(\theta^2+1)^2/9\) on \(\Theta=[-2,2]\). Determine the global maximizer, checking the stationary point and the boundary. The kernel is nonnegative and is treated as the supplied likelihood function up to any positive parameter-independent factor.
Step 1: Differentiate the likelihood. The first derivative is \(L'(\theta)=4\theta(\theta^2+1)/9\). Since \(\theta^2+1>0\), the only stationary point is \(\theta=0\).
Step 2: Classify the stationary point. The second derivative is \(L''(\theta)=(12\theta^2+4)/9\), which is positive for every \(\theta\). In particular, \(L''(0)=4/9>0\), so \(\theta=0\) is a local minimum, not a maximum. Its likelihood is \(L(0)=1/9\approx0.1111\).
Step 3: Evaluate the boundary and verify intermediate values. At \(\theta=-2\) and \(\theta=2\), \(\theta^2+1=5\), so each likelihood is \(5^2/9=25/9\approx2.7778\). At \(\theta=-1\) and \(\theta=1\), \(\theta^2+1=2\), so each likelihood is \(2^2/9=4/9\approx0.4444\). These values confirm that the likelihood increases as \(|\theta|\) increases over this interval.
The global maxima are therefore \(\theta=-2\) and \(\theta=2\), both included in the parameter space, with likelihood \(25/9\). The common mistake avoided is calling every solution of the score equation a maximum. Here the sole stationary point is a minimum, and the global maxima occur at the boundaries. The arithmetic check is important: \(L(\pm1)=4/9\), not zero, and \(L(\pm2)=25/9\), not one.
Worked Example: A Constrained Maximum at an Endpoint
Suppose the observed-data likelihood is \(L(\theta)=\exp(3\theta-\theta^2)\) and the parameter space is \(\Theta=[0,1]\). Determine the maximum and explain why solving the score equation without respecting the restriction would be misleading.
Step 1: Work with the log-likelihood. Since \(L(\theta)>0\), maximize \(\ell(\theta)=3\theta-\theta^2\). Its derivative is \(\ell'(\theta)=3-2\theta\), which is positive for every \(\theta\in[0,1]\), because \(3-2\theta\geq1>0\).
Step 2: Compare allowed values. The log-likelihood is strictly increasing on the allowed interval. Hence its maximum, and the likelihood maximum, occurs at the included endpoint \(\theta=1\). The likelihood values at the endpoints are \(L(0)=e^0=1\) and \(L(1)=e^{3-1}=e^2\approx7.3891\).
The unrestricted score equation would give \(3-2\theta=0\), or \(\theta=1.5\), which is outside \([0,1]\) and cannot be an estimate in this model. There is no interior stationary candidate to select; the constrained global maximum is \(\hat{\theta}=1\). The mistake avoided is reporting an algebraic score solution without checking whether it belongs to the parameter space.
When a Maximum Is Not Attained
Boundary checks can reveal that a maximum does not exist. For example, if the parameter space excludes a point where the likelihood would be largest, values can approach a supremum without ever equaling it. In that case, it is not correct to call the excluded boundary value a maximum likelihood estimate. The distinction between a maximum and a supremum is not merely terminology: an estimate must be an allowed parameter value that attains the likelihood.
Worked Example: A Supremum at an Excluded Boundary
For fixed observed data, suppose the likelihood kernel is \(L(\theta)=\theta/(1+\theta)\), with parameter space \(\Theta=(0,1)\). Determine whether a maximum likelihood estimate exists.
Step 1: Check the slope. Differentiating gives \(L'(\theta)=1/(1+\theta)^2>0\) for every allowed \(\theta\). Thus the likelihood is strictly increasing throughout \((0,1)\), and there is no interior stationary point.
Step 2: Check the boundary behavior. As \(\theta\) approaches 1 from below, \(L(\theta)\) approaches \(1/2\). But \(1\notin\Theta\), and for every allowed \(\theta<1\), \(L(\theta)<1/2\). Therefore \(1/2\) is the supremum, not an attained maximum. There is no maximum likelihood estimate in this parameter space.
The common mistake avoided is to report \(\theta=1\) as the estimate merely because it gives the limiting value. It is not an allowed parameter value; every allowed value has a smaller likelihood.
Common Exam Traps
- Equating zero score with a maximum. A zero first derivative only identifies a stationary candidate. Check the second derivative and compare globally.
- Treating the second-derivative test as a global test. A negative second derivative at one point proves a strict local maximum under the stated conditions, not necessarily a global one.
- Ignoring a boundary. An endpoint can maximize the likelihood even when no interior score solution does. Include endpoints only when they belong to the parameter space.
- Using the log-likelihood at a zero. The logarithm is undefined where \(L=0\); use likelihood values or handle those points separately.
- Confusing a supremum with a maximum. A limiting value at an excluded boundary is not attained and does not yield an estimate.
Check Your Understanding
For each problem, identify whether the candidate is a global maximum, a boundary maximum, or an unattained supremum. The likelihoods below are specified for fixed observed data.
- Let \(L(\theta)=\exp[-(\theta-1)^2]\) for \(\theta\in[0,3]\). Find the global maximizer and state whether it is unique. (Answer: \(\hat{\theta}=1\), the unique global maximum.)
- Let \(L(\theta)=(\theta-3)^2\) for \(\theta\in[0,4]\). Find all global maximizers by comparing stationary points and endpoints. (Answer: \(\hat{\theta}=0\), the unique global maximum, with likelihood 9.)
- Let \(\ell(\theta)=2\theta-\theta^2\) for \(\theta\in[0,3]\). Find the stationary point and determine whether it is the global maximum. (Answer: \(\hat{\theta}=1\), the unique global maximum, since \(\ell''(\theta)=-2\) throughout.)
- Let \(L(\theta)=1/(1+\theta^2)\) for \(\theta\in(0,\infty)\). Does a maximum likelihood estimate exist? (Answer: No; the supremum is 1 as \(\theta\) approaches 0 from above, but 0 is excluded.)
- Let \(L(\theta)=\theta^2e^{-\theta}\) for \(\theta\in[0,\infty)\). Find the global maximizer, checking the boundary and the limit at infinity. (Answer: \(\hat{\theta}=2\), the unique global maximum, with likelihood \(4e^{-2}\approx0.5413\).)