Why Constraints Change the MLE
A likelihood can have its largest value at a parameter estimate that is not allowed by the model. A parameter may be restricted to a specified range, or several parameters may have to satisfy a joint restriction. In either case, the maximum likelihood estimate (MLE) is the feasible parameter value that maximizes the likelihood—not automatically the solution to an unconstrained score equation.
For a single parameter in a closed interval, the practical question is whether the likelihood reaches its largest value at an interior stationary point or at an endpoint. For a closed region with several parameters, the search must also account for its boundary: an edge or a corner can be optimal even when no interior stationary point is feasible.
Candidate Points on a Closed Interval
Suppose \(L(\theta)\) is continuous on \([a,b]\) and differentiable on its interior. The extreme value theorem guarantees that it attains a maximum somewhere on this closed interval. If that maximum is interior, Fermat’s theorem says \(L'(\theta)=0\) there. Otherwise, it must be at \(a\) or \(b\). Thus, when the stated smoothness conditions hold, check the feasible interior stationary points and both endpoints.
It is often easier to work with the log-likelihood \(\ell(\theta)=\log L(\theta)\), because the logarithm is strictly increasing: maximizing \(\ell\) and maximizing a positive \(L\) give the same parameter value. A score equation identifies interior candidates; it does not by itself establish that a candidate is a maximum or that it satisfies the constraint. If the likelihood is not differentiable at a feasible point, that point may also need to be checked. A likelihood with parameter-dependent support may require additional care; a score equation alone is not a complete candidate search in that setting.
Worked Example: A Normal Mean Restricted to an Interval
Assume \(X_1,X_2,X_3\) are independent observations from \(N(\mu,4)\), where the variance is known to be \(4\) and the unknown mean is restricted to \(10\leq\mu\leq14\). The observations are \(11,12,13\). Find the constrained MLE and compare it with the endpoints.
Method. Form the likelihood, find its interior stationary point, and evaluate the log-likelihood at that point and the interval endpoints. There is no interest rate or mortality assumption in this probability model.
Ignoring a positive factor that does not depend on \(\mu\), the likelihood is proportional to \(\exp\{ -\sum_{j=1}^3(X_j-\mu)^2/8\}\). Its log-likelihood, up to an additive constant, is \(\ell(\mu)=-[(11-\mu)^2+(12-\mu)^2+(13-\mu)^2]/8\).
Differentiate and set the score to zero: \( \ell'(\mu)=[(11-\mu)+(12-\mu)+(13-\mu)]/4=(36-3\mu)/4\). Thus \(36-3\mu=0\), giving the stationary point \(\mu=12\), which is feasible.
At \(\mu=12\), the sum of squared deviations is \(1+0+1=2\), so \(\ell(12)=-2/8=-0.25\), apart from the same constant. At the lower endpoint, the sum is \(1+4+9=14\), so \(\ell(10)=-14/8=-1.75\). At the upper endpoint, it is \(9+4+1=14\), so \(\ell(14)=-14/8=-1.75\). The interior value is larger than either endpoint value.
Therefore, the constrained MLE is \(\hat{\mu}=12\). As a check, the second derivative is \(\ell''(\mu)=-3/4<0\), so the log-likelihood is strictly concave and the stationary point is a maximum. The common mistake to avoid is assuming that a constraint always pushes the estimate to an endpoint: here the unconstrained estimate is feasible, and it remains the constrained MLE.
When the Unconstrained Estimate Is Outside
If the unconstrained stationary point lies outside a permitted interval, it cannot be the constrained MLE. For a likelihood that increases up to its unconstrained maximizer and decreases afterward, the constrained maximum is the closest feasible endpoint in the direction of that maximizer. This conclusion relies on the likelihood’s shape; it is not a general rule to “clip” every score solution to the interval. Check monotonicity or compare feasible candidates.
Worked Example: A Binomial Probability With a Lower Bound
Assume \(X\sim\operatorname{Binomial}(10,p)\) and the observed count is \(X=6\). The model restricts \(p\) to the closed interval \([0.65,0.90]\). Find the constrained MLE and verify that the upper endpoint is not better. There is no interest rate or mortality assumption in this binomial model.
The likelihood, omitting the parameter-free binomial coefficient, is \(L(p)=p^6(1-p)^4\). The log-likelihood is \(\ell(p)=6\log p+4\log(1-p)\), with score \(\ell'(p)=6/p-4/(1-p)=(6-10p)/[p(1-p)]\).
The unconstrained score equation gives \(6-10p=0\), so \(\hat p_{\text{unconstrained}}=0.60\). This is below the feasible interval. For every \(p\in[0.65,0.90]\), \(6-10p<0\), while \(p(1-p)>0\); hence \(\ell'(p)<0\) throughout the feasible interval. The log-likelihood is decreasing there, so its largest feasible value occurs at the lower endpoint.
As a numerical endpoint check, \(\ell(0.65)=6\log(0.65)+4\log(0.35)\approx-6.7840\). At the upper endpoint, \(\ell(0.90)=6\log(0.90)+4\log(0.10)\approx-9.8425\). The lower-endpoint value is larger. Therefore, the constrained MLE is \(\hat p=0.65\). The displayed log-likelihoods are rounded to four decimal places.
The common mistake is to report \(0.60\), the unconstrained estimate, without checking feasibility. A score solution outside the permitted parameter set is not a constrained MLE.
Closed Regions: Edges and Corners Matter
For several parameters, a closed feasible region can have an interior, edges, and corners. If the likelihood is continuous on a closed and bounded region, it attains a maximum there. An interior maximum of a differentiable likelihood must satisfy the appropriate stationary conditions, but a maximum on the boundary need not. A complete search must account for feasible boundary candidates as well as interior ones.
A useful way to organize a simple region problem is to ask whether the likelihood improves as a parameter moves toward the boundary. On an edge, the restriction reduces the feasible choices; a corner is where edges meet and must not be overlooked. The next example uses a region defined by nonnegative Poisson rates whose sum is bounded.
Worked Example: Two Poisson Rates With a Sum Constraint
Assume \(X_1\) and \(X_2\) are independent, with \(X_1\sim\operatorname{Poisson}(\lambda_1)\) and \(X_2\sim\operatorname{Poisson}(\lambda_2)\). The observed counts are \(x_1=4\) and \(x_2=4\). The feasible region is \(\lambda_1\geq0,\lambda_2\geq0,\lambda_1+\lambda_2\leq5\). Find the constrained MLE. There is no interest rate or mortality assumption in this model.
Ignoring factorial terms that do not depend on the parameters, the likelihood is \(L(\lambda_1,\lambda_2)=\lambda_1^4\lambda_2^4\exp[-(\lambda_1+\lambda_2)]\). The unconstrained MLEs would be \((4,4)\), which are not feasible because their sum is \(8>5\).
For positive rates, put \(s=\lambda_1+\lambda_2\). At any fixed \(s\), the product \(\lambda_1\lambda_2\) is largest when the rates are equal: \((\lambda_1-\lambda_2)^2\geq0\) implies \(\lambda_1\lambda_2\leq s^2/4\), with equality at \(\lambda_1=\lambda_2=s/2\). Since the likelihood at fixed \(s\) is proportional to \((\lambda_1\lambda_2)^4e^{-s}\), it is maximized by equal rates.
With equal rates, the likelihood is proportional to \((s/2)^8e^{-s}\), for \(0<s\leq5\). Its log, omitting a constant, is \(8\log(s/2)-s\), with derivative \(8/s-1\). This derivative is positive for \(0<s\leq5\), since \(8/s-1\geq8/5-1=0.6>0\). Thus the likelihood increases up to the largest feasible sum, \(s=5\). The maximizing point is on the boundary \(\lambda_1+\lambda_2=5\).
At that sum, equality of the rates gives \(\hat\lambda_1=\hat\lambda_2=2.5\). If either rate is zero, the likelihood is zero because the corresponding observed count is positive; such a corner cannot improve on the positive likelihood at \((2.5,2.5)\). Therefore the constrained MLE is \((2.5,2.5)\). The common mistake is to use the two unconstrained estimates \((4,4)\) despite their violating the sum constraint.
A Reliable Candidate-Checking Process
Write the interval or region explicitly, including whether its endpoints and boundary are allowed.
Use the sampling model and observed data. Remove only factors that do not depend on the parameters.
For one parameter, solve the score equation and retain only feasible solutions. Check whether each is a maximum rather than a minimum or other stationary point.
For an interval, evaluate both endpoints. For a region, examine its feasible edges and corners as well as any relevant interior candidates.
Choose the feasible candidate with the largest likelihood, or log-likelihood. Do not select an inadmissible candidate.
Common Exam Traps
- Reporting an infeasible score solution. A stationary point outside the allowed set cannot be the constrained MLE.
- Checking only the score equation. The score finds interior stationary points; it does not check endpoints, boundary edges, or corners.
- Assuming every constrained estimate is at a boundary. If an unconstrained maximizer is feasible, it may remain the constrained maximum, as in the normal example.
- Taking logarithms without checking the likelihood. The log-likelihood is appropriate where the likelihood is positive; boundary points with zero likelihood should be handled by evaluating the likelihood or its limiting behavior directly.
- Applying a smooth candidate rule without its assumptions. The endpoint-and-stationary-point checklist assumes continuity on the closed interval and differentiability in its interior. A likelihood with changing support can require a separate analysis.
Check Your Understanding
For each problem, identify the feasible maximizer rather than simply reporting an unconstrained score solution.
- Assume \(X\sim\operatorname{Binomial}(12,p)\), observe \(X=9\), and restrict \(p\) to \([0.60,0.80]\). Find the constrained MLE. (Answer: \(0.75\), the feasible interior score solution.)
- Assume \(X\sim\operatorname{Poisson}(\lambda)\), observe \(X=2\), and restrict \(\lambda\) to \([3,6]\). Use the log-likelihood’s direction over the interval to find the constrained MLE. (Answer: \(3\), the lower endpoint.)
- Assume independent observations \(X_1\sim\operatorname{Poisson}(\lambda_1)\), \(X_2\sim\operatorname{Poisson}(\lambda_2)\), observed as \(3\) and \(5\), with \(\lambda_1,\lambda_2\geq0\) and \(\lambda_1+\lambda_2\leq6\). Find the constrained MLE. (Answer: \((2.25,3.75)\).)
- Assume \(X_1,\ldots,X_5\) are independent \(N(\mu,9)\) observations with known variance \(9\). The data are \(4,5,7,8,11\), and \(\mu\in[6,10]\). Find the constrained MLE and state whether it is interior or a boundary value. (Answer: \(\hat{\mu}=7\), an interior value.)
- Assume \(X\sim\operatorname{Binomial}(8,p)\), observe \(X=2\), and restrict \(p\) to \([0.10,0.20]\). Compare the unconstrained score solution with the feasible interval and find the MLE. (Answer: \(0.20\), the upper endpoint.)