From a Likelihood to a Score Equation
A likelihood expresses how compatible observed data are with each possible parameter value. Maximum likelihood estimation seeks a parameter value that maximizes this function. Differentiation provides a useful way to locate some possible maxima: at a differentiable interior maximum, the derivative of the log-likelihood is zero. This equation is called the score equation.
Solving a score equation is an important step, but it does not by itself complete maximum likelihood estimation. A root may not be in the parameter space, may not maximize the likelihood, or may fail to exist even when the likelihood has a boundary supremum. The distinction between finding a root and establishing an MLE is essential.
The likelihood and log-likelihood have the same maximizers wherever the likelihood is positive, because the logarithm is strictly increasing. The log-likelihood is often easier to differentiate: products of probability or density contributions become sums. Constants that do not depend on \(\theta\) can be retained or omitted, since their derivatives are zero.
Why Differentiating the Log-Likelihood Works
Suppose \(X_1,\ldots,X_n\) are independent observations from a model indexed by \(\theta\), and their probability mass functions or densities are \(f(x_i;\theta)\). For observed values \(x_1,\ldots,x_n\), the likelihood is \(L(\theta)=\prod_{i=1}^n f(x_i;\theta)\). Where every factor is positive and differentiable, taking logarithms turns this product into a sum.
The score is also the likelihood derivative divided by the likelihood. This identity explains why a differentiable interior maximum must be a score root.
To justify the identity, differentiate \(\ell(\theta)=\log L(\theta)\) using the chain rule: \(\ell'(\theta)=(1/L(\theta))L'(\theta)\). At an interior maximum where \(L\) is differentiable and positive, Fermat’s theorem gives \(L'(\theta)=0\), so \(U(\theta)=0\). This is a necessary condition under those assumptions, not a sufficient condition for a maximum. A boundary maximum need not have derivative zero, and a score root can be a minimum or another nonmaximizing stationary point.
Use the probability or density of the observed data and state the allowed parameter values.
Convert products to sums and retain parameter-dependent terms accurately.
Differentiate with respect to the parameter to obtain the score function.
Solve the equation, then check whether each root belongs to the allowed parameter space.
A score root is a candidate for an MLE; a separate maximization assessment is still required.
Worked Score Equations
Worked Example: A Three-Category Observation Model
A hypothetical inspection records each item as category A, B, or C. Items are independent, with probabilities \(P(A)=\theta/4\), \(P(B)=\theta/4\), and \(P(C)=1-\theta/2\), where \(0<\theta<1\). In a sample of six items, one is in A, one is in B, and four are in C. Derive and solve the score equation.
Method. Use the independent categorical likelihood, take its logarithm, and differentiate. This example avoids treating a score root as proof of a maximum.
The probabilities sum to one: \[ \frac{\theta}{4}+\frac{\theta}{4}+1-\frac{\theta}{2}=1. \] For the observed counts, the likelihood is
Taking logarithms gives
Differentiate term by term. The first term contributes \(2/\theta\), the constant contributes zero, and the last term contributes \(4(-1/2)/(1-\theta/2)\). Therefore,
Set the score to zero and solve:
The root \(2/3\) lies in the stated parameter space \(0<\theta<1\). Substitution gives \(P(A)=P(B)=1/6\) and \(P(C)=2/3\), which sum to one. Thus \(2/3\) is an interior score-root candidate. The score equation alone does not establish that it maximizes the likelihood.
Common mistake. Treating the four observations in category C as one observation would give the wrong likelihood power and hence the wrong score. Each observed category contributes its probability once, so the C probability appears to the fourth power.
Worked Example: A Score from a Continuous Density
A hypothetical instrument produces independent measurements with density \[ f(x;\theta)=\frac{1+\theta x}{2},\qquad -1<x<1,\quad -1<\theta<1. \] Two recorded values are \(x_1=2/5\) and \(x_2=-3/10\). Find the score root.
Method. Form the product of the two density contributions, take logarithms, and solve the resulting score equation. This avoids differentiating a product unnecessarily.
The density is valid on the stated support: its integral is \[ \int_{-1}^1\frac{1+\theta x}{2}\,dx =\frac{1}{2}\left(2+\theta\int_{-1}^1x\,dx\right)=1, \] and \(1+\theta x>0\) when \(-1<\theta<1\) and \(-1<x<1\). The likelihood for these observations is
The log-likelihood and score are
Within \(-1<\theta<1\), both denominators are positive, so cross-multiplication is valid. Setting the score to zero gives
The value \(5/12\) is in the parameter space. A substitution check gives \[ 1+\frac{2}{5}\frac{5}{12}=\frac{7}{6}, \qquad 1-\frac{3}{10}\frac{5}{12}=\frac{7}{8}. \] The two terms of the score are then \((2/5)/(7/6)=12/35\) and \((3/10)/(7/8)=12/35\); their difference is zero. This confirms the score root, but not yet that it is a maximum.
Common mistake. The derivative of the second logarithm is negative because its inner expression has derivative \(-3/10\). Omitting that minus sign changes the score equation.
Why a Score Root Is Only a Candidate
The derivative condition describes an interior stationary point. It does not state what kind of stationary point has been found. Also, the parameter space may exclude a root, or the likelihood may achieve its largest value at a boundary where the derivative condition does not apply. A model whose support depends on the parameter can require additional care: observations may be possible for some parameter values and impossible for others, so the likelihood’s domain can change with \(\theta\).
Worked Example: A Score Root That Is Not a Maximum
A hypothetical single categorical observation \(Y\) has probabilities \[ P_\theta(Y=1)=\frac{1+\theta^2}{2}, \qquad P_\theta(Y=0)=\frac{1-\theta^2}{2}, \qquad -1<\theta<1. \] The observed value is \(Y=1\). Find the score root and show why it is not an MLE.
Method. Differentiate the log-likelihood, then compare likelihood values. This illustrates why a zero derivative is a candidate condition rather than a maximizing conclusion.
The likelihood and its logarithm are
Differentiating gives
The denominator is positive throughout the parameter space, so \(U(\theta)=0\) gives \(\theta=0\). But the likelihood at this root is \(L(0)=1/2=0.5000\). At the allowed value \(\theta=4/5\),
Since \(0.8200>0.5000\), the score root \(\theta=0\) is not a likelihood maximum. In fact, as \(\theta\) approaches either excluded endpoint \(-1\) or \(1\), the likelihood approaches 1, but no parameter value in the open interval attains 1. Thus this model has no MLE in its stated parameter space. The calculation demonstrates both that a score root need not maximize and that a likelihood can have a boundary supremum without an attained maximum.
Common mistake. Reporting \(\theta=0\) as the MLE merely because it solves the score equation ignores the likelihood values elsewhere in the parameter space.
Conditions and Exam Traps
Before interpreting a score root, check the conditions behind the calculation. The log-likelihood must be defined at the relevant parameter values, and its derivative must exist there. The familiar zero-score condition applies to a differentiable interior maximum. It does not rule out boundary solutions, and a zero derivative is not sufficient to establish a maximum.
- Check the parameter space. An algebraic root outside the allowed parameter values is not an admissible candidate.
- Track signs in chain rules. A logarithm of a decreasing function contributes a negative derivative.
- Do not drop parameter-dependent terms. Terms constant in the data but dependent on \(\theta\) affect the score.
- Distinguish a score root from an estimator. The root is calculated from the observed data; whether it is an MLE requires a further maximization assessment.
- Be cautious when support depends on \(\theta\). A formula obtained by differentiating within one region may not describe the likelihood over its entire domain.
Check Your Understanding
For each problem, derive the score equation where applicable and keep the conclusion limited to what the score calculation establishes.
- A hypothetical categorical model has probabilities \(\theta/3,\theta/3,1-2\theta/3\), for \(0<\theta<1\). Independent observations produce counts \(2,1,3\), respectively. What interior score root results? (Answer: \(\theta=3/4\), an interior score-root candidate.)
- A density is \(f(x;\theta)=(1+\theta x)/2\) for \(-1<x<1\) and \(-1<\theta<1\). Independent observations are \(1/2\) and \(-1/4\). What value solves the algebraic score equation, and is it an interior root? (Answer: the algebraic root is \(\theta=1\), which is excluded; there is no interior score root.)
- For \(0<\theta<1\), suppose the observed-data likelihood is proportional to \(\theta^3(1-\theta)^2\). Derive the score root and state its status. (Answer: \(\theta=3/5\), an interior score-root candidate.)
- Suppose \(\theta>0\) and the log-likelihood is \(\ell(\theta)=-4\log\theta-12/\theta+C\), where \(C\) does not depend on \(\theta\). Find its score root. (Answer: \(\theta=3\), an interior score root.)
- For \(0<\theta<1\), a likelihood is proportional to \((1-\theta)^4\). Does its score have an interior root, and where does the likelihood’s supremum occur? (Answer: no interior score root; the supremum is approached as \(\theta\) tends to 0, which is excluded.)