Score Contributions and Likelihood-Based Diagnostics

Why Examine Scores Observation by Observation?

A fitted likelihood model is usually judged first by its total likelihood and by inferences about its parameters. Yet the total can conceal how different observations contribute: large positive and negative score contributions may cancel, leaving a total score close to zero. Examining the contributions separately can show which cases are pulling the estimate in different directions and which cases merit closer substantive or data-quality review.

A score contribution is a local diagnostic, not a verdict about an observation. It depends on the model, the parameterization, and the fitted parameter value. A large contribution can reflect an unusual but valid outcome, a data error, an omitted feature of the model, or simply the variability expected under the assumed model. Nor does a small total score establish identification: it is a first-order condition at the fitted estimate, not a check that distinct parameter values imply distinct probability distributions.

Observation-Level Score Contributions

Suppose the data comprise \(n\) independent observations, with observation \(i\) having density or probability mass function \(f_i(y_i;\theta)\), where \(\theta\) is a \(k\)-dimensional parameter vector. Independence gives a log likelihood that is a sum of observation-level terms. The score contribution for observation \(i\) is the gradient of that term with respect to the parameter.

Definition: Write \(\ell_i(\theta)=\log f_i(y_i;\theta)\) for observation \(i\)'s log-likelihood contribution. Its score contribution is the \(k\)-vector \(s_i(\theta)=\partial\ell_i(\theta)/\partial\theta\). The total log likelihood and score are \(\ell(\theta)=\sum_{i=1}^n\ell_i(\theta)\) and \(S(\theta)=\sum_{i=1}^n s_i(\theta)\).

When the maximum-likelihood estimate \(\hat{\theta}\) is an interior point of the parameter space and the log likelihood is differentiable there, it satisfies the likelihood first-order condition \(S(\hat{\theta})=0\). Thus the individual vectors sum to zero at an interior maximum. This does not require each \(s_i(\hat{\theta})\) to be zero. Their cancellation is often precisely what makes the individual patterns informative.

The independence assumption is important for the observation-level decomposition as written. If the independent sampling units are clusters, for example, it is generally more appropriate to define one contribution per cluster using that cluster's joint likelihood. Treating dependent observations as independent contributions can give a misleading diagnostic even when the fitted marginal quantities look plausible.

Worked Example: Contributions in a Two-Group Logistic Model

Consider eight hypothetical independent binary outcomes. Let \(z_i\) equal 0 or 1, and suppose \(Y_i\) follows a Bernoulli distribution with probability \(p_i\), where \(\operatorname{logit}(p_i)=\beta_0+\beta_1z_i\). The data contain two successes and two failures for \(z=0\), and three successes and one failure for \(z=1\). Calculate the fitted probabilities and individual score vectors.

For a Bernoulli observation, its log-likelihood contribution is \(y_i\log(p_i)+(1-y_i)\log(1-p_i)\). Differentiating with respect to \((\beta_0,\beta_1)\) gives \(s_i(\beta)=(y_i-p_i)(1,z_i)^\mathsf{T}\). The group success proportions are \(2/4=0.5\) and \(3/4=0.75\). Because the model fits a separate logit for each of the two groups, these proportions give the maximum-likelihood fitted probabilities. Hence \(\hat{\beta}_0=\operatorname{logit}(0.5)=0\) and \(\hat{\beta}_1=\operatorname{logit}(0.75)-\operatorname{logit}(0.5)=\log(3)=1.099\).

For \(z=0\), a success contributes \((1-0.5)(1,0)^\mathsf{T}=(0.5,0)^\mathsf{T}\), and a failure contributes \((0-0.5)(1,0)^\mathsf{T}=(-0.5,0)^\mathsf{T}\). There are two of each, so these contributions sum to \((1-1,0)^\mathsf{T}=(0,0)^\mathsf{T}\).

For \(z=1\), a success contributes \((1-0.75)(1,1)^\mathsf{T}=(0.25,0.25)^\mathsf{T}\), and a failure contributes \((0-0.75)(1,1)^\mathsf{T}=(-0.75,-0.75)^\mathsf{T}\). The three successes and one failure therefore sum to \(3(0.25,0.25)^\mathsf{T}+(-0.75,-0.75)^\mathsf{T}=(0,0)^\mathsf{T}\). Across both groups the total score is zero, as required at this interior fit. The failure in the \(z=1\) group has a larger contribution in magnitude than any one success, but the contributions alone do not show that this observation is erroneous or incompatible with the model.

Reading Patterns Without Overinterpreting Them

A score vector points in the direction in parameter space in which an observation's log likelihood increases most rapidly, locally at the parameter value where the score is evaluated. Its components refer to particular parameters and can have different scales. A score of a given numerical size is therefore not automatically “large” in a useful, parameterization-independent sense. Re-expressing the parameters changes the coordinates of score vectors; raw component sizes should not be compared carelessly across parameters or models.

For interpretation, examine contributions in context: which parameters have large components, whether observations with a common covariate or subgroup show a recurring direction, and whether a small number of observations dominate a pattern. Contributions can also be summarized over a substantively meaningful group by summing \(s_i(\hat{\theta})\) within that group. A group sum need not be zero even though the overall sum is zero. A concentrated group pattern can motivate checking data coding, the sampling structure, or whether an important feature is absent from the model.

When a common scale is useful, the observed information at the estimate is \(J(\hat{\theta})=-\partial^2\ell(\hat{\theta})/\partial\theta\partial\theta^\mathsf{T}\). If this matrix is positive definite, one possible dimensionless diagnostic magnitude is \(s_i(\hat{\theta})^\mathsf{T}J(\hat{\theta})^{-1}s_i(\hat{\theta})\). It measures a contribution relative to the curvature of the total likelihood. It is a descriptive scaling, not a universal cutoff or a formal test; its meaning still depends on the model and on whether the local quadratic approximation is useful.

Case Deletion and Local Influence

A direct way to ask whether an observation matters to an estimate is to compare the full-data estimate with the estimate obtained after removing that observation. Re-fitting the model after each deletion can be computationally practical, but a local approximation explains how the score contribution relates to the change. Define \(\ell_{-i}(\theta)=\ell(\theta)-\ell_i(\theta)\), the log likelihood with observation \(i\) removed, and let \(J_{-i}(\hat{\theta})\) be its observed information evaluated at the full-data estimate.

Local deletion approximation: If the deleted-data likelihood is smooth, its information is nonsingular near \(\hat{\theta}\), and the change in its score is adequately approximated by a first-order expansion, then \( \hat{\theta}_{-i}-\hat{\theta}\ \approx\ -J_{-i}(\hat{\theta})^{-1}s_i(\hat{\theta}) \). When removing one observation has little effect on total curvature, \(J_{-i}(\hat{\theta})\) may in turn be approximated by \(J(\hat{\theta})\).

The approximation follows from the score equation, not from treating the score as an effect size. Since \(S(\hat{\theta})=0\), the deleted-data score at the full-data estimate is \(S_{-i}(\hat{\theta})=S(\hat{\theta})-s_i(\hat{\theta})=-s_i(\hat{\theta})\). A first-order expansion at \(\hat{\theta}\), writing \(\Delta=\hat{\theta}_{-i}-\hat{\theta}\), gives \(S_{-i}(\hat{\theta}+\Delta)\approx -s_i(\hat{\theta})-J_{-i}(\hat{\theta})\Delta\). Setting this approximation to zero and solving yields \(\Delta\approx-J_{-i}(\hat{\theta})^{-1}s_i(\hat{\theta})\). A large predicted change flags a case for closer inspection, but the approximation can be poor when deletion changes the fit substantially. In that situation, refit the model and report the actual change.

Worked Example: Comparing Exact and Approximate Deletion Effects

Suppose five hypothetical independent binary outcomes are fitted with an intercept-only logistic model. Two are successes and three are failures. Let \(\eta\) be the logit of the common success probability \(p\). Find the fitted value, the scores, and the effect of deleting one success or one failure.

The fitted probability is \(\hat{p}=2/5=0.4\), so \(\hat{\eta}=\log(0.4/0.6)=\log(2/3)=-0.4055\). For this model each contribution is \(s_i(\hat{\eta})=y_i-\hat{p}\): each success contributes \(0.6\), and each failure contributes \(-0.4\). Their total is \(2(0.6)+3(-0.4)=0\).

Deleting a success leaves one success and four failures, so the new probability estimate is \(1/5=0.2\), and the new logit is \(\log(0.2/0.8)=\log(1/4)=-1.386\). The exact change from the full-data estimate is \(-1.386-(-0.4055)=-0.9808\). Deleting a failure leaves two successes and three failures, giving probability \(2/4=0.5\) and logit 0; the exact change is \(0-(-0.4055)=0.4055\).

At the full-data estimate, the observed information for one Bernoulli logit observation is \(p(1-p)\). After deleting one case, the information from the remaining four cases, evaluated at \(\hat{\eta}\), is \(J_{-i}=4(0.4)(0.6)=0.96\). The deletion approximation is \(-s_i/J_{-i}\). For a success it is \(-0.6/0.96=-0.625\); for a failure it is \(-(-0.4)/0.96=0.4167\). These directions agree with the exact changes, and the approximations are closer to the exact changes than one based on full-sample information would be. Their remaining differences are not surprising: with only five observations, deletion changes the fitted probability and the curvature appreciably. The refitted estimates are the relevant comparison here.

Contribution Patterns and Model Tension

An unusually large score contribution can indicate that a case has substantial local influence, but influence and model fit are different questions. Influence concerns how the estimate responds to a case; model tension concerns whether a pattern in the data is poorly represented by the model. A single unusual contribution may be consistent with ordinary sampling variation. Repeated contributions in a similar direction within a subgroup can instead suggest a feature worth investigating, such as heterogeneity or a missing predictor. Any such explanation requires substantive judgment and, where appropriate, further model assessment.

Worked Example: A Large Contribution in a Poisson Intercept Model

Consider five hypothetical independent counts \(1,1,2,2,10\), modeled as identically distributed Poisson outcomes with common rate \(\lambda\). Use the log-rate parameter \(\eta=\log(\lambda)\) to locate a potentially influential observation.

For one count \(y_i\), the log-likelihood contribution is \(y_i\eta-\exp(\eta)-\log(y_i!)\). Its score with respect to \(\eta\) is \(s_i(\eta)=y_i-\exp(\eta)=y_i-\lambda\). The likelihood first-order condition is \(\sum_i(y_i-\lambda)=0\), giving \(\hat{\lambda}=(1+1+2+2+10)/5=16/5=3.2\) and \(\hat{\eta}=\log(3.2)=1.163\).

The individual score contributions are \(-2.2,-2.2,-1.2,-1.2,\) and \(6.8\). Their sum is \(-2.2-2.2-1.2-1.2+6.8=0\). The count of 10 contributes much more strongly in the positive direction than any other observation contributes in the negative direction. Removing it changes the estimated rate to \((1+1+2+2)/4=1.5\), so the exact change in log rate is \(\log(1.5)-\log(3.2)=\log(1.5/3.2)=-0.7577\).

For comparison, the Poisson observed information in \(\eta\) from five observations is \(5\hat{\lambda}=16\). Using full-sample information gives the rough change \(-6.8/16=-0.425\), which has the correct direction but understates the actual change. Removing the count has a substantial effect on this small sample. The score identifies the count as a case to review; it does not show whether the value is a data error, a valid rare outcome, or evidence that a common-rate Poisson model is inadequate.

Common Mistakes and Appropriate Use

Assuming every contribution must vanish at the maximum. The first-order condition requires the total score to be zero at an interior maximum. Individual contributions can be nonzero and cancel one another.
Treating a large score as proof of a bad observation. A large contribution can identify a case worth checking, but it is not a reason by itself to discard that case. Verify the data and assess the model before deciding how to proceed.
Calling a zero total score evidence of identification. The score equation is a local stationarity condition. It does not establish that the parameter is uniquely determined by the probability model, nor does a score-contribution diagnostic test identification.
Comparing raw score sizes without considering scale. Components depend on parameter units and parameterization. Interpret them in context or use a stated curvature-based scaling, and treat any resulting magnitude as diagnostic rather than as a universal threshold.
Trusting a deletion approximation after a major change in fit. The approximation relies on local curvature around the full-data estimate. If it predicts a large change, refitting the model after deletion is especially important.

Key takeaway: Observation-level scores decompose the likelihood’s first-order condition into contributions that can reveal cancellation, local influence, and patterns worth investigating. They guide diagnostics; they do not, on their own, establish an error, model failure, or identification.

Check Your Understanding

Use the definitions and examples above to answer these questions.

Why can the total score equal zero at an interior maximum even when no individual score contribution is zero?
For a Bernoulli logistic model, what is the score contribution of an observation with covariate vector \(x_i\), outcome \(y_i\), and fitted probability \(p_i\)?
What assumptions and local approximation lead to the case-deletion formula?
A case has a large score contribution. Give two different explanations that do not imply a recording error.
Why is a zero total score not a test of identification?