Why Refit After Deleting a Case?
A residual describes how an observation disagrees with a fitted model; it does not, by itself, show how much the model depends on that observation. A case can have a large residual yet make only a modest difference to the estimates, or it can substantially alter a coefficient or fitted probability. Case-deletion analysis addresses this by removing a selected observation, refitting the same model to the remaining data, and comparing the results with the full-data fit.
For a generalized linear model (GLM), suppose the independent responses \(Y_1,\ldots,Y_n\), conditional on their predictors, have means \(\mu_i\) and follow a specified exponential-family distribution. The model relates the mean to predictors through a link: \(g(\mu_i)=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}\), possibly with an offset. The distribution, link, predictors, and any offsets are held fixed in a deletion comparison. Only the selected case is removed.
The goal is not to label an observation “bad” merely because some quantity changes. Instead, ask whether deletion changes the interpretation of important coefficients, materially shifts relevant predicted means or probabilities, or changes a substantive conclusion. These are refit comparisons, not a new model-selection procedure.
What to Compare
A useful comparison starts with the quantities that matter for the application. Coefficients may be compared on their original scale and, when useful, through the effects they imply, such as odds ratios in logistic regression. Fitted probabilities or means should be compared at the same predictor settings. Deviance can also be reported, but the full-data and deleted-data deviances are calculated from different sets of observations; their difference is not automatically a standard model-comparison test.
| Quantity | Case-deletion comparison | Interpretive question |
|---|---|---|
| Coefficient estimates | \(\widehat{\boldsymbol{\beta}}\) versus \(\widehat{\boldsymbol{\beta}}_{(-i)}\) | Do signs, magnitudes, or practical interpretations change? |
| Fitted means or probabilities | Compare at common predictor values | Do predictions important to the application shift materially? |
| Residual deviance | Report \(D\) and \(D_{(-i)}\), noting their sample sizes | Does the retained data have a different overall discrepancy from its fitted model? |
For logistic regression, a fitted mean is a probability. For a Poisson model, it is an expected count for the observation’s exposure and predictors. In either case, comparing different cases’ fitted values can confuse the effect of deleting a case with the effect of changing predictors. A clean comparison holds predictor values fixed.
Why the Estimates Change
The maximum-likelihood estimates solve score equations: the derivatives of the log-likelihood with respect to the model parameters equal zero. Deleting case \(i\) removes its contribution to those equations. The refitted estimates must then balance the contributions from the remaining cases, so they generally differ from the full-data estimates. This also explains why deleting a case does not simply subtract its residual from a coefficient: the refit changes the balance of the whole estimating system.
For an independent Poisson model with a log link, \(\log(\mu_i)=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}\), the score equation is \(\sum_i \mathbf{x}_i(y_i-\mu_i)=\mathbf{0}\). After deleting case \(k\), it becomes \(\sum_{i\ne k}\mathbf{x}_i(y_i-\mu_i)=\mathbf{0}\), with the means evaluated at the refitted coefficients. The deleted case’s contribution is gone, and all remaining fitted means may adjust.
Worked Refit Examples
Worked Example: Deleting a Large Count From a Poisson Model
A hypothetical insurer models independent claim counts \(Y_i\) as Poisson with a common mean \(\mu\), using an intercept-only log-link model. Four hypothetical policy groups have counts \(1, 2, 3,\) and \(10\), with equal exposure. Assess deletion of the observation with count \(10\), comparing the estimated mean and residual deviance. Use \(D=2\sum_i[y_i\log(y_i/\widehat{\mu})-(y_i-\widehat{\mu})]\), with the \(y_i=0\) term defined as zero.
Full-data estimate. The count total is \(1+2+3+10=16\), and there are four observations. Thus \(\widehat{\mu}=16/4=4\), and \(\widehat{\beta}_0=\log(4)=1.3863\) to four decimals.
Full-data deviance. The four contributions are \(2[1\log(1/4)-(1-4)]=3.2274\), \(2[2\log(2/4)-(2-4)]=1.2274\), \(2[3\log(3/4)-(3-4)]=0.2739\), and \(2[10\log(10/4)-(10-4)]=6.3258\). Therefore \(D=3.2274+1.2274+0.2739+6.3258=11.0545\).
Refit without the count of \(10\). The remaining total is \(1+2+3=6\), with three observations, so \(\widehat{\mu}_{(-i)}=6/3=2\) and \(\widehat{\beta}_{0,(-i)}=\log(2)=0.6931\). The estimated mean decreases by \(4-2=2\), or 50% of its full-data value.
The retained-data deviance contributions at mean \(2\) are \(2[1\log(1/2)-(1-2)]=0.6137\), \(2[2\log(2/2)-(2-2)]=0\), and \(2[3\log(3/2)-(3-2)]=0.4328\). Hence \(D_{(-i)}=1.0465\). The model’s estimated count level is clearly sensitive to this observation. The deviance is also smaller, but that calculation uses only three observations rather than four, so it does not establish that deleting the case improved the model.
Method and mistake avoided. The method is to recompute the maximum-likelihood estimate from the retained counts, then evaluate deviance on those same retained observations. Do not compare the two deviance totals as though they came from identical data sets.
Worked Example: Deleting a Success From an Intercept-Only Logistic Model
A hypothetical review team records binary outcomes for ten independent accounts. Each account has the same modeled success probability \(p\), with a logit link \(\log[p/(1-p)]=\beta_0\). Four outcomes are successes and six are failures. Delete one success and compare the coefficient, fitted probability, odds, and deviance. For individual Bernoulli observations, use \(D=-2\sum_i[y_i\log(\widehat p)+(1-y_i)\log(1-\widehat p)]\).
Full-data fit. The likelihood estimate is \(\widehat p=4/10=0.4\). Its odds are \(0.4/0.6=2/3\), so \(\widehat\beta_0=\log(2/3)=-0.4055\). Every account has fitted probability \(0.4\).
Full-data deviance. The four successes contribute \(-2(4\log 0.4)\), and the six failures contribute \(-2(6\log 0.6)\). Thus \(D=-2[4(-0.916291)+6(-0.510826)]=13.4602\).
Refit after deletion. There are now three successes among nine retained accounts, so \(\widehat p_{(-i)}=3/9=1/3\). The odds are \((1/3)/(2/3)=1/2\), and \(\widehat\beta_{0,(-i)}=\log(1/2)=-0.6931\). The coefficient change is \(-0.6931-(-0.4055)=-0.2876\); the fitted probability decreases by \(0.4-1/3=0.0667\), to four decimals.
The refitted deviance is \(-2[3\log(1/3)+6\log(2/3)]=11.4573\). As in the Poisson example, this is a deviance for a different set of observations, not a like-for-like comparison of two fits on the same response vector.
Method and mistake avoided. Re-estimate the success proportion and then transform it to the log-odds scale. Do not treat a change in \(\beta_0\) as a change in probability of the same numerical size: the logit and probability scales are different.
Worked Example: Deleting a Case From a Poisson Model With a Group Predictor
A hypothetical analyst models independent counts \(Y_i\sim\operatorname{Poisson}(\mu_i)\) with a log link, \(\log(\mu_i)=\beta_0+\beta_1z_i\). Let \(z=0\) for group A and \(z=1\) for group B; exposure is one unit for every observation. Group A has counts \(1,2,3\), and group B has counts \(2,3,4\). Delete the group A observation with count \(3\). Compare the coefficients, fitted means, and residual deviance.
Full-data fit. For this model, the fitted mean within each group equals that group’s sample average. Group A has mean \((1+2+3)/3=2\), and group B has mean \((2+3+4)/3=3\). Therefore \(\widehat\beta_0=\log 2=0.6931\), and \(\widehat\beta_1=\log 3-\log 2=\log(3/2)=0.4055\). The fitted means are 2 for group A and 3 for group B.
Full-data deviance. Group A’s contributions at mean 2 are \(0.6137, 0,\) and \(0.4328\), totaling \(1.0465\). Group B’s contributions at mean 3 are \(2[2\log(2/3)-(2-3)]=0.3781\), \(0\), and \(2[4\log(4/3)-(4-3)]=0.3015\), totaling \(0.6796\). Thus \(D=1.0465+0.6796=1.7261\), subject to rounding.
Refit without the selected case. Group A’s retained mean is \((1+2)/2=1.5\); group B’s mean remains 3. Consequently, \(\widehat\beta_{0,(-i)}=\log(1.5)=0.4055\), and \(\widehat\beta_{1,(-i)}=\log 3-\log(1.5)=\log 2=0.6931\). The intercept changes by \(-0.2877\), while the group effect changes by \(0.2877\). Predictions at the two group settings change from \(2\) to \(1.5\) for group A and stay at \(3\) for group B.
The retained group A deviance is \(2[\log(1/1.5)-(1-1.5)]+2[2\log(2/1.5)-(2-1.5)]=0.1891+0.1507=0.3398\). Group B still contributes \(0.6796\), so \(D_{(-i)}=1.0194\). The coefficients and group A fitted mean change even though the group B prediction does not.
Method and mistake avoided. Refit the model with the same group predictor, then compare coefficients and predictions at fixed group values. Do not interpret the deviance decrease alone as evidence of improvement: the refit omits one response, and its deviance is evaluated on fewer cases.
Interpreting a Case-Deletion Comparison
A case-deletion result is most useful when it is tied to a decision. For example, a small coefficient change may still matter if it reverses the sign of a predictor’s estimated effect. Conversely, a noticeable coefficient change may have little practical consequence if the predicted probabilities in the relevant range barely move. State which parameter or prediction matters, compare it on a consistent scale, and explain the practical consequence.
There is no universal cutoff for a “large” case-deletion change. The relevant scale depends on the response, model, sample size, and application. For probability predictions, report absolute changes at meaningful predictor settings; for coefficients, explain the corresponding model effect when possible. If a deletion changes a conclusion, investigate the case and the model specification rather than automatically removing the observation.
- Keep the model fixed. A deletion comparison changes the data, not the chosen distribution, link, predictor set, or offset.
- Compare predictions fairly. Evaluate the full and refitted models at the same covariate values.
- Report the data size. The deleted fit uses one fewer case, so its deviance is computed over a different set.
- Do not equate influence with error. A consequential observation may be valid and important; verify data quality and context before drawing conclusions.
Case-deletion refitting complements residual-based diagnostics and other influence summaries by directly answering a practical question: what would the model say if this case were absent? It is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus, where interpretation matters as much as computation.
Check Your Understanding
For each problem, identify the relevant refit comparison and check that your conclusion accounts for the changed data set.
- An intercept-only logistic model has 3 successes and 5 failures. One failure is deleted. Find the full and deleted fitted probabilities and intercepts. (Answer: \(p=3/8=0.3750\), \(\widehat\beta_0=\log(3/5)=-0.5108\); after deletion, \(p_{(-i)}=3/7=0.4286\), \(\widehat\beta_{0,(-i)}=\log(3/4)=-0.2877\).)
- An intercept-only Poisson model has counts \(0,1,2,5\), with equal exposure. A case with count 5 is deleted. Find the full and deleted fitted means. (Answer: \(2\) and \(1\), respectively.)
- A Poisson log-link model has a group indicator. Group A counts are \(2,4,6\); group B counts are \(3,3,6\). Find the fitted group means before deletion, then delete the group B case with count 6 and find the refitted means. (Answer: full means A \(=4\), B \(=4\); refitted means A \(=4\), B \(=3\).)
- A logistic fit has a full-data fitted probability of \(0.30\) at a specified predictor value and a refitted probability of \(0.36\) after one case is deleted. What is the absolute probability change, and what must be held fixed for the comparison to be interpretable? (Answer: an increase of \(0.06\); compare at the same predictor value under the same model specification.)
- A deleted-data fit has smaller deviance than the full-data fit. Is this alone evidence that the deleted-data model is superior? Explain. (Answer: no; the deviances are calculated on different sets of observations, so the decrease alone is not a like-for-like model comparison.)