Why Examine the Influence of Individual Data?
A model-based development decision may depend on a predicted response, a probability of meeting a target, or a choice among doses. A fitted model summarizes the data, but a small number of observations can sometimes have a large effect on that summary. If removing one patient’s data changes a decision from “go” to “no-go,” that is useful information about the decision’s dependence on the observed data. It is not, by itself, evidence that the patient’s data are erroneous.
Influence diagnostics address a specific question: How much would the decision-relevant result change if one observational unit were left out and the model were refit? The answer depends on what counts as a unit, which model is fitted, and which decision quantity is examined. For example, a unit might be a participant with all of their repeated measurements, rather than one measurement record. The unit should match the independent contribution assumed by the analysis.
Measure Changes in the Decision Quantity
Let \(D\) denote the full data and \(D_{-i}\) the data after deleting unit \(i\). Let \(q(D)\) be a decision-relevant model output, such as a predicted response at a target dose or the estimated probability of meeting a development target. A direct influence measure is the signed change
The sign shows the direction of change; the magnitude shows its size in the units of \(q\). If the decision is a binary rule, such as “proceed if \(q\geq c\),” record both the numerical change and whether the decision changes sides of the threshold. For choices among several options, compare the selected option and, where useful, the difference in decision scores between the leading options. A small numerical shift can matter near a threshold, while a larger shift may not affect a decision far from it.
A proper case-deletion comparison uses the same outcome definition, model specification, target population, and decision rule for the full and deleted data. Parameters must be re-estimated after deletion. If the analysis procedure includes data processing or model selection that is meant to be repeated for each dataset, specify whether those steps are repeated too; otherwise the diagnostic can understate the sensitivity of the full procedure. For Bayesian analyses, the analogous comparison refits the model and recomputes the posterior decision quantity without the deleted unit.
A Simple Derivation for a Mean-Based Decision
The sample mean provides a transparent example of how a case affects a fitted quantity. Suppose the \(n\) observations are \(y_1,\ldots,y_n\), with mean \(\bar{y}\). After deleting \(y_i\), the remaining observations sum to \(n\bar{y}-y_i\), so
Subtracting the full-data mean gives the exact deletion change:
Thus, for a mean, a unit’s influence grows with its distance from the full-data mean and decreases as the sample size grows. This algebra describes the change in the mean, not whether the observation is wrong. It also does not replace the full refit required for more complex models, where deleting one unit can change several fitted parameters together.
Worked Example: Does One Response Change a Threshold Decision?
Hypothetical data. A development team uses the mean improvement in a biomarker, measured in mg/dL, as a simplified decision quantity. Eight independent participants have improvements of 11, 12, 9, 10, 11, 12, 10, and 25 mg/dL. Under this illustrative rule, proceed if the fitted mean is at least 11.5 mg/dL.
The full-data sum is \(100\) mg/dL, so the fitted mean is
The rule therefore says proceed, because \(12.5\geq 11.5\). If the participant with an improvement of 25 mg/dL is deleted, the remaining sum is \(75\) mg/dL and
The decision changes to do not proceed because \(10.714<11.5\). The deletion formula gives the same change: \((12.5-25)/(8-1)=-1.786\) mg/dL. This participant is influential for this threshold decision. The calculation alone does not establish that 25 mg/dL is implausible or should be excluded; that requires a separate review of the measurement and study context.
Decision Changes Can Occur Without an Extreme Measurement
Influence is defined by the effect on a result, not by how surprising a raw value looks. For a threshold decision, a unit can be influential because the full-data estimate is close to the threshold. In a regression model, a unit can also matter because its covariates place it in a part of the design space that is sparsely represented. Residual size and influence are therefore related but distinct: a large residual indicates that a response differs from its fitted value, while leverage describes how unusual a unit’s covariate values are relative to the other observations. A combination of leverage and residual can produce substantial influence on fitted coefficients or predictions.
Worked Example: A Response Rate Near the Decision Boundary
Hypothetical data. In a small dose-finding cohort, 6 of 10 participants meet a prespecified response criterion. A binomial model with a common response probability is fitted by maximum likelihood, so the estimated response probability is the observed proportion. The simplified decision rule is to proceed if the estimate is at least 0.60.
For the full data, \(\hat{p}=6/10=0.600\), exactly at the threshold, so the rule says proceed. Delete one responder: the estimate becomes \(5/9=0.5556\), below the threshold, and the decision changes. Its influence change is \(0.5556-0.600=-0.04444\), or -4.444 percentage points. Delete one nonresponder instead: the estimate is \(6/9=0.6667\), an increase of 0.06667, or 6.667 percentage points; the decision remains proceed.
The model assumption here is that the observations are independent Bernoulli outcomes with a common response probability. In practice, dependence, variation in response probability across participants, or a different analysis model may require a different refit. The example also shows why reporting only whether the decision flips is incomplete: both the numerical change and the threshold position help describe sensitivity.
Regression Influence on a Target-Dose Prediction
For a fitted dose-response model, a useful diagnostic is often the change in the predicted outcome at a decision-relevant dose, rather than the change in a coefficient considered in isolation. The next example uses ordinary least squares (OLS) for clarity. OLS fits a straight line \(y=b_0+b_1x\) by minimizing the sum of squared residuals. Its conventional inferential assumptions include a correctly specified linear mean, independent errors with mean zero and constant variance; normal errors are additionally used for exact small-sample \(t\)-based inference. Here the calculation illustrates influence, not a claim that a four-point dataset can validate those assumptions.
Worked Example: Deleting a High-Leverage Dose Observation
Hypothetical data. Four dose-response observations are \((x,y)=(1,2),(2,4),(3,6),(4,20)\), where dose \(x\) is in mg and response \(y\) is in response units. The fitted line is used to predict response at 3.5 mg. Under a simplified rule, select the dose strategy if predicted response is at least 10 units.
For the full data, \(\bar{x}=2.5\), \(\bar{y}=8\), and
The full-data prediction at 3.5 mg is \(-6+(5.6)(3.5)=13.6\) response units, so the rule selects the strategy. After deleting the observation at 4 mg, the remaining three points lie exactly on \(y=2x\). The prediction at 3.5 mg is then \(7\) response units, below the threshold. The deletion change is \(7-13.6=-6.6\) response units, and the decision flips.
The point at 4 mg has high leverage in this tiny design: its OLS leverage is \(1/4+(4-2.5)^2/5=0.70\). Its full-model residual is \(20-13.6=3.6\) response units. These figures help explain why it merits investigation, but they do not establish an error. The dataset is too small to support a reliable assessment of the linear model assumptions, and the team should not treat this prediction as a validated dose recommendation.
Investigate Influence; Do Not Automatically Delete
An influential observation and an implausible observation are different findings. Influence is a property of an observation’s effect on a specified fitted result. Implausibility concerns whether the recorded value could reasonably have arisen given the measurement process, protocol, and scientific context. A value can be influential but valid, or unusual but have little impact on the decision. Data review should therefore follow, rather than be replaced by, an influence calculation.
A practical investigation asks whether the record is correctly transcribed, whether units and visit times are correct, whether an assay or instrument issue is documented, and whether the observation belongs to the analysis population as defined. If a problem is confirmed, address it using prespecified data-handling rules and document the reason. If no error is found, retain the observation in the primary analysis and report the sensitivity analysis. Excluding data solely because their removal produces a preferred decision would make the conclusion difficult to defend.
The most useful diagnostic is usually tailored to the decision. A parameter-change measure or a general regression influence statistic can identify cases affecting the fitted model, but a large change in a coefficient does not necessarily matter at the target dose. Conversely, a modest coefficient change can shift a decision quantity across a threshold. Recomputing the decision-relevant prediction or utility after deletion makes that distinction explicit.
| Approach | What it emphasizes | Key limitation |
|---|---|---|
| Decision-specific case deletion | Change in the prediction, probability, score, or selected action of interest | Depends on the chosen decision quantity and deletion unit |
| General regression diagnostics, such as leverage or Cook’s distance | Potential effect on fitted values or model coefficients | May not correspond to the actual decision target |
| Residual and data-quality review | Unexpected outcomes or possible measurement and recording problems | A surprising value is not necessarily influential or erroneous |
| Global sensitivity analysis | Effects of varying model inputs or assumptions over a defined range | Does not, by itself, measure dependence on one observed unit |
Case deletion is also limited as a robustness check. It changes one unit at a time, so it may miss influence that arises only when several units are removed or when assumptions change together. Results can depend on the chosen observational unit: deleting one measurement from a participant with repeated measures may not represent deleting that participant. With a small dataset, individual deletion estimates can also be unstable. Report the model, the unit deleted, the decision quantity, and the observed changes so readers can interpret the diagnostic in context.
Common Pitfall
Correction: A decision change establishes influence for the chosen model and decision quantity; it does not establish implausibility. Check the record and measurement process independently. If the data are valid, keep them in the primary analysis and communicate how the conclusion changes under deletion.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- A mean-based decision quantity is \(q(D)=8.0\) units for \(n=5\). One observation is 13 units. What is the mean after deleting it, and what is the deletion change?
- A fitted probability is 0.72, and the proceed threshold is 0.70. A deletion refit gives 0.68. What can be concluded, and what cannot be concluded?
- In a regression, an observation has a large residual but covariates similar to many other participants. Does that alone establish that it will strongly change the target-dose prediction?
- State one reason to delete a participant rather than only one record when conducting a case-deletion analysis.
Solutions. (1) The full-data sum is \(5(8.0)=40\). After deletion it is \(40-13=27\), so the mean is \(27/4=6.75\) units and the change is \(6.75-8.0=-1.25\) units. (2) The refit crosses the threshold, so this case is influential for the specified proceed decision. The result does not show that the observation is erroneous or that the decision should be changed by excluding it. (3) No. A large residual is not sufficient to establish strong influence; covariate leverage, the fitted model, and the prediction target also matter. Calculate the target-specific deletion refit. (4) If the participant contributes repeated measurements that are dependent within person, deleting the participant as a whole matches the independent observational unit more closely than deleting one record alone.