Leave-One-Donor-Out Sensitivity Analysis

Why Recompute an Estimate Without Each Donor?

A population-level result can look precise while depending heavily on one donor. This can happen when donor responses differ, when a donor has an unusual estimate, or when an analysis gives some donors more influence than others. A direct way to examine that dependence is to repeat the analysis after excluding each donor in turn.

This procedure is a sensitivity analysis: it asks how much the estimated result changes under a specified set of omissions. It does not identify a donor as erroneous, prove that a conclusion generalizes, or replace the primary analysis. Its value is diagnostic. It makes dependence on individual observed donors visible and helps readers judge whether the substantive conclusion is stable to their inclusion.

The phrase “repeating the analysis” matters. For a summary defined as the equally weighted mean of donor-level estimates, recalculate that mean over the remaining donors. For a fitted model, remove the donor’s observations and refit the same model with the same outcome, covariates, contrast, and estimation procedure. Simply deleting a donor’s displayed estimate from a table is not enough if the original estimate came from a model whose parameters need refitting.

Define the Target and the Recalculation

Before calculating anything, specify the target estimate. If \(\hat{\theta}_i\) is the estimate for donor \(i\), one common target is the equally weighted mean across donors:

This target gives each donor equal weight, regardless of how many cells were sampled from that donor. It is appropriate when the scientific summary is the average donor-level response. Other analyses may use a different estimator, such as a regression coefficient or a model-based contrast. In those cases, the leave-one-donor-out quantity must be defined by refitting that same estimator after omission. The resulting estimates need not equal an average of the individual donor estimates.

A useful comparison is the change from the full-data estimate:

The sign of \(\Delta_i\) indicates the direction of movement after donor \(i\) is removed; its magnitude gives the change in the estimate’s units. For an estimate of a log fold change, for example, \(\Delta_i\) is also in log-fold-change units. A percentage change can sometimes help describe scale, but is unhelpful when the full estimate is near zero or changes sign. Report the estimates and their units rather than relying on a percentage alone.

A Useful Identity for an Equally Weighted Mean

For the equally weighted mean, the relationship between each donor estimate and its leave-one-out change can be derived directly. Since the sum of all donor estimates is \(D\hat{\theta}\), omitting donor \(i\) gives:

Subtract the full-data mean and simplify:

Thus, for this estimator, omitting a donor whose estimate exceeds the overall mean moves the mean downward, and omitting a donor whose estimate is below the mean moves it upward. The size of the shift grows with the donor’s distance from the overall mean and shrinks as the number of donors increases. This identity is a calculation check, not a general formula for a mixed-model coefficient or another refitted estimator.

Reading the Pattern, Not Just the Largest Change

A useful review considers the full set of recalculated estimates rather than focusing only on the most extreme donor. Ask whether the scientific interpretation is preserved across omissions: does the estimate keep its direction, remain similar in magnitude, or cross a scientifically important threshold? For a fitted model, also check whether standard errors, intervals, or other reported uncertainty summaries change materially. A stable point estimate does not guarantee stable uncertainty, and a stable sign does not by itself establish a robust biological conclusion.

Interpretation depends on the pattern. If one omission produces a large shift and the other omissions have little effect, the result may be sensitive to that particular donor. If many omissions produce substantial changes, the conclusion may be broadly dependent on the small donor set. If the estimates remain similar, that is evidence of stability to these particular single-donor omissions, not proof of stability to every plausible data or modeling change.

This analysis complements, rather than duplicates, influential-donor diagnostics. An influence measure may summarize how unusual a donor’s contribution is under a particular analysis; leave-one-donor-out estimates show the result of an explicit refit or recalculation. In a donor-level analysis, both perspectives can be informative. Neither licenses removal merely because the result becomes more convenient without a donor.

Assumptions and Practical Diagnostics

Leave-one-donor-out sensitivity analysis is a recalculation, not a test with a universal null distribution. Its interpretation nonetheless relies on a clearly specified and coherent analysis:

• Donor is the omission unit. All observations from a donor are excluded together. Treating cells from the same donor as separate omissions would not assess sensitivity to biological replication.
• The estimate has the same meaning after each omission. Use the same outcome definition, condition contrast, covariates, and estimator. A change in the estimand makes the comparison difficult to interpret.
• The remaining data support the recalculation. Check that each refit converges, relevant groups or conditions remain represented, and the estimate is not undefined. If a model fails after an omission, report that failure rather than quietly replacing the procedure.
• Donor estimates are comparable when averaged. An equally weighted mean requires a meaningful donor-level estimate on a common scale for every included donor. If precision weighting or a fitted model defines the primary estimator, repeat that estimator rather than switching to an unweighted mean for the sensitivity analysis.

A compact diagnostic table should identify the omitted donor, the recalculated estimate, the change from the full estimate, and—when relevant—the corresponding uncertainty summary or model-fit status. Examine the range and direction of changes. Also inspect whether influence aligns with donor-level characteristics or data-quality concerns, while avoiding post hoc exclusions based only on the desired result.

Limitations and Common Mistakes

Each leave-one-out analysis still uses almost the same data as the full analysis, so the recalculated estimates are not independent replications. They also cannot reveal the effects of donors who were never observed. With few donors, a single omission may leave a very small analysis set, making estimates unstable or a model impossible to fit. These limitations are reasons to report the donor count and all recalculated results, not reasons to suppress them.

A common mistake is to interpret a large leave-one-out shift as proof that the omitted donor is bad data. The reasoning is wrong because influence describes how the estimate responds to an observation; it does not diagnose the observation’s validity. The correction is to check data quality and the scientific context, report the sensitivity, and retain the donor unless there is a defensible, documented reason to exclude it.

Another mistake is to call a conclusion “robust” merely because every recalculated estimate has the same sign. Sign stability is useful, but the magnitude or uncertainty may still change substantially. Conversely, a small change in the point estimate does not ensure stable uncertainty. State separately what happened to the estimate, its uncertainty, and the substantive interpretation.