Test-Set Reuse and Adaptive Overfitting

Why a Test Set Can Stop Being a Final Check

A test set is useful because it provides a check on predictions using observations that did not guide the model-fitting decisions being evaluated. But a test set is not automatically protected from those decisions. If analysts repeatedly inspect its scores and use what they learn to choose predictors, revise the model, or select among alternatives, the test set has become part of the development process.

This can make the final score look better than the model’s performance on new observations from the same target population. The problem is not limited to directly fitting model parameters to test outcomes. Human decisions based on a score—such as keeping a change because the score improved—can also adapt the analysis to the particular test observations.

Let \(S\) denote a score calculated on a test set using a fixed scoring rule, with lower values representing better performance. The score is an estimate of how the procedure performs on new observations from the target population. It can vary from one test sample to another. If a model or decision is selected partly because it happened to receive a favorable score on one finite test set, that favorable score may reflect both genuine performance and chance variation in that sample.

Choosing Among Noisy Scores

A simple calculation shows why inspecting multiple scores can create optimism. Suppose several candidate models have the same true population score, denoted by \(\theta\). Write each observed test score as its true score plus an error:

Here \(\varepsilon_j\) represents the difference between candidate \(j\)’s test-set score and its true score. For this justification, assume the candidates have the same \(\theta\), each error has expected value zero, and the scores use the same evaluation target and scoring rule. A negative error is favorable because lower scores are better. The errors need not be independent; using the same test set often makes them related.

If the analyst reports the lowest of \(m\) observed scores, then the selected error is the minimum of the errors. For every set of values, the minimum cannot exceed their average. Therefore:

Taking expected values preserves this inequality. Since each error has expected value zero, the expected average error is zero as well. Thus the expected selected score is no greater than the common true score:

If the errors differ with positive probability, the inequality is strict: sometimes at least one score has a more favorable error than the others, so the minimum is below their average. This is a selection effect, not evidence that the scoring rule is incorrect. When candidates have different true scores, choosing among them may identify a genuinely better model; nevertheless, the selected test score can still be too favorable because selection also favors chance improvements.

The calculation explains one form of adaptive overfitting, but actual reuse can be more involved. An analyst may make one choice, inspect the score, make another choice, and inspect it again. The later candidates are then shaped by earlier feedback. Even if each revision seems small, the sequence can gradually use information about peculiarities of the test set.

How Repeated Inspection Adapts a Model

Adaptive overfitting can happen without an automated search over hundreds of candidates. Suppose a model is evaluated, a feature is added, the score is checked, and another change is tried only if the score improves. The test outcomes have now influenced the path of revisions. The final model is not simply a fixed procedure evaluated on an untouched sample; it is the result of a sequence of decisions informed by that sample.

The key distinction is between using the test set to calculate a score and using that score to decide what to do. A score can be calculated once after the development choices are complete. If the result prompts additional modeling choices, that first evaluation has become feedback, and a later score on the same observations is no longer a fresh final check.

When a Fresh Evaluation Dataset Is Needed

A practical rule is: if test-set results have influenced any choice that affects the model or the reported procedure, do not present a later score on that same test set as an independent final evaluation. The appropriate next step depends on the goal. If an honest final performance assessment is needed, evaluate the now-fixed procedure on new observations that were not used to make those choices. The new evaluation data should not guide further revisions if its score is to remain a final check.

For instance, an analyst can use development data and resampling methods to compare or tune alternatives, then reserve a separate test set for a final evaluation. The essential feature is not the label attached to a data split; it is whether the final evaluation observations and their outcomes stayed out of the decisions being assessed. A dataset called “test” does not remain independent merely because it keeps that name.

If a test set has already been inspected and used repeatedly, there may be no untouched data available. In that case, be transparent: the score describes performance on data that also informed model development, and it may be optimistic. Do not treat relabeling the same observations, or fitting the chosen model again without changing them, as creating a fresh evaluation. A new sample from the intended target population, collected or set aside without guiding the development decisions, can provide a fresh check.

Common Mistake and Practical Reporting

A common mistake is to reason: “The test set was never used to estimate the model coefficients, so it is still independent.” This is not sufficient. If its scores led the analyst to add or remove predictors, choose a model, or keep a particular revision, test outcomes influenced the final procedure even if no equation was fitted directly to them.

The correction is to track all decisions informed by the test set, including human decisions. Use development data for comparisons and revisions, then evaluate the completed procedure once on data that did not guide those choices. If results from the supposed test set have already guided choices, describe that limitation and seek a fresh evaluation before making a new-data performance claim.

Repeated cross-validation or another resampling approach can be useful during development, but the same principle applies: if results guide model choices, they are part of development. Methods designed to account for selection, such as nested cross-validation, address different evaluation designs; they do not make repeated inspection of a particular test set harmless. The main practical question remains whether the data used for a final claim were kept independent of the decisions being evaluated.