Confusion Matrices for Dose-Target Decisions

Why a Dose-Target Confusion Matrix Needs a Precise Question

A dose recommendation may be judged by whether it predicts that a patient will reach a therapeutic target. A confusion matrix summarizes, for the same patients, the predicted classification and the classification based on the observed outcome. Its simplicity is useful, but the table is meaningful only after the target and assessment time have been specified. Changing either can change which patients are counted as attaining the target.

The matrix is a descriptive summary of paired classifications. It does not by itself show how serious a mismatch is, whether the recommendation improves outcomes, or whether a classification rule is clinically appropriate. Those questions require the decision context and consequences of errors, not just the table’s counts.

Define the Two Classifications Before Counting

Suppose the endpoint is a steady-state trough concentration, measured in mg/L, and the target range is 8–12 mg/L, inclusive. A value is classified as attained if it is at least 8 and at most 12; values below 8 or above 12 are classified as not attained. The word “inclusive” matters: exactly 8 and exactly 12 count as attained under this definition.

The predicted classification must also be operationally defined. For a point prediction, apply the same target rule to the predicted endpoint. If a model instead supplies a probability of attainment, a separate probability cutoff is needed to turn that probability into a binary prediction. Do not treat a probability and a binary classification as interchangeable.

Use \(n_{AA}\) for predicted attained and observed attained, \(n_{AN}\) for predicted attained and observed not attained, \(n_{NA}\) for predicted not attained and observed attained, and \(n_{NN}\) for predicted not attained and observed not attained. The first letter always refers to the prediction and the second to the observation. State this convention alongside any table; otherwise row and column labels can be misread.

Here \(N\) is the number of patients with both a usable prediction and a corresponding observed outcome. Overall agreement is the fraction whose predicted and observed labels match:

Agreement is not a measure of the clinical importance of the errors. It gives equal weight to the two off-diagonal types of disagreement, even if underattainment and overattainment have different consequences. It also depends on how often each category occurs in the evaluated group.

Why the Target Rule Changes the Table

A target is part of the estimand being evaluated, not a cosmetic label added after predictions are made. A range, a lower-bound threshold, and a target with a different boundary convention define different binary outcomes. Consequently, the same concentration values can produce different cells and different agreement. The target should be chosen for clinical reasons and reported in enough detail to reproduce the classification.

Probability-Based Predictions Need a Cutoff

Some dosing models provide \(P(\text{attainment})\) rather than a single predicted concentration. A confusion matrix still requires a binary predicted label. One transparent rule is to classify as predicted attained when the model probability meets or exceeds a prespecified cutoff, and as predicted not attained otherwise. The cutoff should be stated; it is not supplied automatically by the probability itself.

Align Measurement Timing and the Evaluated Endpoint

A prediction and an observation must refer to the same endpoint at a comparable time. A steady-state prediction should not be compared as if it were a day-7 prediction, and a trough concentration should not be paired with a peak concentration under the same target label. When time or endpoint differs, a disagreement may reflect that mismatch rather than prediction performance for the intended question.

Construction Checks, Alternatives, and Limitations

A reliable construction follows a short sequence: lock the endpoint, target bounds, boundary convention, assessment time, and prediction rule; apply those rules to every eligible patient; then count each paired label exactly once. Check that all four cells are nonnegative integers and that their sum equals the number of complete prediction–observation pairs. If records are missing, state how many were excluded and why; do not silently use different denominators for different parts of the table.

The matrix is complementary to, not a replacement for, other evaluations. Continuous prediction-error summaries retain the magnitude of concentration differences that binary labels discard. Utility or loss analyses can distinguish clinically consequential mismatches from relatively minor ones. The confusion matrix instead makes the direction and frequency of categorical agreement or disagreement visible. No one summary answers all three questions.

A frequent mistake is to read a cell from only one margin. For example, under the inclusive 8–12 mg/L target, patient E has a predicted value of 8 mg/L and an observed value of 7 mg/L. The wrong reasoning is, “The observed value is below target, so E is not attained in both classifications.” The correction is to classify prediction and observation separately: predicted attained, observed not attained, so E contributes to \(n_{AN}\). Boundary checks should be performed on each side of the table.

Other warning signs include totals that do not equal \(N\), reversed row and column conventions, an unreported probability cutoff, or a target chosen after inspecting which rule yields the most favourable agreement. Agreement can also be high when one category is very common, so it should be read alongside the cell counts and the clinical question. A small table is a description of the evaluated records; it does not establish transportability to other patients or settings.