Hold the Outcome Definition Constant Across Candidate Models

Why the Outcome Must Stay Fixed

A regression model connects an outcome to one or more predictors. When several candidate models are under consideration, it can be tempting to compare their results as if they answer the same question. That comparison is meaningful only when the outcome has the same definition in every candidate model. Otherwise, a change in the results may reflect a change in the target being modeled, not a change in the predictors or model specification.

An outcome definition says precisely what is measured for each observational unit, including the measurement scale, the rule for coding or calculating the value, and the relevant time period. For example, “blood pressure” is incomplete as a definition: systolic or diastolic pressure? Measured when? In millimeters of mercury or another unit? A candidate-model comparison needs one specific answer to each relevant question.

Definition: Candidate regression models hold the outcome definition constant when they use the same outcome variable, measured on the same observational units using the same coding, measurement rule, and time window. Changing predictors while keeping that definition fixed changes the candidate specification; changing the outcome definition changes the target of the analysis.

For a basic linear regression model, write the response for observational unit \(i\) as

Y_i = \beta_0 + \beta_1 X_{i1} + \cdots + \beta_k X_{ik} + \varepsilon_i.

Here \(Y_i\) is the defined outcome, the \(X\) terms are predictors, and \(\varepsilon_i\) is the error. The usual linear-model conditions for standard inference include a correctly specified conditional mean, independent errors with mean zero and constant variance, and normally distributed errors when exact small-sample \(t\)- and \(F\)-inference is used. These conditions concern the model and its errors; they do not make two differently defined outcomes equivalent.

For a fair comparison of candidate specifications, also keep the analysis units and sample consistent when comparing fit summaries. If one model is fitted to a different set of units because of missing predictor values, a difference in its fit statistic may partly reflect the changed sample. Keeping the outcome fixed is essential, but it does not by itself guarantee that every other aspect of a comparison is fair.

Changing Units Is Not the Same as Changing the Outcome

A pure unit conversion multiplies all outcome values by the same positive constant. For instance, converting minutes to hours divides each value by 60. This preserves which observational units have larger outcomes and represents the same measured quantity. Coefficients and residuals change numerical scale, so they must be interpreted in the new units. Some fit summaries, such as \(R^2\), do not change under this conversion.

The scaling result follows directly from the least-squares criterion. Suppose \(Y_i^* = a + bY_i\), where \(a\) and \(b\) are constants and \(b \ne 0\). If a fitted model on \(Y\) has fitted values \(\hat{Y}_i\), then the corresponding transformed fitted values are \(a+b\hat{Y}_i\). Each residual is multiplied by \(b\), so the residual sum of squares is multiplied by \(b^2\). The total sum of squares is also multiplied by \(b^2\), leaving their ratio, and hence \(R^2\), unchanged:

\frac{\sum_i (Y_i^*-\hat{Y}_i^*)^2} {\sum_i (Y_i^*-\bar{Y}^*)^2} = \frac{b^2\sum_i (Y_i-\hat{Y}_i)^2} {b^2\sum_i (Y_i-\bar{Y})^2} = \frac{\sum_i (Y_i-\hat{Y}_i)^2} {\sum_i (Y_i-\bar{Y})^2}.

This is a property of a common linear rescaling of the outcome, not a reason to freely change outcome definitions across candidates. A threshold, a different follow-up window, or a composite rule generally changes which values are assigned to each unit. Those changes alter the target itself.

Worked Example: Converting an Outcome from Minutes to Hours

Suppose a hypothetical analyst models commute time in minutes using distance from home to work, in kilometers. Two candidate models use the same 30 commuters and the same outcome definition; they differ only in which predictors they include. For one fitted candidate, the estimated equation is

\widehat{\text{Commute time}}_{\text{minutes}} = 42 + 3.5(\text{distance in km}).

At a distance of 8 km, the predicted commute time is \(42 + 3.5(8) = 70\) minutes. Converting the response to hours divides both the intercept and slope by 60:

\widehat{\text{Commute time}}_{\text{hours}} = \frac{42}{60} + \frac{3.5}{60}(\text{distance in km}) = 0.700 + 0.05833(\text{distance in km}).

At 8 km, this gives \(0.700 + 0.05833(8) \approx 1.167\) hours, which is \(70\) minutes to the displayed precision. The slope now means an estimated increase of about \(0.05833\) hours, or \(3.5\) minutes, per additional kilometer. It is the same fitted relationship expressed in different units, not a new scientific outcome.

If the candidate’s residual sum of squares in minutes is 1,800 minutes squared, then in hours it is \(1,800/60^2 = 0.5000\) hours squared. Its \(R^2\) is unchanged. Raw residual sums of squares in minutes and hours cannot be compared as though they shared a scale. To compare candidate models, express their fit summaries on a common outcome scale and use the same commuters.

Thresholds Define Different Outcomes

Turning a quantitative measurement into an indicator by applying a cutoff is not a unit conversion. For example, “systolic pressure at least 140 mmHg” and “systolic pressure at least 130 mmHg” assign different outcome values to some people. Each binary outcome is a valid possibility, but the two address different questions. A model coefficient for one threshold is not automatically comparable in meaning to a coefficient for the other.

For a binary outcome coded 1 for an event and 0 otherwise, the sample mean is the observed event proportion. In a model with an intercept and a single group indicator, the least-squares coefficient for that indicator equals the difference in observed group proportions. This arithmetic describes the fitted association; by itself, it does not establish a causal effect or provide an inferential conclusion.

Worked Example: Changing a Blood-Pressure Cutoff

In a hypothetical sample, 20 people are in each of two groups. Consider two ways to define a binary outcome from a single blood-pressure measurement.

Group	At least 130 mmHg	At least 140 mmHg
A	10 of 20	8 of 20
B	15 of 20	10 of 20

For the cutoff of 130 mmHg, the observed proportions are \(10/20 = 0.50\) in group A and \(15/20 = 0.75\) in group B. The group difference, B minus A, is \(0.75 - 0.50 = 0.25\), or 25 percentage points. In a regression using the binary 130-mmHg outcome and a 0/1 indicator for group B, the fitted group coefficient is 0.25.

For the cutoff of 140 mmHg, the proportions are \(8/20 = 0.40\) and \(10/20 = 0.50\), giving \(0.50 - 0.40 = 0.10\), or 10 percentage points. The difference between the two results does not show that the same outcome model changed its estimate: the outcome rule changed. The first comparison concerns meeting the 130-mmHg threshold, while the second concerns meeting the 140-mmHg threshold. Neither calculation alone says anything about whether group membership caused a difference.

Composite Outcomes Change the Target

A composite outcome records whether any of several events occurred. For example, a study might define an event as “readmission or death within 30 days.” This can summarize a broader clinical question, but it is not interchangeable with readmission alone. The composite’s meaning depends on which events are included, how the time window is defined, and how an event is coded when more than one component occurs.

For a binary composite coded 1 if at least one component occurs, a person who experiences both components is counted once in the composite event total. Adding separate component counts without accounting for overlap would double-count such people. Even when the arithmetic is correct, the composite may be driven mainly by its more common component; reporting the component definitions helps readers understand what the outcome represents.

Worked Example: Readmission Alone Versus a Composite

Suppose two hypothetical groups each contain 100 people followed for 30 days. In group A, 12 people are readmitted, 3 die without being readmitted, and 2 experience both events. In group B, the corresponding counts are 18, 2, and 4.

For readmission alone, the group A event count is 12, including the 2 people who also died; its rate is \(12/100 = 0.12\). The group B rate is \(18/100 = 0.18\). The difference, B minus A, is \(0.18 - 0.12 = 0.06\), or 6 percentage points.

For the composite “readmission or death,” count the union of the events. In group A, this is \(12 + 3 = 15\) people: the 12 readmitted already include the 2 who also died. The composite rate is \(15/100 = 0.15\). In group B, it is \(18 + 2 = 20\) people, for a rate of \(20/100 = 0.20\). The composite difference is \(0.20 - 0.15 = 0.05\), or 5 percentage points.

These are results for two different outcomes, not two estimates of precisely the same target. A candidate model using readmission alone and one using the composite answer different questions, even though both use the same people, predictors, and 30-day period. The 2 and 4 people with both events are counted once each in the composite, not twice.

A Practical Check Before Comparing Candidates

Before comparing candidate specifications, write the outcome definition in a sentence precise enough that another analyst could assign the same value to each observational unit. Then check that every candidate uses that same definition and the same analysis units. If a proposed alternative changes the unit, threshold, event list, measurement time, or calculation rule, state that it is a different outcome and interpret it as a different target rather than as a like-for-like specification change.

State the outcome rule.
Name what is measured, the unit or coding, the observational unit, and any relevant time window.

Check the candidate models.
Confirm that every candidate response is generated by the same rule. Predictor changes do not justify changing the outcome.

Separate scale changes from definition changes.
A uniform unit conversion rescales coefficients and residual measures. A new cutoff or composite rule usually changes the target.

Make the comparison interpretable.
Use a common outcome scale and, where relevant, the same analysis units before comparing fit summaries or coefficients.

Common Mistake

A common mistake is to say, “The model with the smaller residual sum of squares is better,” when the models use different outcome definitions or scales. That conclusion is not justified: residual sums of squares depend on outcome units, and a changed threshold or composite changes what errors are being measured. First align the outcome definition and sample; if outcomes differ, explain that the models target different questions rather than treating their raw fit statistics as directly comparable.

Another mistake is to treat a threshold as merely a convenient recoding that leaves the target unchanged. A cutoff can discard distinctions among values and change which units count as events. It may be scientifically appropriate, but its choice should be defended as part of defining the outcome, not hidden inside an otherwise unchanged candidate comparison.

Check Your Understanding

Use the outcome-definition checks from this tutorial to answer the following questions.

A response is recorded in centimeters in one candidate model and meters in another. What changes in the coefficients and residual sum of squares, and what remains unchanged under this conversion?
Why do outcomes defined by systolic pressure at least 130 mmHg and at least 140 mmHg represent different targets?
In a composite outcome defined as “readmission or death,” why should a person experiencing both events count once in the composite total?
Two candidate models use the same response rule but different samples because one predictor is missing for several units. Why might their fit summaries not make a fair direct comparison?
Give one example of an outcome-definition change that is a unit conversion and one that changes the target being modeled.