Why Covariates Need a Substantive Reason
A regression program can add or remove variables according to a rule, but it cannot determine whether a variable represents a plausible cause of both an exposure and an outcome. That judgment requires information about what happened, when it happened, and what the recorded measurements mean. Selecting covariates on those grounds makes the adjustment strategy interpretable and helps prevent a convenient software rule from silently defining the analysis.
Suppose \(X\) is an exposure or predictor of interest, \(Y\) is the outcome, and \(C\) is a proposed covariate. The target association must be stated first: for example, the association between receiving a job-training program and later earnings. Whether \(C\) belongs in a candidate adjustment set depends on that target and on substantive knowledge about the variables, not merely on whether \(C\) is available in the data.
The earlier material on defining an association and on directed acyclic graphs provides a useful starting point. A candidate covariate is especially compelling when knowledge supports a pathway in which it precedes \(X\) and may influence both \(X\) and \(Y\). In a causal diagram this common-cause structure is \(X \leftarrow C \rightarrow Y\). Adjusting for \(C\) can block that backdoor path, provided the diagram and the measurements adequately represent the relevant processes.
For a numerical illustration, a linear regression might be written as
This equation describes a conditional mean that is linear and additive in the displayed variables. It is only an illustrative model form: including a covariate does not itself establish that the model is appropriate, that the covariate is measured well, or that the adjusted association has a causal interpretation.
Three Sources of Substantive Evidence
A practical covariate decision draws on three related questions. First, timing: was the proposed covariate measured before the exposure, and could its relevant value have existed before exposure occurred? A date recorded after exposure may still describe an earlier state, but that needs to be established rather than assumed. Conversely, a measurement taken after exposure may have been changed by the exposure and therefore cannot be treated automatically as a baseline covariate.
Second, consider the mechanism. Could differences in \(C\) help explain who receives or experiences \(X\), who has outcome \(Y\), or both? The rationale should describe a plausible process rather than just report that \(C\) correlates with \(X\) or \(Y\). A variable can be strongly associated with the outcome without being a source of confounding for the exposure-outcome association.
Third, examine measurement. What construct does the recorded variable represent? Is it measured at the right time, with a useful level of detail, and for the people in the analysis? A noisy or coarse measure may only partly represent the factor that motivates adjustment. A label in a data file is not automatically a meaningful covariate.
Name the exposure, outcome, population, and association or effect of interest. The same variable can play different roles for different questions.
Write down when exposure, outcome, and candidate covariates were measured, and whether the measurement refers to a pre-exposure condition.
For each candidate, explain how it could be related to exposure assignment or occurrence and to the outcome. Sketch the corresponding paths.
Confirm that the available variable represents the proposed factor at the needed time and with adequate detail. Record important limitations.
List the candidates with their substantive reasons before fitting the model. Treat automated selection output as a diagnostic or sensitivity aid, not as the reason for inclusion.
Worked Applications
Worked Example: Prior Education and Job Training
Hypothetical setting. An analyst wants to study the association between participation in a job-training program in 2022 (\(X\)) and annual earnings in 2023 (\(Y\)). Educational qualification was completed in 2019 and recorded from a certificate: 12 years for one applicant and 16 years for another. Program staff explain that applicants with more education were more likely to be referred to the program, and that education could also affect later earnings.
Timeline check. Education was completed three years before the program: \(2022-2019=3\) years. On the stated facts, the recorded qualification precedes \(X\), rather than being a response to participation in the program.
Mechanism and diagram. The referral process gives a reason for education to be related to \(X\), and the earnings process gives a reason for education to be related to \(Y\). The proposed common-cause path is \(X \leftarrow C \rightarrow Y\), where \(C\) is prior education. The candidate set for this path therefore includes education: \(\{C\}=\{\text{prior education}\}\).
Model and interpretation limits. A possible model is \(E(Y\mid X,C)=\beta_0+\beta_1X+\beta_2C\). Here \(\beta_1\) describes the difference in modelled mean earnings associated with a one-unit change in the program indicator, comparing applicants with the same recorded education, under this model. The example justifies considering education; it does not establish that education is the only relevant common cause or that the model identifies a causal effect.
Worked Example: Baseline Severity and a Treatment Decision
Hypothetical setting. A clinic records a symptom-severity score from 0 to 10 before a treatment decision, then records the score again four weeks later. The question is whether treatment (\(X\)) is associated with the four-week score (\(Y\)). Clinicians say that people with more severe symptoms are more likely to receive treatment and tend to have higher later scores.
Measurement and timing check. The baseline score is recorded before the treatment decision. For three hypothetical patients, baseline scores of 6, 7, and 8 have mean
This arithmetic summarizes those three scores; it does not show that the score is a perfect measure of severity. The analyst should check how the scale was administered, whether the same scoring procedure was used for everyone, and whether the score captures the severity relevant to treatment decisions.
Candidate decision. The stated treatment process links baseline severity to \(X\), while the clinical account links it to \(Y\). Baseline severity is therefore a substantively justified candidate covariate. The four-week score is the outcome, not a baseline measure. A possible model uses \(C=\) baseline severity, but the fact that the post-decision score is available does not make it an appropriate substitute for the baseline measure.
The proposed adjustment set for the described common-cause pathway is \(\{\text{baseline severity}\}\). If severity is measured coarsely or inconsistently, adjustment may not fully account for the differences clinicians described; the measurement limitation should be reported rather than hidden by treating the recorded score as exact.
Worked Example: Why a Software Rule Does Not Decide
Hypothetical setting. A company is evaluating whether a mentoring offer (\(X\)) is associated with employee retention one year later (\(Y\)). A software routine retains only variables with a screening \(p\)-value below \(0.05\). Age is recorded at the time employees become eligible. Managers explain that age influenced which employees were offered mentoring, and that age may also be related to staying with the company. In a pilot model, age has a screening \(p\)-value of \(0.21\).
Apply the substantive check. Age is measured at eligibility, before the mentoring offer. The staffing explanation supplies a possible link from age to \(X\), while the retention rationale supplies a possible link from age to \(Y\). These mechanisms support considering age in the adjustment set, even though \(0.21\) is greater than the routine’s \(0.05\) cutoff.
The software rule answers a different question: whether age meets a chosen statistical screening threshold in that pilot analysis. It does not test whether age is a plausible common cause, and failure to meet the threshold is not evidence that the substantive mechanisms do not exist. A candidate model could therefore include age, with the proposed path written \(X \leftarrow \text{age} \rightarrow Y\). The analyst should still assess the model and acknowledge that other causes of mentoring offers or retention may be unmeasured.
The selection decision is not “include every variable that has a small \(p\)-value.” Nor is it “include only variables that pass a threshold.” It is “include variables supported by the target question and substantive account, then assess the consequences of reasonable alternatives.”
A Short Justification for the Common-Cause Criterion
The reason a common cause is a candidate for adjustment can be seen directly from the path \(X \leftarrow C \rightarrow Y\). Without conditioning on \(C\), people with different values of \(X\) may also differ in \(C\). If \(C\) is related to \(Y\), that difference can contribute to the observed \(X\)-\(Y\) association even when comparing exposure groups is the intended question.
In a diagram, conditioning on \(C\) blocks this particular fork: once comparisons are made within levels of \(C\), the path through \(C\) no longer connects \(X\) to \(Y\). This is the justification for proposing \(C\) in the adjustment set for that path. It depends on the diagram adequately representing the mechanism and on \(C\) being measured usefully. Blocking one represented path does not prove that all other non-comparabilities have been addressed.
Documenting the Decision and Avoiding Pitfalls
A useful covariate record is brief but specific: name the variable, give its measurement time, state the proposed mechanism, and describe any measurement concern. For example: “Adjust for qualification completed before program eligibility because it may influence referral and later earnings; qualification is recorded from certificates, but other aspects of educational preparation are unavailable.” This is more informative than “education was significant” or “the model selected education.”
Timing is an important screen, not a complete decision rule. A variable measured before exposure is not automatically a confounder, and an available variable should not be included merely because it predicts the outcome. Likewise, a measurement taken after exposure requires careful scrutiny: it may represent a changed condition rather than a pre-existing difference. Deciding whether such a variable has a different causal role is a separate question that should be answered before adjustment.
A common mistake is to let automated selection define the adjustment set: “Age did not pass the \(p\)-value cutoff, so it cannot confound the association.” The correction is to separate the substantive question from the screening rule. Use timing, mechanisms, and measurement evidence to propose covariates; then use statistical analyses to assess the consequences of the proposal, not to replace its rationale.
Check Your Understanding
For each question, give a reasoned answer in context.
- A variable was measured before the exposure. Does that fact alone justify adjustment? Explain what additional evidence would help.
- In the job-training example, state the proposed common-cause path and explain why prior education was considered for the candidate set.
- Why might a baseline symptom score be more useful than a score recorded after a treatment decision when assessing baseline differences?
- A software routine drops a plausible common cause because its screening \(p\)-value exceeds \(0.05\). What does that result establish, and what does it not establish?
- Name two measurement questions to ask before treating a recorded covariate as an adequate representation of a proposed common cause.