Prespecify Candidate Terms Before Examining Associations

Why Set the Plan Before Looking at Associations?

A regression model is not just a list of variables that happen to be available. Its terms should reflect the question being asked, the way relevant concepts were measured, and the intended use of the analysis. If investigators first inspect which variables appear most strongly associated with the outcome and then decide what to include, the eventual specification may reflect features of this particular sample rather than the original scientific or practical rationale.

Prespecifying candidate terms means deciding and recording which terms will be considered, and why, before examining their associations with the outcome. It does not mean pretending that no changes will ever be needed. Data problems, changed measurement definitions, or other documented constraints may require a revision. The important distinction is whether the choice was made from the question and information available in advance, or in response to observed outcome associations.

Definition: A prespecified candidate-term plan is a written record, made before examining associations between candidate predictors and the outcome, that identifies the terms to be considered in the analysis and the rationale for including, representing, or excluding them. It also records key decisions needed to interpret those terms, such as how a measure is defined, its role in the specification, and any data or timing constraints relevant to its use.

A candidate-term plan is not the same as a final fitted model. It sets out a defensible set of terms to consider; later analysis may involve comparing candidate specifications or making transparent changes for documented reasons. Nor does prespecification by itself make a model correct. The plan can still omit an important concept, use a poor measure, or fail to match the research question. Its value is that the choices can be evaluated against stated reasons rather than reconstructed after results are known.

What to Record in Advance

A useful plan gives enough detail that another analyst can tell what the intended terms meant and why they were considered. The earlier material on turning research questions into candidate terms, assigning term roles, and tracing predictors to measurement processes provides the groundwork. Prespecification makes those decisions explicit before outcome associations are inspected.

Question and purpose. State the research question and whether the specification is explanatory or predictive. These purposes can call for different candidate-term plans.
Outcome definition. Record the outcome and its measurement scale as fixed for the candidate comparisons. Do not quietly change its definition to favor one specification.
Candidate terms. Name each proposed term precisely. When a construct has several possible measures, state which measure is planned and whether another will be considered for a stated reason.
Rationale and role. For each term, explain its connection to the question, measurement process, or subject-matter account, and state its intended role in the specification.
Operational details. Record the units, coding, reference category where relevant, timing, and source of each measure. These details help prevent an ambiguous variable name from standing in for a precise definition.
Exclusions and constraints. Note plausible terms deliberately left out and why, such as a measure unavailable at the intended prediction time or a variable that does not represent the target concept.
Timing and changes. State when the plan was recorded and keep a dated account of later amendments, including the reason for each.

The level of detail should be sufficient to identify the intended candidate terms, not to prescribe every later diagnostic or comparison. In particular, a plan should not be rewritten after looking at outcome associations and then described as if it had been fixed in advance.

Association Checks and Other Data Checks

Prespecification concerns the timing of decisions relative to outcome associations. Looking at a predictor’s association with the outcome includes inspecting their scatterplot or group differences, calculating a correlation, fitting a preliminary regression involving that predictor, or reviewing an individual test statistic or p-value. If these results influence whether a term enters the plan, the plan was not set independently of those results.

Not every contact with the data is an outcome-association check. Confirming that a variable exists, verifying units against a data dictionary, finding impossible codes, or checking whether a measure is available for the intended analysis can be necessary data-quality work. Such checks do not establish which terms are associated with the outcome. Still, they can affect feasibility, so record what was checked and any resulting changes. Do not use a data-quality check as a label for an informal search through outcome relationships.

Key distinction: A candidate-term decision is prespecified when its rationale and intended form are recorded before the analyst examines how that candidate relates to the outcome. If an outcome-association result prompts the decision, record that the decision was made after examining the result; do not describe it as prespecified.

A Short Justification for Prespecification

Let \(Q\) denote the research question, \(M\) the measurement information, \(K\) the substantive rationale, and \(A\) the observed outcome-association results. A plan formed in advance can be represented as a decision based on \(Q\), \(M\), and \(K\), rather than on \(A\):

\text{Plan}=g(Q,M,K), \qquad \text{not } g(Q,M,K,A).

This distinction matters because observed associations depend partly on the particular sample. If the analyst uses those associations to decide what the original plan supposedly was, the explanation for including a term becomes entangled with the result it is meant to help analyze. Recording the plan first preserves a clear account of which decisions came from the question and prior rationale, and which—if any—were made after seeing the sample’s outcome relationships. This does not guarantee that the plan is scientifically adequate; it makes its basis and timing assessable.

Worked Example: Planning an Explanatory Analysis of Exam Scores

Hypothetical setting. Before examining outcome relationships, a university team plans an explanatory analysis using records from 240 first-year students. The outcome is each student’s final examination score, measured from 0 to 100 points. The research question concerns how study conditions and prior academic preparation relate to scores. The team has recorded each student’s prior GPA, weekly study hours, average nightly sleep during the term, paid-work hours per week, and commute time in minutes.

Step 1: Fix the question and outcome. The team writes that it will describe relationships with final examination score in points. It records the outcome definition and does not consider alternative score cutoffs after examining associations.

Step 2: Name candidate terms and rationale. Prior GPA represents academic preparation before the term. Study hours represent time devoted to the course. Sleep, paid-work hours, and commute time represent distinct conditions that could be relevant to students’ study opportunities or routines. The rationale is stated before the team looks at plots, correlations, or fitted coefficients.

Step 3: Record operational details and limits. The team records the units for each measure and the period they describe. It notes that the five measures are candidates to consider, not findings that any one is important. It also records that student ID is excluded because it identifies records rather than representing a general explanatory concept.

Step 4: State what was not inspected. The dated plan says that no candidate-versus-score plots, correlations, or individual tests were used to form the list. The completed plan is therefore an advance rationale for considering these terms, not a claim that they will have particular estimated effects. The association results, once examined, must be reported as results rather than as the reason the original terms were chosen.

Worked Plans for Different Uses

The same discipline applies whether a specification is explanatory or predictive, but the rationale must match its purpose. For a predictive analysis, the plan must be consistent with information available for the intended prediction. For an explanatory analysis, the plan should state how each term relates to the question. These are not interchangeable justifications.

Worked Example: A Candidate Plan for Readmission Prediction

Hypothetical setting. A hospital team plans to develop a prediction model for 30-day readmission using records from 500 discharges. Before inspecting any predictor’s association with readmission, the team defines the outcome as whether a patient has a readmission within 30 days of discharge. The planned prediction point is discharge.

Step 1: Apply the purpose and timing. Because the intended use is prediction at discharge, the team lists only candidate measures known by that time. It records age at discharge, diagnoses recorded during the admission, prior hospital visits in the preceding year, and discharge destination as candidates, with their sources and definitions.

Step 2: Record a constraint. A follow-up appointment completed 10 days after discharge is not included in this candidate plan: it would not be known at the stated prediction point. The reason is timing, not whether appointment completion is associated with readmission in this sample.

Step 3: Preserve the decision trail. The plan lists the four candidates, the prediction point, and the excluded post-discharge measure, and is dated before the team examines outcome associations. If the team later discovers that a candidate was not recorded reliably, it can document the data-quality problem and the revision. It should not replace the timing rationale with a new explanation based on observed readmission results.

Worked Example: Choosing and Recording a Measure Before Outcome Inspection

Hypothetical setting. A research group plans to study how neighborhood resources relate to a health outcome among 180 clinics’ patient catchment areas. Two available measures might represent neighborhood resources: the percentage of households within a defined distance of a public library, and the number of public libraries per 10,000 residents. Before examining either measure’s association with the outcome, the group must state how the candidate measure or measures will be handled.

Step 1: Connect the choice to the construct. The group’s question concerns residents’ practical access to a resource, so it selects the percentage of households within the prespecified distance as the primary candidate measure. It records the distance definition, geographic unit, and data source. This rationale concerns what the measure represents, not how strongly it relates to the health outcome in the observed records.

Step 2: Document the alternative and exclusion rationale. The group records the library count per 10,000 residents as an alternative measure to consider only if the access measure cannot be constructed consistently. It explains that the count captures resource availability but not household proximity in the same way. The plan does not say to choose whichever measure later has the larger estimated association.

Step 3: State the contingency before results are viewed. The plan specifies that inconsistent geographic boundaries would trigger review of whether the primary measure can be calculated comparably. If the measure proves infeasible, the team will document that constraint and the alternative used. This makes the decision rule and its reason visible while keeping the outcome relationship out of the initial measure choice.

When a Plan Changes

A prespecified plan is not a reason to ignore newly discovered errors or to use a measure that cannot be constructed as defined. The responsible response to a necessary change is to make the change visible. Preserve the original plan, record what was learned, state the revised decision and its rationale, and distinguish analyses based on the original plan from those prompted by later information.

This record is useful even when the change seems minor. For example, correcting a unit conversion because a data dictionary shows that a variable was stored in seconds rather than minutes is a data correction. Replacing a candidate because its observed association is weaker than expected is a result-driven choice. Both may alter the analysis, but they have different justifications and should not be presented as the same kind of decision.

Common Mistakes

Calling an outcome-informed list prespecified. A list assembled after reviewing correlations or plots was formed after examining associations. Record its actual timing.
Recording names without definitions. A term called “access” is ambiguous without its measure, units, population, and timing. State what was recorded and how it represents the intended concept.
Treating every available variable as a candidate. Availability alone is not a rationale. Connect each proposed term to the question or explain why it is included for the intended use.
Ignoring exclusions. Recording why plausible alternatives were not planned helps make the specification understandable and reduces the temptation to invent a rationale later.
Hiding revisions. A plan can change for sound reasons, but silently replacing the original record obscures whether decisions preceded or followed the outcome results.

Key takeaway: Before inspecting outcome associations, write down the candidate terms, their definitions and roles, the rationale for considering them, and the reasons for important exclusions. Date the plan, and document later changes with their actual reasons and timing.

Check Your Understanding

Try these questions using the distinction between a rationale made in advance and a decision prompted by observed outcome associations.

What makes a candidate-term plan prespecified, and how is it different from a final fitted model?
Name three operational details that can make a candidate term more precisely defined.
An analyst checks a predictor’s scatterplot against the outcome before deciding to include it. How should the timing of that decision be described?
Why might checking units or impossible codes be different from examining outcome associations? What should be recorded if such a check changes the plan?
A team chooses a measure because it matches the research question, then later finds it cannot be constructed consistently. What should the team document?