Why Predictor Choice Starts With the Question
A regression question identifies an outcome, but the outcome alone does not tell us which other variables to include. The same outcome—such as exam score—could be studied in relation to one particular feature, used to describe several patterns, or predicted from information available before an exam. Each purpose suggests a different choice of predictors.
A predictor is a variable used alongside an outcome to describe a pattern, examine an association, or generate a prediction. In a regression question, the outcome is the quantity of interest; predictors are the variables considered in relation to that outcome or used as inputs. Choosing predictors is therefore not just a matter of listing every recorded variable. It is a matter of deciding which variables serve the stated question.
The earlier material on classifying description, explanation, and prediction questions distinguishes three broad purposes. A descriptive question asks about patterns in the observed data. An explanatory question asks about an association of interest. A prediction question uses available information to estimate an outcome. Those purposes overlap in some applications, but they need not use identical predictor sets.
A Practical Rule for Choosing Predictors
Begin with the outcome and the exact wording of the question. Identify the variable or variables the question is about, then consider whether other variables are needed to describe the requested pattern, address the requested association, or provide useful information for a prediction. Finally, check the scope: a variable cannot serve the intended role if it is not recorded for the analysis or, for a prediction, not available when the prediction must be made.
This can be expressed as a bookkeeping rule. Suppose there are \(m\) candidate variables, and let \(R_j=1\) when candidate \(j\) fits the question’s purpose and scope, and \(R_j=0\) otherwise. Then the number \(k\) of selected predictors is
This sum is only a count of the variables selected after considering the question. It is not a statistical score, a test of whether a variable is important, or a guarantee that the chosen set is best. The real work is deciding, in context, which candidates fit the purpose and scope. A short predictor set can be right for a focused question; a broader set can be right for a broader question.
It is also useful to distinguish a focal predictor from other possible predictors. The focal predictor is the variable whose pattern or association the question emphasizes. Other variables may be included if the question calls for them, but they do not automatically replace the focal variable or become relevant merely because they are available.
Worked Example: Describing a Focused Pattern
Hypothetical data: A teaching team records the final exam score, study hours, course section, and year of enrollment for four students. The question is, “How do final exam scores vary with study hours among these students?” The recorded values are shown below.
| Student | Study hours | Final exam score | Section | Year |
|---|---|---|---|---|
| A | 4 | 68 | Morning | 1 |
| B | 6 | 74 | Afternoon | 2 |
| C | 8 | 81 | Morning | 1 |
| D | 10 | 85 | Afternoon | 3 |
The outcome is final exam score, measured in points. The question specifically asks how that score varies with study hours, so study hours is the focal predictor. Section and year are recorded, but the question does not ask whether the pattern differs by section or year.
For an explicit selection check, let \(R_j=1\) for a candidate that fits this question’s stated focus and \(R_j=0\) otherwise. In the candidate order study hours, section, year, the choices are \(R_1=1\), \(R_2=0\), and \(R_3=0\). Thus
The predictor set for this focused question contains study hours alone. This does not establish that section or year could never matter; it says only that they are outside the question as worded. If the question instead asked whether the study-hours pattern differed by section, section would be relevant to that broader question.
Predictors for an Association Question
For an explanatory question, the predictor set should reflect the association the question asks about. At minimum, identify the focal predictor and outcome clearly. Additional variables belong when the question asks about a broader pattern—for example, whether the association varies across a stated group—or when the planned analysis has a specific reason to include them.
There is no general rule that every available variable should be added. Including variables without a question-driven reason can make the analysis harder to interpret, while omitting a variable that defines the intended comparison can fail to answer the question. In particular, calling a variable a predictor does not establish a causal effect. Association and causal interpretation are different claims; the predictor label alone cannot justify the latter.
Worked Example: Choosing Predictors for an Association
Hypothetical data: A clinic records waiting time in minutes, staffing ratio (staff members per 10 patients), and whether each clinic-day is a weekday or weekend. A researcher asks, “How is staffing ratio associated with waiting time, and does the pattern differ between weekdays and weekends?” The records are:
| Clinic-day | Staff per 10 patients | Waiting time (minutes) | Day type |
|---|---|---|---|
| 1 | 2 | 38 | Weekday |
| 2 | 3 | 30 | Weekday |
| 3 | 2 | 45 | Weekend |
| 4 | 3 | 36 | Weekend |
The outcome is waiting time in minutes. Staffing ratio is the focal predictor because the question asks about its association with waiting time. Day type is also relevant because the question explicitly asks whether the pattern differs by day type. The chosen predictor set is therefore staffing ratio and day type.
Using the candidate order staffing ratio, day type, the indicators are \(R_1=1\) and \(R_2=1\). The count is
This selection identifies what the question asks to examine; it does not show that staffing causes shorter waits. The four records provide a small hypothetical illustration of variable choice, not a basis for a broad conclusion. If the question had asked only about the staffing-ratio association, day type would not be required by that wording, though another question might justify including it.
Predictors for a Prediction Question
For prediction, a candidate variable must be useful for the intended estimate and available at the time the estimate is needed. This timing check can change the predictor set. A measurement made only after the outcome occurs cannot serve as an input to a prediction that must be made beforehand, even if it is strongly related to the outcome in a completed data set.
“Useful” here is a question about the intended task, not a claim that any particular variable will improve accuracy. That claim would require evidence from an appropriate evaluation. At the planning stage, the researcher can specify the outcome to estimate, when the estimate is needed, and which information could be available then. Variables that do not meet those requirements should not be treated as eligible inputs for that prediction.
Worked Example: Selecting Inputs for a Next-Day Prediction
Hypothetical setting: A library wants to estimate the number of visitors on the next day. At the end of today, it has the next day’s calendar category, a weather forecast for the next day, and today’s visitor count. It also has the actual next-day weather, but that measurement is recorded only after the next day has passed. The question is, “Which recorded variables could be considered as inputs for estimating next-day visitor count at the end of today?”
The outcome is next-day visitor count. The calendar category, weather forecast, and today’s count are all available at the stated estimation time, so they are candidates for this prediction task. Actual next-day weather is not available at that time and therefore cannot be used as an input for an estimate made at the end of today.
In the candidate order calendar category, next-day weather forecast, today’s count, and actual next-day weather, the eligibility indicators are \(R_1=1\), \(R_2=1\), \(R_3=1\), and \(R_4=0\). The count of eligible candidate inputs is
The three eligible variables could be considered for the prediction question. This does not prove that all three improve predictions; it establishes that they fit the stated timing constraint. The actual next-day weather might be related to visitor count, but its later availability makes it unsuitable for an estimate that must be produced today.
A Short Justification: Why the Question Must Come First
The selection rule follows from the meaning of the task. A regression question specifies an outcome and a purpose. A candidate variable belongs only if it helps address that purpose within the stated scope. For prediction, it must also be available at the relevant time. Thus a candidate that is irrelevant to the question, outside its scope, or unavailable for the intended use fails at least one requirement. Counting the remaining candidates is useful for recording a decision, but the count does not make that decision statistically correct by itself.
This reasoning also explains why a single data set can support different predictor choices without contradiction. A variable can be outside the scope of one question and central to another. The choice is not an eternal label attached to a variable; it is a decision made for a particular outcome, purpose, and intended use.
Common Mistakes and Why the Choice Matters
- Including every recorded variable. Availability alone does not make a variable relevant. Start from the question, then include candidates that fit its purpose and scope.
- Leaving out the focal predictor. If a question asks how an outcome varies with a particular feature, that feature must be identified as a predictor rather than replaced by a list of unrelated variables.
- Treating a predictor as a cause. “Predictor” describes a role in an analysis. It does not establish that changing the predictor would change the outcome.
- Using information unavailable at prediction time. A variable measured only after the outcome cannot be an input to a prediction that must be made before that measurement exists.
- Assuming more predictors are always better. A larger set may be appropriate for a broader question, but its size alone does not guarantee a clearer answer or better prediction.
Careful predictor choice makes a regression question interpretable. It clarifies which relationship is being described, which association is being examined, or what information is being used for a prediction. It also helps prevent a common mismatch: fitting a model to whatever variables happen to be on hand, then treating its output as an answer to a question the variable choices never represented.
Check Your Understanding
For each situation, identify the outcome and explain which candidate variables fit the stated purpose and scope.
- A question asks how weekly exercise hours vary with age. A data set also records preferred music genre. Which variable is the focal predictor, and is music genre required by the question as written?
- A question asks whether the association between class size and average quiz score differs between online and in-person classes. Which variable is the focal predictor, and why is class format relevant?
- A school wants to estimate tomorrow’s absences using information available this afternoon. It has the calendar and today’s attendance, as well as tomorrow’s final absence count after the day is over. Which of these can be considered as inputs at the stated time?
- Explain why including a variable as a predictor does not by itself show that the variable causes the outcome.
- A researcher has ten recorded candidate variables but asks a narrowly focused question about one of them. Explain why the number of available variables does not determine the predictor set.