Worked Workshop on Defining Regression Questions

Turning a Scenario Into a Regression Question

A regression question is more than a request to “see whether two variables are related.” A well-defined question specifies what is being described or predicted, which variables are involved, whose outcomes matter, and what quantity would count as an answer. Those choices also determine what conclusions the data can support.

This workshop applies the distinctions between descriptive, explanatory, predictive, and causal questions to several hypothetical scenarios. In each one, the key is to write down the question before choosing an analysis. A carefully stated question can prevent a common mistake: reporting a result that is numerically correct but answers a different question from the one of interest.

Definition: The outcome is the response or quantity being described, predicted, or explained. A predictor is a variable used to describe or predict variation in the outcome; calling a variable a predictor does not, by itself, mean it causes the outcome. The target population is the group of units the question concerns. The quantity of interest is the specific feature of that population or its outcomes the question seeks to learn about, such as a conditional mean, an association, a future value, or a causal effect.

A Short Specification Before Analysis

Start by deciding what kind of answer the question requests. A descriptive question asks how an outcome varies across values of a predictor in a stated population. An explanatory question asks about an association between variables, without necessarily claiming that one causes another. A predictive question asks for an outcome value for a future or otherwise unobserved case. A causal question asks what would change under an intervention or treatment assignment compared with an alternative.

The same outcome and predictor can appear in different question types. For example, study time and exam score could be used to describe score differences, estimate an association, predict a future score, or investigate a causal effect of an assigned study program. Those are different questions and require different interpretations.

For a quantitative outcome \(Y\) and predictor \(X\), a descriptive or associational target might be the conditional mean \(E(Y\mid X=x)\): the mean outcome among units with predictor value \(x\), in the population being considered. A contrast between two predictor values is another possible quantity of interest:

E(Y\mid X=x_2)-E(Y\mid X=x_1)

This contrast describes a difference in conditional means. It does not become a causal effect merely because it is written as a subtraction. Whether it supports a population conclusion depends on how the data were collected, and whether it supports a causal conclusion depends on the study design and assumptions.

Worked Questions

Worked Example: Describing Commute Time by Distance

Hypothetical scenario. A university surveys 120 students enrolled on its main campus about their one-way commute time and distance from campus. The research team wants to describe how commute time differs with distance among main-campus students.

Question type. This is descriptive: the aim is to summarize how commute times vary across distances, not to claim that distance alone causes the difference.

Outcome and predictor. The outcome \(Y\) is one-way commute time in minutes. The predictor \(X\) is distance from campus in miles. A particular summary could compare students living 2 miles away with those living 6 miles away.

Target population. The stated target is students enrolled on the main campus during the period of interest. The surveyed students are the observed sample. Extending a sample summary to all such students requires a sampling process that gives a reasonable basis for representing that population; surveying 120 students does not, on its own, guarantee that condition.

Quantity of interest and what the data support. Suppose, for illustration, that the sample mean commute time is 22 minutes for students living 2 miles away and 38 minutes for those living 6 miles away. The sample contrast is

38-22=16\text{ minutes}

In this hypothetical sample, the mean commute time in the 6-mile group is 16 minutes longer than in the 2-mile group. That directly describes the two observed groups. If the sample represents the target population, the contrast may also inform the corresponding population conditional-mean difference. It does not establish that moving a student farther from campus would increase that student’s commute by 16 minutes.

Worked Example: Associating Age With Annual Health-Care Visits

Hypothetical scenario. Investigators collect survey data from 240 adults living in a county. Each person reports age and the number of health-care visits during the previous year. The question is whether visit frequency is associated with age in the county’s adult population.

Question type. This is an explanatory or associational question. “Explanatory” here refers to studying how the outcome varies with a predictor; it does not mean the data prove that age causes a particular number of visits.

Outcome and predictor. The outcome is the number of health-care visits per adult in the previous year, measured in visits per year. Age in years is the predictor. The outcome is recorded for each adult; the predictor is used to compare or summarize outcome patterns across ages.

Target population. The target is adults living in the county during the period represented by the survey. Whether the 240 respondents support conclusions about all county adults depends on recruitment, participation, and how well the respondents represent that group.

Quantity of interest and what the data support. Suppose the sample mean is 2.1 visits per year among adults aged 20–39 and 3.0 among adults aged 60–79. The observed difference is

3.0-2.1=0.9\text{ visits per year}

The older age group in this sample averages 0.9 more visits per year than the younger age group. The sample supports this descriptive statement about its respondents. With suitable sampling and measurement, it may support an association statement about county adults. Other factors, such as health status, could also be related to both age and visit frequency, so the observed group difference alone does not show that age caused the difference.

Worked Example: Predicting a Future Student’s Exam Score

Hypothetical scenario. A department has past-term records of midterm and final-exam scores and wants to predict final-exam scores for students taking the course next term. A prediction rule developed from the past records is, for illustration, \(\widehat{Y}=42+0.55X\), where \(X\) is midterm score and \(\widehat{Y}\) is predicted final-exam score, both measured in points.

Question type. This is prediction. The target is an outcome for a future student, not an average association that should be interpreted as a causal effect.

Outcome and predictor. The outcome is the future student’s final-exam score in points. The predictor is that student’s midterm score in points, available before the final exam.

Target population. The intended population is students taking this course next term who have a midterm score and will take the final exam. The past-term data can inform predictions for them only to the extent that the relationship and measurement conditions remain relevant. A prediction for a student whose midterm score is far outside the scores represented in the past data would require extra caution.

Quantity of interest and what the data support. For a student with a midterm score of 80 points, the stated rule gives

\widehat{Y}=42+0.55(80)=42+44=86\text{ points}

The rule predicts a final score of 86 points for that case. It does not guarantee the student will earn 86, and it does not say that raising a midterm score by one point would cause a final score to rise by 0.55 points. To judge whether the rule predicts well for next term’s students, its predictions should be checked on relevant data not used to develop it. The target is predictive performance for future cases, not simply how closely the rule fits the past records.

Worked Example: Estimating the Effect of an Assigned Program

Hypothetical scenario. Researchers randomly assign 100 eligible adults with elevated blood pressure to be offered either a 12-week counseling program or usual care. They measure systolic blood pressure at the end of 12 weeks. The research question is whether assignment to the program changes average systolic blood pressure compared with assignment to usual care.

Question type. This is causal because it asks about the effect of an assigned intervention relative to an alternative. Random assignment makes the groups comparable on average before the intervention, which supports a causal interpretation of the assignment contrast when follow-up and measurement are adequate.

Outcome and predictor. The outcome is systolic blood pressure after 12 weeks, in millimeters of mercury (mm Hg). The predictor is randomized assignment: counseling program or usual care. The causal contrast concerns assignment to these conditions, not necessarily actual attendance or adherence.

Target population. The direct study population is the 100 eligible adults enrolled in this hypothetical trial. Extending the result to other adults requires considering whether they are sufficiently similar to those enrolled and whether the program is delivered comparably.

Quantity of interest and what the data support. Suppose the observed mean is 126.4 mm Hg in the program-assignment group and 130.1 mm Hg in the usual-care group. The sample assignment contrast, program minus usual care, is

126.4-130.1=-3.7\text{ mm Hg}

The program-assignment group’s observed mean is 3.7 mm Hg lower at 12 weeks. With valid random assignment and sufficiently complete, comparable outcome measurement, the difference can estimate the average causal effect of assignment for the enrolled participants. It does not automatically establish the effect of receiving every counseling session, nor does it guarantee the same effect in a different population.

A Compact Specification Process

Before analyzing a proposed study, write down the following items in order. The answers should fit together: the quantity of interest should answer the stated question, and the data collection should provide relevant information about that quantity.

Name the question type.
Decide whether the aim is description, association, prediction, or a causal comparison. Do not infer the type just from the fact that a regression may be used.

Specify the outcome and predictors.
Name each variable in context and include its measurement unit when relevant. Identify which outcome is observed or sought, and which variables supply information about it.

Define the target population and time.
State whose outcomes matter and when they are relevant. For prediction, identify the future cases and the time at which their outcomes will be observed.

State the quantity of interest.
Be specific: for example, a conditional mean at a stated predictor value, a difference between two conditional means, a prediction for a future case, or an average effect of assignment.

Limit the conclusion to what the design supports.
Check how units were selected, how predictors and outcomes were measured, whether the predictor was assigned, and whether the data match the intended population and use.

Why the Distinctions Matter

A result can be accurate for one purpose and inadequate for another. A sample mean difference may describe the observed sample well but offer weak evidence about a broader population if participants were selected in a way that excluded important groups. A strong association in historical data may help predict future outcomes if the setting remains similar, yet still fail to answer a causal question. Conversely, randomized assignment can support a causal comparison within a study, but does not by itself establish that the same effect applies to people outside the enrolled group.

A useful check is to complete this sentence before interpreting any output: “For [target population], we want to learn [quantity of interest] about [outcome], using [predictor or assignment], based on [proposed data].” If the sentence cannot be completed precisely, the question is not yet ready for analysis.

Mistake: treating every predictor as a cause. A predictor is a variable used in description or prediction; its name does not establish a causal relationship. State an association as an association unless the design and assumptions support a causal claim.
Mistake: leaving the population unstated. “The mean difference is 4” is incomplete without saying which groups, outcome, units, and population are meant. Give the context and measurement units.
Mistake: treating a prediction as a guarantee. A predicted value is an estimate for a case, not a promise about that case’s observed outcome. Assess predictive performance for cases like those where the rule will be used.
Mistake: assuming a sample automatically represents a target population. The number of observations alone does not justify generalization. Consider recruitment, participation, measurement, and the connection between the observed units and the target.

Key takeaway: Specify the question type, outcome, predictors, target population, and quantity of interest before interpreting a regression result. Then state only the conclusions justified by the way the data were collected.

Check Your Understanding

For each scenario, identify the question type, outcome, predictor or predictors, target population, quantity of interest, and a conclusion the proposed data could support.

A city surveys residents about weekly exercise hours and age to describe exercise patterns among adults in the city.
A store uses customers’ past purchase totals to predict next month’s spending for customers who return next month. What population and predictive target should be specified?
A researcher observes that people who attend more tutoring sessions tend to have higher course grades. What can this association say, and what does it not establish by itself?
A hypothetical trial randomly assigns patients to two approved care plans and measures symptom scores after six weeks. What causal contrast is the study designed to estimate?
Why is “the predictor’s coefficient is 2” not a complete interpretation without naming the outcome, predictor units, population, and question type?