Turning Economic Questions Into Empirical Questions

From an Economic Concern to a Question Data Can Answer

Economic questions often begin broadly: Does job training improve employment? That wording names an important issue, but it leaves several empirical choices unresolved. Which workers count? What does “employment” mean here: having a job, hours worked, or earnings? What comparison is relevant? Is the aim to describe what happened, learn what a program caused, or forecast outcomes for particular workers?

Making those choices explicit is not just a matter of phrasing. Different choices lead to different data requirements and different answers. A well-formed empirical question states what outcome is of interest, who or what is observed, the relevant comparison or prediction inputs, and the setting and time period to which the answer is meant to apply.

Definition: An empirical question specifies a question that evidence from observations could address. Its core elements are an outcome (what is to be described, compared, or predicted), an observational unit (what each record or case concerns), a comparison or set of inputs (what distinguishes cases or informs a prediction), and a scope (the population, setting, and time period of interest). Its empirical target is the particular answer sought, such as an average, a difference, an intervention effect, or predictive accuracy.

A useful first draft is: “For [units] in [setting and period], what is [outcome], and how does it relate to or differ by [comparison or inputs]?” Then identify the kind of question being asked. The same economic topic can generate a descriptive, causal, or predictive question; these are not interchangeable labels for one analysis.

Three Kinds of Empirical Questions

A descriptive question asks what is observed in a defined group or how observed outcomes differ across groups. For example: “Among employees at these five stores in March, what was average weekly paid work time?” The answer describes those observations. Extending it to a larger population would require additional grounds for doing so.

A causal question asks what would change if a defined intervention, policy, or exposure were changed. “What is the effect of offering a transit subsidy on workers’ missed workdays?” is causal because it asks about an effect of an offer. A comparison of people who did and did not receive offers may be relevant evidence, but the observed difference by itself is not necessarily the effect of offering the subsidy.

A predictive question asks how well an outcome can be forecast for cases not yet observed or not used to construct the prediction. It specifies the outcome and the information available when making the forecast. A predictor can be useful for forecasting even if changing it would not cause the outcome to change.

For any of the three, scope matters. “Workers” could mean employees of one firm, workers in a particular region, or a broader labor force; “next year” and “over the next month” define different horizons. Naming the scope keeps the question from claiming more than its intended answer.

Worked Examples: Making the Question Specific

Worked Example: Describing Weekly Work Time

A fictional analyst asks, “How many hours do employees at this small company work?” To make the question answerable, the analyst specifies five surveyed employees, the month of March, and weekly paid work time in hours. The question is: “What was the mean weekly paid work time among these five employees in March?”

The observational units are the five employees; the outcome is each employee’s weekly paid work time, measured in hours per week. There is no treatment or group contrast in the question. Its target is the average among these five observed employees, not a causal effect and not automatically an average for all company employees.

Suppose their recorded weekly hours are 24, 32, 28, 20, and 26. The descriptive mean is the sum of the five values divided by five:

\bar{x}=\frac{24+32+28+20+26}{5} =\frac{130}{5} =26.0\ \text{hours per week}.

Thus, the five employees worked an average of 26.0 paid hours per week in the stated period. This calculation answers the defined descriptive question. On its own, it does not establish what caused those hours or what the average was for employees outside these five.

Worked Example: Asking About a Subsidy’s Effect

A fictional employer wants to know whether offering a transit subsidy changes missed workdays. Specify workers as the observational units, missed workdays per worker during the four weeks after the offer as the outcome, and an offer versus no offer as the contrast. The causal question concerns the effect of the offer, whether or not each worker takes up the subsidy.

Suppose eight workers are assigned by lottery: four are offered the subsidy and four are not. Assume in this hypothetical example that all eight outcomes are recorded over the same four-week period and the offer to one worker does not change another worker’s outcome. Missed workdays are 1, 0, 2, and 1 in the offer group, and 2, 1, 3, and 2 in the no-offer group. The observed group-average difference, defined as offer minus no offer, is:

\frac{1+0+2+1}{4}-\frac{2+1+3+2}{4} =\frac{4}{4}-\frac{8}{4} =1-2 =-1.00\ \text{missed workdays per worker over four weeks}.

The observed average was 1.00 fewer missed workdays in the offer group. Because the setup stipulates lottery assignment, complete outcomes, and no spillovers, this comparison is designed to address the causal question about the offer. It does not directly answer the effect of taking up the subsidy. The small hypothetical calculation also does not guarantee that the same difference would occur in another group of workers.

Why an observed gap is not automatically causal: For a worker $i$, let $Y_i(1)$ denote missed workdays if offered the subsidy and $Y_i(0)$ missed workdays if not offered it. The observed outcome is $Y_i=D_iY_i(1)+(1-D_i)Y_i(0)$, where $D_i=1$ indicates an offer and $D_i=0$ indicates no offer. The difference between observed group averages can be written as:

E[Y\mid D=1]-E[Y\mid D=0] = E[Y(1)-Y(0)\mid D=1] + \bigl(E[Y(0)\mid D=1]-E[Y(0)\mid D=0]\bigr).

The first term is the average effect of the offer among workers offered it. The second is a difference in what the two groups’ outcomes would have been without an offer. The identity follows by adding and subtracting $E[Y(0)\mid D=1]$ from the observed difference. It shows why a group gap can reflect more than an intervention effect. A research question should therefore state whether it seeks a descriptive gap or a causal effect; the design and assumptions needed to interpret evidence causally are separate matters.

Worked Example: Defining a Prediction Task

A fictional lender asks whether it can forecast next month’s household electricity spending. Define the observational unit as a household-month forecast, the outcome as spending in dollars in the following month, and the inputs as information available at the time the forecast is made. The question is predictive: how accurate are forecasts for households and months like those in the intended use?

Suppose a forecasting procedure is evaluated on four held-out household-months. The actual spending values are $90, $110, $100, and $120; the corresponding predictions are $95, $105, $98, and $112. One simple measure of prediction accuracy is the mean absolute error (MAE), the average absolute difference between predicted and actual values:

\mathrm{MAE} =\frac{|95-90|+|105-110|+|98-100|+|112-120|}{4} =\frac{5+5+2+8}{4} =\frac{20}{4} =\$5.00.

The four predictions missed actual spending by $5.00 on average in absolute terms. This reports performance on these held-out cases; whether it represents future performance depends on whether the evaluation cases and the intended future cases are sufficiently comparable. The question is not whether changing last month’s spending would cause next month’s spending to change. Here, earlier information is used to forecast a later outcome.

A Practical Specification Before Looking at Results

Before choosing an analysis, write a compact specification. Each item prevents a different kind of ambiguity:

Outcome: What quantity or event is to be described, compared, or predicted? Include its unit where relevant.
Observational unit: Is each case a worker, firm, household, transaction, or something else?
Comparison or inputs: For a descriptive contrast or causal question, what groups or intervention are being compared? For a prediction, what information is available at forecast time?
Scope: Which people or entities, location, and time period does the question concern?
Kind of answer: Is the aim description, a causal effect, or a forecast and its performance?

These are specifications of the question, not guarantees that the available data can answer it. For instance, a study may not observe the desired outcome, may lack the relevant comparison, or may contain information recorded only after the prediction would have been made. Such limitations should lead to a narrower or revised question, not an unsupported conclusion.

Common Mistakes

Using a broad outcome word without defining it. “Income” might mean hourly wages, annual earnings, or household income. State the intended outcome and its time scale.
Leaving the unit implicit. A record could refer to a worker, a firm, or a worker-month. Name what one case represents before interpreting a summary.
Calling any group difference an effect. “Workers offered a subsidy had fewer missed days” describes a comparison. “The offer reduced missed days” makes a causal claim that needs appropriate evidence and assumptions.
Treating a predictor as a cause. A variable that helps forecast an outcome need not be something whose change would alter the outcome.
Letting the answer outrun the scope. A result for five surveyed employees is not automatically a result for all workers in the company, let alone workers elsewhere.

Key takeaway: Turn an economic concern into a question with a specified outcome, observational unit, comparison or prediction inputs, and scope. Then state whether the intended answer is descriptive, causal, or predictive; that choice determines what evidence the question requires.

Check Your Understanding

For each question, identify what needs to be specified and decide whether the intended answer is descriptive, causal, or predictive.

“Do higher local wages reduce employee turnover?” Name a possible outcome, comparison, unit, and time period that would make this question more precise.
A report gives the average monthly rent for 12 neighborhoods. What might one observational unit be, and what does the reported average describe?
Why does finding fewer missed workdays among subsidy recipients than nonrecipients not, by itself, establish the effect of the subsidy?
A firm wants to forecast next quarter’s sales using information available today. What makes this a predictive rather than a causal question?
Suppose a study reports an average for five surveyed households. What additional scope information would you want before interpreting it as representative of a larger group?