From a Continuous History to a Treatment Rule
At a treatment decision, the relevant question is not simply whether a treatment works on average. It is whether the expected outcome under treatment is better than the expected outcome without treatment for a person with the history observed at that decision. When that history is continuous, a fitted Q-model can make the treatment contrast vary smoothly across its values, producing a threshold rule.
Consider a single decision with a binary treatment \(A\in\{0,1\}\), a continuous pre-treatment history \(H\), and an outcome \(Y\) for which larger values are better. Write \(Y^a\) for the potential outcome under action \(a\). The conditional causal mean is \(Q(h,a)=E[Y^a\mid H=h]\). A Q-model specifies or estimates these conditional means; the preferred action at history \(h\) is the action with the larger value.
For causal interpretation of the observed-data regression as a model for \(Q\), assume consistency, \(Y=Y^A\); conditional exchangeability, \(Y^a\perp A\mid H\) for each action; and positivity, \(P(A=a\mid H=h)>0\) for both actions at histories in the target population. Also require that the conditional means are well-defined and that the fitted Q-model is adequate on the history range where the rule will be used. These conditions are not guaranteed by fitting a regression: they are substantive and modelling assumptions.
A Linear Interaction Q-Model
A basic model allows the outcome to vary linearly with history and allows the treatment contrast to change linearly with history:
Here \(a\) is coded as 0 or 1. The terms \(\beta_0+\beta_1h\) describe the modelled mean under action 0. Under action 1, the mean is \(\beta_0+\beta_1h+\beta_2+\beta_3h\). Subtracting the first from the second gives
Thus \(\beta_2\) is the treatment contrast at \(h=0\), while \(\beta_3\) is the change in that contrast for a one-unit increase in history. The prognostic terms \(\beta_0\) and \(\beta_1\) affect predicted outcomes under both actions, but cancel from the comparison between actions.
Proof. The difference in predicted means is \(\Delta(h)=\beta_2+\beta_3h\). If \(\beta_3\ne0\), solving \(\Delta(h)=0\) gives \(h=-\beta_2/\beta_3=c\). Rewriting the contrast as \(\Delta(h)=\beta_3(h-c)\), its sign is the sign of \(h-c\) when \(\beta_3>0\), and the opposite sign when \(\beta_3<0\). A positive contrast favors action 1; a negative contrast favors action 0; zero means equal modelled means. If \(\beta_3=0\), then \(\Delta(h)=\beta_2\) for every \(h\), so the comparison does not depend on history. \(\square\)
This result is about the rule implied by the fitted model, not a guarantee that the rule is optimal in the population. If the threshold falls outside the history range where the model is supported, the fitted rule may assign the same action throughout the observed range. If both \(\beta_2=0\) and \(\beta_3=0\), the model ties the actions at every history.
Worked Example: Reading the Coefficients
Suppose a fitted model is \(\hat Q(h,a)=10+2h+3a-1.5ah\). Calculate the threshold and identify the preferred action on each side.
The estimated treatment contrast is
Set this contrast to zero: \(3-1.5h=0\), so \(h=3/1.5=2\). Since the interaction coefficient is negative, action 1 is preferred below the threshold and action 0 above it. For example, at \(h=0\), \(\hat\Delta(0)=3>0\), so action 1 is preferred. At \(h=4\), \(\hat\Delta(4)=3-6=-3<0\), so action 0 is preferred. At \(h=2\), the contrast is \(3-3=0\), so the model ties the actions.
The positive coefficient on \(h\) in the baseline part of the model does not determine which action is preferred. It raises both predicted means equally; the treatment decision is governed by \(3-1.5h\).
Fitting the Model and Deriving the Cutoff
For observed data \((H_i,A_i,Y_i)\), the linear Q-model can be fitted by regressing \(Y_i\) on an intercept, \(H_i\), \(A_i\), and \(A_iH_i\). The fitted coefficients are then substituted into \(\hat\Delta(h)=\hat\beta_2+\hat\beta_3h\). The cutoff is a feature of this estimated contrast, not an independently chosen clinical boundary.
The following example uses a deliberately simple data set so that the regression fit can be checked directly. It has observations under both actions at each of four history values, and the outcome is exactly generated by a linear interaction model.
Worked Example: Fitting a Q-Model From Paired History Values
Suppose the data are as follows. At each value of \(H\), there is one observation under each action.
| History \(H\) | Action \(A\) | Outcome \(Y\) |
|---|---|---|
| -2 | 0 | 6 |
| -2 | 1 | 12 |
| -1 | 0 | 8 |
| -1 | 1 | 12.5 |
| 1 | 0 | 12 |
| 1 | 1 | 13.5 |
| 2 | 0 | 14 |
| 2 | 1 | 14 |
Among the \(A=0\) observations, the outcomes follow \(Y=10+2H\): the four predictions are \(6,8,12,14\), exactly the four observed values. Thus the fitted baseline coefficients are \(\hat\beta_0=10\) and \(\hat\beta_1=2\). At each history, subtract the action-0 outcome from the action-1 outcome. The contrasts are \(6,4.5,1.5,0\) at \(H=-2,-1,1,2\), respectively.
These contrasts follow \(3-1.5H\): at \(H=-2\), \(3+3=6\); at \(H=-1\), \(3+1.5=4.5\); at \(H=1\), \(3-1.5=1.5\); and at \(H=2\), \(3-3=0\). Therefore \(\hat\beta_2=3\) and \(\hat\beta_3=-1.5\). The fitted model is
The design matrix has full column rank: the untreated rows identify the intercept and history slope, and the treatment rows at multiple distinct history values identify both the treatment main effect and interaction. The displayed coefficients therefore give an exact zero-residual fit. The resulting rule favors action 1 below \(H=2\), ties at \(H=2\), and favors action 0 above \(H=2\). In this small example, the tie occurs at the largest observed history value; conclusions beyond \(H=2\) would be extrapolations.
Worked Example: The Same Threshold With the Opposite Orientation
Suppose history is measured on the range \(0\le h\le10\), and a fitted model is \(\hat Q(h,a)=50-2h+a(-6+1.2h)\). The contrast is \(\hat\Delta(h)=-6+1.2h\), so the cutoff is \(c=6/1.2=5\). Because the interaction coefficient is positive, action 1 is preferred above 5 and action 0 below 5.
Check the direction by evaluating both sides. At \(h=2\), the contrast is \(-6+2.4=-3.6\). The action-0 prediction is \(50-4=46\), while the action-1 prediction is \(46-3.6=42.4\); action 0 is preferred. At \(h=8\), the contrast is \(-6+9.6=3.6\). The action-0 prediction is \(50-16=34\), while the action-1 prediction is \(34+3.6=37.6\); action 1 is preferred. At \(h=5\), the contrast is \(-6+6=0\), and both predictions equal \(40\).
This example also shows why one must calculate the sign of the interaction rather than memorize a direction such as “treat below the cutoff.” The sign determines which side favors treatment.
Support, Diagnostics, and Alternatives
A mathematical cutoff need not be a useful decision boundary. If \(H\) is observed only on \([L,U]\), compare \(c\) with that interval. A cutoff below \(L\) or above \(U\) implies a constant action over the observed range under the fitted linear model; it does not establish that the same action is preferable beyond the range. A cutoff at an endpoint leaves one side without observed history values. In all such cases, the decision relies on the model’s form, and any extrapolation deserves particular caution.
Before interpreting the rule, check whether both actions are represented across the relevant histories. A fitted regression can return coefficients even when treatment assignment is sparse in part of the history range, but the corresponding contrast may be weakly supported by data. Also inspect whether the assumed linear relationship between history and treatment contrast is plausible. Residual patterns, influential observations, and comparisons with a more flexible contrast can reveal lack of fit; they cannot by themselves verify causal exchangeability or positivity.
The cutoff is a ratio, \(-\hat\beta_2/\hat\beta_3\). When \(\hat\beta_3\) is near zero, a small change in estimated coefficients can move the cutoff greatly, or change whether it lies inside the observed range. A reported point estimate alone can therefore give a misleading impression of a precisely located boundary. If \(\hat\beta_3=0\), the fitted contrast is constant; if \(\hat\beta_2\) is also zero, the model gives a tie for every history.
A simple threshold rule is easy to communicate and can be appropriate when a linear treatment contrast is substantively credible. A model without a treatment-by-history interaction instead imposes a constant treatment contrast, so it cannot produce a history-dependent cutoff. A more flexible Q-model can represent curved contrasts or several decision boundaries, but its resulting rule is only as reliable as its fit and support in the relevant histories. These alternatives change the model assumptions; none removes the need for causal identification conditions and careful validation.
A common mistake is to interpret \(\hat\beta_1\), the history coefficient, as the change in treatment benefit. That coefficient describes how the action-0 mean changes with history. The treatment benefit is \(\hat\beta_2+\hat\beta_3h\), and only the treatment main effect and treatment-by-history interaction enter that contrast. Another mistake is to state a treatment direction without checking the sign: if \(\hat\beta_3<0\), action 1 is favored below the cutoff, not above it.
Check Your Understanding
Use the contrast, not the baseline outcome coefficients, to answer each question.
- A fitted model has \(\hat\Delta(h)=8-2h\). Find the cutoff and state the preferred action on either side.
- A fitted model has \(\hat\Delta(h)=-4+0.5h\), but the target history range is \(0\le h\le6\). What does the model imply on this range, and what is the limitation?
- In the model \(Q(h,a)=\beta_0+\beta_1h+\beta_2a+\beta_3ah\), which coefficients determine the treatment contrast, and why do the other two cancel?
- What happens to the rule if \(\beta_3=0\)? What if both \(\beta_2\) and \(\beta_3\) equal zero?
Solutions. (1) Solve \(8-2h=0\), giving \(c=4\). The interaction slope is negative, so action 1 is preferred for \(h<4\), action 0 for \(h>4\), and the model ties at 4. (2) The cutoff is \(c=4/0.5=8\). For every \(h\in[0,6]\), \(-4+0.5h\le-1<0\), so action 0 is preferred throughout the target range. The cutoff is outside that range, and the model does not establish the contrast’s behavior beyond the observed support. (3) The contrast is \(\beta_2+\beta_3h\). The terms \(\beta_0+\beta_1h\) appear in both action-specific means and subtract to zero. (4) If \(\beta_3=0\), the contrast is the constant \(\beta_2\), so the preferred action does not depend on history unless the contrast is zero. If both coefficients are zero, the model ties the actions for every history.