Why Diagnose a Fitted Blip Model?
A fitted structural nested mean model (SNMM) summarizes how treatment changes an outcome through blip functions. G-estimation uses treatment variation, conditional on measured history, to estimate parameters in those functions. But a numerical estimate alone does not show that the chosen blip function is an adequate description. A constant treatment effect, for example, may be too restrictive if the effect varies with a measured covariate or with treatment dose.
Diagnostics examine implications of the fitted model that should hold when the blip specification and the identifying conditions are appropriate. They can reveal systematic failures, such as remaining treatment-related patterns in a blipped-down outcome. They cannot prove the model correct: unmeasured confounding, a misspecified treatment model, weak treatment variation, and limited sample size can complicate the same checks.
The Moment Restriction Being Checked
Let \(\mathcal H_t\) denote the measured history available just before treatment \(A_t\), and let \(e_t(\mathcal H_t)=E(A_t\mid\mathcal H_t)\) be its conditional mean. For binary treatment, this is the conditional treatment probability. Let \(U_t(\psi)\) denote the outcome transformed by blipping down the modeled treatment effects through time \(t\), using the proposed parameter \(\psi\). The precise transformation depends on the SNMM and the treatment times; the diagnostic principle is the same.
The key qualification is “at the true blip parameter.” If \(\psi\) is wrong, blipping down with that value can leave treatment effects in \(U_t(\psi)\), so the transformed outcome may still vary with treatment after conditioning on history. A diagnostic looks for this remaining pattern, rather than assuming every candidate parameter satisfies the moment.
For binary \(A_t\), write \(e=e_t(\mathcal H_t)\) within a fixed history stratum. When \(0<e<1\), the conditional moment has a useful form:
To see this, expand the expectation over the two treatment values. The treated contribution is \(e(1-e)E(U_t\mid A_t=1,\mathcal H_t)\); the untreated contribution is \(-(1-e)eE(U_t\mid A_t=0,\mathcal H_t)\). Their sum is the expression above. Thus a zero moment means equal conditional means of the blipped-down outcome in the treated and untreated groups, for the history stratum in question. The factor \(e(1-e)\) also shows why this diagnostic has little information when treatment is nearly deterministic.
This argument relies on the causal identification conditions used for G-estimation: consistency, sequential exchangeability given the measured history, and adequate treatment variation, together with a correctly specified treatment mean \(e_t(\mathcal H_t)\). These conditions are not established by a residual plot. In particular, a moment failure may reflect a wrong blip model, but it may also reflect a wrong treatment model or a failure of exchangeability.
Inspecting Residual Patterns
After fitting a candidate blip model, calculate the corresponding \(U_t(\hat\psi)\) and treatment residual \(R_t=A_t-\hat e_t(\mathcal H_t)\). Useful checks include estimates of \(E\{R_tU_t(\hat\psi)\mid\mathcal H_t\}\) across scientifically meaningful strata, and plots or smooth summaries of the transformed outcome against treatment residuals and history variables. A systematic nonzero pattern suggests that the fitted model has not removed all treatment-related mean structure.
The conditional restriction is stronger than a single overall average restriction. Positive and negative conditional moments can cancel when averaged over histories. Accordingly, a near-zero overall moment does not rule out a pattern in subgroups. Conversely, slicing data into many small subgroups produces noisy estimates and can create apparent patterns by chance. Use clinically or scientifically motivated summaries, show uncertainty where possible, and treat exploratory patterns as prompts for model checking rather than as definitive tests.
Worked Example: A Constant Blip Leaves a History Pattern
Consider a hypothetical one-time binary treatment, with a binary measured covariate \(L\). Suppose treatment is randomized with \(e(L)=0.5\) in both strata, and the hypothetical outcome means are:
| History stratum | Mean outcome if untreated | Mean outcome if treated |
|---|---|---|
| \(L=0\) | 10 | 12 |
| \(L=1\) | 10 | 14 |
Assume consistency, no interference, and the stated randomized treatment assignment. The table has no confounding by \(L\), and treatment varies within each stratum. A constant additive blip \(\psi=2\) gives the blipped-down outcome \(U(\psi)=Y-\psi A\). For \(L=0\), the untreated and treated means of \(U(2)\) are 10 and \(12-2=10\); the moment is \(0.5(1-0.5)(10-10)=0\).
For \(L=1\), the corresponding means are 10 and \(14-2=12\), so \(E\{(A-0.5)U(2)\mid L=1\}=0.5(1-0.5)(12-10)=0.50\). The constant blip therefore leaves a positive treatment-related pattern in the \(L=1\) stratum. This is consistent with effect modification: the hypothetical treatment contrast is 2 when \(L=0\) and 4 when \(L=1\), not one common value. In a real analysis, sampling variation would need to be considered before interpreting an estimated moment of this size.
Use Alternative Blip Specifications Deliberately
A residual pattern can suggest what feature of the blip function may be missing. For example, a pattern across a baseline covariate may motivate an interaction; curvature with a continuous dose may motivate a nonlinear dose function; and a pattern across treatment times may motivate different time-specific blips. These are candidate explanations to examine, not conclusions established by a plot.
Compare the original specification with an alternative that has a substantive rationale. Re-estimate the blip parameters under the alternative, then check whether the conditional moment patterns are reduced and whether the resulting effects remain interpretable. A more flexible model will often fit the observed moments more closely simply because it has more parameters. The improvement must be judged with uncertainty and attention to overfitting, not by fit alone.
Worked Example: Adding a Blip Interaction
Use the hypothetical means in the preceding table. Replace the constant blip by \(\gamma(A,L)=A(\psi_0+\psi_1L)\), so \(U(\psi)=Y-A(\psi_0+\psi_1L)\). Assume again that treatment is randomized with probability 0.5 in each \(L\) stratum, and that the additive mean model is the proposed model. In the \(L=0\) stratum, the treated-minus-untreated outcome contrast is 2, so the fitted blip requires \(\psi_0=2\). In the \(L=1\) stratum, the contrast is 4, so \(\psi_0+\psi_1=4\), giving \(\psi_1=2\).
Check the moments. When \(L=0\), the treated transformed mean is \(12-(2+2\cdot0)=10\), equal to the untreated mean 10; hence the moment is \(0.25(10-10)=0\). When \(L=1\), it is \(14-(2+2\cdot1)=10\), again equal to the untreated mean; the moment is \(0.25(10-10)=0\). Under these idealized cell means, the interaction removes the pattern left by the constant model.
This calculation does not establish that the interaction is the uniquely correct causal model. It shows that this alternative represents the displayed conditional contrasts and satisfies these particular cell-level moment checks. With observed data, parameter uncertainty, treatment-model adequacy, and the number of comparisons would also matter.
Overidentifying Moment Checks
G-estimation can use several moment conditions, often formed by multiplying a treatment residual by functions of measured history. At the true parameter, valid moments have the form \(E\{g_j(\mathcal H_t)R_tU_t(\psi_0)\}=0\), where \(g_j\) is a specified function. With \(q\) moments and \(p\) free blip parameters, more moments than parameters (\(q>p\)) provide overidentifying restrictions: the model cannot generally force every sample moment to zero by parameter choice alone.
A generalized method of moments (GMM) overidentification test summarizes whether the extra restrictions are collectively inconsistent with the data. In a standard large-sample setting, with valid moments, a correctly specified model, suitable regularity conditions, and an appropriate covariance estimate, the test statistic is compared with a chi-squared distribution with \(q-p\) degrees of freedom. This is an asymptotic result, so small samples or weak treatment variation can make the approximation unreliable.
Worked Example: Interpreting an Overidentification Statistic
Suppose one blip parameter is estimated using two moment conditions, so \(q=2\) and \(p=1\). For illustration, assume the fitted parameter makes the first sample moment zero, while the second sample moment is 0.020. Assume \(n=100\), that the estimated covariance matrix of the individual moment contributions is \(\hat\Sigma=\begin{pmatrix}0.04&0\\0&0.04\end{pmatrix}\), and that the stated covariance approximation and regularity conditions are appropriate for this hypothetical calculation.
The GMM quadratic form is \(J=n\bar g^{\,T}\hat\Sigma^{-1}\bar g\), where \(\bar g=(0,0.020)^T\). Since \(\hat\Sigma^{-1}\) has diagonal entries \(1/0.04=25\), \(J=100\{0^2(25)+(0.020)^2(25)\}=100(0.010)=1.00\).
There is \(q-p=1\) degree of freedom. Comparing \(J=1.00\) with a chi-squared distribution on one degree of freedom gives a p-value of about 0.317. At a 0.05 significance level, this calculation does not reject the joint moment restrictions. It does not show that the blip model is true, that the causal assumptions hold, or that important departures are absent; the test may have limited power, and its conclusion depends on the stated covariance and regularity assumptions.
What Diagnostics Can—and Cannot—Establish
Diagnostics matter because a blip model is a causal summary, not merely a convenient regression curve. If the model is misspecified, the estimated parameter may be a compromise determined by the chosen moments rather than the treatment effect pattern the analyst intended to represent. Conditional checks can show where that compromise leaves systematic structure, while alternative specifications can test whether a plausible refinement addresses it.
A diagnostic failure is not uniquely attributable to the blip function. Revisit whether the treatment mean \(e_t(\mathcal H_t)\) was modeled adequately, whether the available history supports sequential exchangeability, and whether treatment variation is sufficient. The earlier material on treatment variation and positivity is relevant here: if some histories almost always receive one treatment, the conditional contrast and its moment are poorly informed. Correcting a blip function cannot supply information that the data do not contain.
Likewise, passing a finite collection of checks is not proof of adequacy. A test may have low power, selected moments may be insensitive to a particular departure, and unmeasured confounding can remain undetected. Report the blip specification, the moment functions examined, the patterns found, and the uncertainty in those patterns. Interpret non-rejection as “these checks did not detect a departure,” not as “the model has been verified.”
Common Mistakes
- Assuming the moment is zero for every proposed parameter. It is a restriction at the true parameter under the model and identification conditions. A wrong blip value can leave treatment-related structure in the transformed outcome.
- Checking only the overall moment. Conditional failures can cancel across history strata. Examine meaningful history patterns as well as aggregate summaries.
- Treating a residual pattern as proof of blip misspecification. Treatment-model error or failed exchangeability can also produce a violation. These explanations require separate scrutiny.
- Taking a non-significant overidentification test as confirmation. Non-rejection may reflect limited power or uninformative moments; it is not proof that all relevant restrictions hold.
- Adding interactions until every pattern disappears. Flexible specifications can overfit. Choose alternatives for substantive reasons and account for uncertainty and the number of checks performed.
Check Your Understanding
Use the definitions and examples above to answer the following questions.
- Why is \(E\{[A_t-e_t(\mathcal H_t)]U_t(\psi)\mid\mathcal H_t\}=0\) not guaranteed for an arbitrary proposed \(\psi\)?
- For binary treatment with \(e=0.4\), suppose the treated and untreated conditional means of \(U_t(\psi_0)\) are both 7. What is the conditional treatment-residual moment?
- Why might an overall moment close to zero fail to reveal a conditional pattern across history strata?
- A model uses three moment restrictions to estimate two blip parameters. Under the standard large-sample overidentification result, how many degrees of freedom are used, and what does rejection indicate?
- Name two reasons, other than a misspecified blip function, that could contribute to a nonzero fitted moment.