Pretrend Tests and Sensitivity to Trend Violations

Why Test Pre-Treatment Coefficients?

Difference-in-differences and event-study analyses use changes in outcomes for comparison groups to learn about what treated groups would have experienced without treatment. Their causal interpretation depends on a parallel-trends condition: in the absence of treatment, the relevant treated and comparison groups would have had the same changes in mean outcomes over the periods being compared. Pre-treatment event-study coefficients are often examined as a diagnostic because they summarize estimated differences in earlier changes.

A joint test can detect some departures from a specified pre-treatment pattern. It cannot verify the unobserved post-treatment counterfactual trend, and a non-significant result does not establish that parallel trends holds. Sensitivity analysis addresses a different question: how much could a plausible violation of parallel trends change the estimated treatment effect?

Definition: Let \(k\) denote event time, with \(k=-1\) the last pre-treatment period and \(k=0\) the first treatment period. Let \(\beta_k\) be an estimated event-time contrast relative to \(k=-1\), constructed using a stated comparison group and an estimator appropriate for the adoption design. A pre-treatment coefficient is a \(\beta_k\) for an event time \(k<-1\). A joint pretrend test evaluates whether a specified collection of these coefficients is zero together.

The definition is deliberately tied to the estimator and comparison group: a coefficient is not automatically a clean estimate of a causal pre-treatment contrast merely because it is labeled a “lead.” In staggered-adoption settings, the earlier material on group-time estimation and interaction-weighted event studies explains why the choice of estimator and valid controls matters. Pretrend testing does not repair a contaminated comparison or an unsuitable event-study specification.

What the Joint Test Evaluates

Choose the pre-treatment event times to be tested, say \(k\in\mathcal{K}_{-}\), and collect their estimates in the vector \(\hat{\boldsymbol{\beta}}_{-}\). The usual joint null is that all population coefficients in this selected set equal zero. A Wald statistic uses their estimated covariance matrix, so correlations among coefficient estimates are taken into account.

Definition: For \(q\) selected pre-treatment coefficients, the joint null is \(H_0:\boldsymbol{\beta}_{-}=\boldsymbol{0}\), against the alternative that at least one selected coefficient differs from zero. If \(\hat{\boldsymbol{\beta}}_{-}\) has estimated covariance matrix \(\widehat{V}\), the Wald statistic is \(W=\hat{\boldsymbol{\beta}}_{-}^{\mathsf T}\widehat{V}^{-1}\hat{\boldsymbol{\beta}}_{-}\). Under the null and suitable large-sample regularity conditions, \(W\) is approximately distributed as a chi-square random variable with \(q\) degrees of freedom.

The covariance estimate must reflect the dependence structure in the data and the design—for example, clustering at the level at which treatment is assigned when that is appropriate. The approximation also relies on the estimator’s large-sample inference being valid. If the covariance matrix is singular, the stated inverse-based statistic is not defined as written; the tested restrictions or inference method must be adjusted rather than silently applying this formula.

Worked Example: Calculating a Joint Pretrend Test

Suppose a hypothetical analysis estimates two pre-treatment coefficients, in outcome units, as \(\hat{\boldsymbol{\beta}}_{-}=(0.12,-0.08)^{\mathsf T}\). The reported covariance matrix is \(\widehat{V}=\begin{pmatrix}0.0100&0.0030\\0.0030&0.0144\end{pmatrix}\). Assume the event-study estimator and its covariance estimate are appropriate, and use the large-sample Wald reference distribution.

The determinant is \(0.0100(0.0144)-(0.0030)^2=0.000135\), so \(\widehat{V}^{-1}=(1/0.000135)\begin{pmatrix}0.0144&-0.0030\\-0.0030&0.0100\end{pmatrix}\). Thus

W = \frac{ 0.0144(0.12)^2-2(0.0030)(0.12)(-0.08)+0.0100(-0.08)^2 }{0.000135} = \frac{0.00032896}{0.000135} \approx 2.437.

There are \(q=2\) restrictions, so the reference distribution is chi-square with 2 degrees of freedom. Its upper-tail p-value is approximately \(P(\chi^2_2\geq 2.437)=0.296\). At a 5% significance level, this test does not reject the joint null. The result says that these estimates do not provide strong evidence against the specified zero-coefficient restrictions under the model and inference procedure. It does not show that the pre-treatment trends are exactly equal.

Why Non-Rejection Does Not Prove Parallel Trends

The test concerns a finite set of estimated pre-treatment contrasts. Parallel trends, by contrast, concerns the unobserved outcomes treated units would have experienced without treatment during the relevant post-treatment periods. Even precisely estimated zero leads cannot directly reveal those counterfactual outcomes. Further, a test may fail to reject because the sample provides little information to distinguish modest departures from zero; this is a power problem, not evidence that the departures are absent.

There are other limits. A test may be sensitive to the choice of pre-periods, event-study specification, comparison group, and covariance estimator. A rejection can flag a problem with the proposed design or specification, but does not by itself identify the cause. In particular, it is not a license to choose specifications or periods only after seeing which produce a preferred treatment estimate. Pretrend tests are most informative when the analysis plan, tested coefficients, and inferential method are clearly stated.

A useful way to see what is at stake is to write the treated-control gap as a sum of a treatment effect and a difference in untreated outcomes. For a simple treated-group versus comparison-group contrast, let \(d_t\) be the gap in mean untreated potential outcomes at time \(t\), and let \(\tau_t\) be the treated group’s treatment effect at that time. The observed gap is \(d_t+\tau_t\). Taking its change from a pre-treatment baseline gives the following identity.

\big[(d_k+\tau_k)-(d_{-1}+\tau_{-1})\big] = (\tau_k-\tau_{-1})+(d_k-d_{-1}).

Here the baseline \(k=-1\) is pre-treatment, so, under no anticipation, \(\tau_{-1}=0\). The post-treatment change in the observed gap is then \(\tau_k+(d_k-d_{-1})\). The second term is the violation of parallel trends over that comparison: it is zero if the untreated gap stays constant. This identity distinguishes the causal effect from the trend-related bias; it does not say that an estimated pretrend can determine the post-treatment value of that bias.

Worked Example: Separating Effect From Trend Violation

Consider hypothetical mean outcomes for a treated group and a valid comparison group. At the pre-treatment baseline, their observed gap is 1.00 outcome units. At a post-treatment time, the observed gap is 1.75 units. Suppose external substantive judgment, not the observed post-treatment outcomes alone, makes a possible untreated gap of 1.15 units at that time plausible.

The observed change in the gap is \(1.75-1.00=0.75\) units. The untreated gap change is \(1.15-1.00=0.15\) units. Since the baseline is pre-treatment and has no treatment effect, the implied effect under this hypothetical counterfactual is

\tau_k = \underbrace{(1.75-1.00)}_{\text{observed gap change}} - \underbrace{(1.15-1.00)}_{\text{untreated gap change}} = 0.75-0.15 = 0.60\text{ outcome units}.

If instead the untreated gap had stayed at 1.00, the effect would be 0.75 units. The 0.15-unit difference is the bias from the specified trend violation. These numbers illustrate the decomposition; they do not establish which untreated gap is correct in an actual application.

Sensitivity Bounds for Trend Violations

A sensitivity analysis replaces the exact parallel-trends requirement with an explicit bound on how much the untreated trend may depart from it. There is no single bound that is correct for every study. The restriction should be described in the units of the outcome and justified using substantive knowledge, study design, and the observed pre-treatment history. The following is one transparent, deliberately simple option for a single treated-versus-comparison contrast.

Definition: Let \(d_t\) again denote the gap in untreated mean outcomes, and define the period-to-period untreated gap change as \(\delta_j=d_j-d_{j-1}\). Let \(\mathcal{J}_{-}\) be a specified set of pre-treatment periods over which these changes are assessed, and let \(B\) be a chosen upper bound on their absolute size. A relative-magnitude sensitivity restriction assumes that each post-baseline untreated gap change satisfies \(|\delta_j|\leq M B\), where \(M\geq0\) is a sensitivity parameter. This restriction is an assumption, not a consequence of passing a pretrend test.

For this particular restriction, one may set \(B\) to a substantively chosen scale informed by the largest plausible pre-treatment gap change. Using the largest estimated pre-period change as \(B\) is a descriptive choice, not proof that all future changes are bounded by it; sampling uncertainty and limited pre-treatment data should be considered. \(M=0\) imposes no post-treatment gap changes, while larger values allow larger departures. The restriction bounds each period’s change, not the total change from baseline.

Derivation: If there are \(h\) periods from the pre-treatment baseline to event time \(k\), then \(d_k-d_{-1}=\sum_{j=1}^{h}\delta_j\). Under \(|\delta_j|\leq MB\), the triangle inequality gives \(|d_k-d_{-1}|\leq\sum_{j=1}^{h}|\delta_j|\leq hMB\). Because the observed gap change equals \(\tau_k+(d_k-d_{-1})\), an estimate \(\hat{\Delta}_k\) of that change implies \(\tau_k\in[\hat{\Delta}_k-hMB,\hat{\Delta}_k+hMB]\) under this bound.

This argument is a bound, not a claim that the trend violation reaches either endpoint. With multiple periods, the worst-case cumulative bias can grow with the horizon because each period’s change may contribute. A different sensitivity model—such as one that restricts changes in the trend rather than the size of each change—would generally produce different bounds and must be stated and derived separately.

Worked Example: How Much Trend Deviation Would Change the Conclusion?

Suppose a hypothetical analysis estimates a post-treatment gap change of \(\hat{\Delta}_k=0.50\) outcome units, with standard error \(0.18\). Its conventional 95% confidence interval is \(0.50\pm1.96(0.18)\), or approximately \([0.147,0.853]\) outcome units. Suppose the post-treatment event time is \(h=3\) periods after the pre-treatment baseline. For illustration, take \(B=0.030\) outcome units per period and examine \(M=1.5\). The bound on the cumulative trend bias is

hMB=3(1.5)(0.030)=0.135\text{ outcome units}.

The point estimate’s sensitivity interval for the treatment effect is therefore \([0.50-0.135,\;0.50+0.135]=[0.365,0.635]\) outcome units. To account for sampling uncertainty as well, enlarge both ends of the conventional confidence interval by the maximum bias:

[0.147-0.135,\;0.853+0.135]=[0.012,\;0.988]\text{ outcome units}.

Under the stated sensitivity restriction and the assumed validity of the conventional interval for the observed gap change, this enlarged interval remains just above zero. At \(M=2\), the maximum bias is \(3(2)(0.030)=0.180\) units, and the enlarged interval becomes approximately \([-0.033,1.033]\). It then includes zero. Thus the conclusion’s robustness depends on the allowed magnitude of trend deviations; neither value of \(M\) is automatically appropriate.

This calculation treats \(B\) as fixed for illustration. In practice, if \(B\) is estimated from noisy pre-treatment changes, uncertainty in that estimate can matter. A careful analysis should explain how the sensitivity scale was chosen and, where material, account for its uncertainty rather than presenting the bound as known exactly.

Reporting and Common Pitfalls

Report the tested event times, the joint statistic, degrees of freedom, p-value, and the covariance or clustering approach used for inference. Explain what comparison and estimator generated the coefficients. Then distinguish the test result from the causal assumption: say that the test did or did not reject the selected zero-lead restrictions, not that it proved or disproved parallel trends in all periods.

For sensitivity analysis, state the restriction, the scale \(B\), the value or range of \(M\), the horizon \(h\), and the resulting effect or confidence interval, with outcome units. Showing a range of \(M\) values is often more informative than choosing a single value without context. The sensitivity interval is conditional on the specified bound and other identification and inference assumptions; it is not a guarantee against every possible violation.

Mistake: “The pretrend test is not significant, so parallel trends is confirmed.” A non-rejection may reflect limited power, and the test concerns selected pre-treatment contrasts rather than the unobserved post-treatment counterfactual. Report it as a diagnostic, then assess how conclusions respond to explicit trend-violation bounds.
Mistake: “The observed post-treatment change is the treatment effect.” It also includes the change in the untreated treated-control gap. Subtract or bound that component using a stated sensitivity model.
Mistake: ignoring the baseline. The decomposition uses \((\tau_k-\tau_{-1})+(d_k-d_{-1})\). It reduces to \(\tau_k+(d_k-d_{-1})\) only when the baseline is untreated and has no anticipation, so that \(\tau_{-1}=0\).
Mistake: treating a bound as data-proven. Pre-treatment estimates can inform a sensitivity scale, but they do not establish that post-treatment deviations obey it. Justify and vary the restriction.

Key takeaway: A joint pretrend test evaluates a specified set of pre-treatment restrictions; it does not certify the post-treatment counterfactual. Sensitivity bounds make the implications of possible trend violations explicit, while keeping the assumptions behind those bounds visible.

Check Your Understanding

Answer each question using the definitions and assumptions stated above.

What null hypothesis does a joint test of three selected pre-treatment coefficients evaluate?
A joint pretrend test has a large p-value. Give two reasons this result does not establish parallel trends after treatment.
If the baseline is pre-treatment, write the observed change in the treated-control gap as the sum of the treatment-effect change and the untreated-gap change.
Under the restriction \(|\delta_j|\leq MB\), what bound applies to the cumulative untreated-gap change across \(h\) periods?
In a sensitivity analysis, why should the analyst explain how \(B\) and \(M\) were chosen?