Sensitivity Curves for Departures From Parallel Trends

Why Examine Departures From Parallel Trends?

A difference-in-differences (DiD) estimate is often interpreted under a parallel-trends assumption: without treatment, the treated and comparison groups would have had the same average outcome change over the study interval. That assumption cannot usually be verified from observed post-treatment outcomes. Pre-treatment trends and other diagnostics can inform its plausibility, but they do not reveal exactly what the treated group’s untreated post-period trend would have been.

A sensitivity curve makes this uncertainty explicit. It asks how the estimated average treatment effect on the treated (ATT) would change if the treated group’s untreated trend differed from the comparison group’s trend by a specified amount. The result is not a new estimate of the unknown departure. It is a set of conditional estimates: one for each departure value considered.

What the Adjustment Represents

Consider a two-group, two-period setting. Let \(Y_{T,pre}\) and \(Y_{T,post}\) denote average observed outcomes in the treated group before and after treatment, and let \(Y_{C,pre}\) and \(Y_{C,post}\) denote the corresponding comparison-group averages. The usual sample DiD contrast is

Suppose treatment does not affect the pre-period outcome (no anticipation), the comparison group is untreated over the contrast, and the observed group averages refer to comparable populations over time. Write \(\Delta Y_T(0)\) for the treated group’s counterfactual untreated change and \(\Delta Y_C(0)\) for the comparison group’s untreated change. The average treatment effect on the treated over the post period is the difference between the treated group’s observed post outcome and its counterfactual untreated post outcome. Consequently, the treated group’s observed change equals its untreated change plus the ATT. The comparison group’s observed change equals its untreated change. Subtracting gives the population relation

Rearranging yields \(ATT=D-\delta\), and replacing the population contrast with its sample estimate gives the sensitivity-adjusted estimate. Under exact parallel trends, \(\delta=0\), so the usual DiD estimate is recovered. If the treated group would have had a more favorable untreated trend, \(\delta>0\), the unadjusted DiD contrast overstates the ATT by that amount. If its untreated trend would have been less favorable, \(\delta<0\), the adjustment increases the estimated ATT.

The parameter must be expressed on the same scale as the DiD contrast. If the outcome is measured in days, \(\delta\) is a difference in changes measured in days. If the outcome is a rate per 1,000 people, \(\delta\) is in rate points per 1,000 people. With a pooled or repeated-period DiD contrast, define \(\delta\) for that exact contrast; the relevant departure is the difference in counterfactual changes after applying the same weights as the estimate.

Reading and Plotting the Sensitivity Curve

For a fixed DiD estimate, the adjusted estimate is a straight-line function of \(\delta\). Its intercept is \(\widehat{D}\), and its slope is \(-1\): increasing the assumed departure by one outcome unit decreases the adjusted estimate by one outcome unit. Plot \(\delta\) on the horizontal axis and \(\widehat{ATT}(\delta)\) on the vertical axis. Mark a substantively plausible range of departures, and show the horizontal zero line to make sign changes easy to see.

The zero crossing solves \(\widehat{D}-\delta=0\), so it occurs at \(\delta=\widehat{D}\). For a positive unadjusted estimate, departures larger than the estimate produce a negative adjusted estimate; departures smaller than it produce a positive one. This is a tipping point for the sign of the point estimate, not necessarily for its statistical significance or practical importance.

Adding Sampling Uncertainty

A sensitivity curve for point estimates does not show sampling uncertainty. If a confidence interval for the DiD contrast is available and a departure \(\delta\) is treated as fixed, the corresponding interval for the adjusted estimate is obtained by subtracting \(\delta\) from both endpoints. This is a translation of the interval, not a recalculation of its standard error. It relies on using the same sampling procedure and uncertainty estimate for the DiD contrast at each fixed departure.

For instance, with a large-sample normal approximation, a 95% confidence interval for the DiD contrast has the form \(\widehat{D}\pm1.96SE\). If inference uses another reference distribution or a cluster-robust method, use the interval from that method instead; the sensitivity adjustment still shifts both endpoints by \(-\delta\). The resulting interval is conditional on the chosen departure and does not account for uncertainty about which departure is appropriate.

Sign, Statistical, and Practical Tipping Points

Different conclusions can have different tipping points. The sign tipping point is where the adjusted point estimate reaches zero. For a positive estimate, a confidence interval’s lower endpoint may reach zero at a smaller departure. If the interval for \(\widehat{D}\) is \([L,U]\), the shifted interval is \([L-\delta,U-\delta]\). Its lower endpoint is zero when \(\delta=L\); for larger departures, that interval no longer lies entirely above zero. This statement concerns the chosen confidence-interval procedure and a fixed departure.

A practical threshold can be different again. If effects smaller than a specified meaningful magnitude are not considered substantively important, compare \(\widehat{ATT}(\delta)\) with that threshold across the curve. Statistical significance, effect direction, and practical importance are distinct criteria. A curve should therefore be interpreted against the question being asked, not only by whether it crosses zero.

A useful sensitivity analysis reports how the plausible departure range was chosen. The range might be informed by pre-treatment trend differences, subject-matter knowledge, or comparisons with outcome changes in relevant settings. Those sources can motivate scenarios, but a pre-treatment trend is not automatically the post-period counterfactual departure. Present the range as an assumption for sensitivity analysis, not as a measured fact.

Common Mistake: Treating the Curve as a Test of Parallel Trends

A common mistake is to see a sensitivity curve that remains positive over a chosen range and conclude that parallel trends has been demonstrated. That reasoning confuses robustness to specified departures with evidence that the actual departure is zero. The curve only reports what the estimate would be under each value supplied by the analyst. It cannot identify the unobserved counterfactual trend or determine which values are plausible without additional substantive reasoning.

A related mistake is to add \(\delta\) to the DiD estimate when \(\delta\) is defined as the treated group’s untreated change minus the comparison group’s untreated change. The observed contrast includes that trend difference as well as the treatment effect, so it must be removed: \(\widehat{ATT}(\delta)=\widehat{D}-\delta\). State the sign convention before interpreting the direction of an adjustment.