Sensitivity Analysis for Violations of Parallel Trends

Why Analyze Departures From Parallel Trends?

Difference-in-differences (DiD) uses changes in an untreated comparison group to estimate how the treated group would have changed without treatment. Its causal interpretation depends on a parallel-trends condition: absent treatment, the groups would have had the same average outcome change over the relevant periods. In applications, that condition is not directly verifiable. A sensitivity analysis asks a more useful supplementary question: how would the treatment-effect conclusion change if untreated trends differed by a specified, bounded amount?

The answer is not a test of whether parallel trends is true. Instead, it makes the consequence of a possible violation explicit. A small departure may leave an estimate and its uncertainty substantively similar; a larger one may make a previously positive or negative conclusion compatible with zero. The bound must be stated in the outcome’s units and justified as a plausible scenario.

A Bounded-Bias Model for DiD

Definition: Let $\hat{\tau}$ be a DiD estimate of a treatment effect $\tau$. Represent its departure from the target as $\hat{\tau}=\tau+b+e$, where $b$ is bias from a violation of parallel trends and $e$ is sampling error. A bounded-bias sensitivity analysis assumes $b\in[-M,M]$, for a chosen bound $M\geq 0$. Thus the unobserved difference in untreated trends may shift the estimate by at most $M$ outcome units in either direction.

For the basic two-group, two-period design, the link between the bias and the parallel-trends violation is direct. Suppose $D=1$ denotes the treated group, and “pre” and “post” denote the periods before and after treatment. Write $Y(0)$ for the outcome that would be observed without treatment. The difference between the groups’ changes in untreated outcomes is

\delta = \left\{E[Y_{\mathrm{post}}(0)\mid D=1]-E[Y_{\mathrm{pre}}(0)\mid D=1]\right\} - \left\{E[Y_{\mathrm{post}}(0)\mid D=0]-E[Y_{\mathrm{pre}}(0)\mid D=0]\right\}.

When treatment affects the treated group only in the post period, the DiD contrast equals the average treatment effect on the treated plus $\delta$. Under parallel trends, $\delta=0$; under bounded departures, assume $|\delta|\leq M$. This interpretation also requires a well-defined treatment and outcome, no interference between units, no anticipation in the pre period, and a comparison group that supplies the relevant counterfactual trend. The sensitivity bound addresses the trend restriction; it does not repair failures of those other design conditions.

Suppose the sampling procedure gives a valid $100(1-\alpha)\%$ confidence interval for $\tau+b$, with standard error $SE$ and critical value $c$. For example, $c$ may be a normal critical value under an appropriate large-sample approximation, or a critical value from the design’s specified inference procedure. The sensitivity interval must account for both sampling uncertainty and bounded bias.

Constructing a Sensitivity Interval

For a fixed value of the bias $b$, subtracting $b$ from the ordinary confidence interval for $\tau+b$ gives an interval for $\tau$. If all values in $[-M,M]$ are considered possible, take the union of those intervals. Its lower endpoint occurs at $b=M$, and its upper endpoint at $b=-M$.

\begin{aligned} \text{ordinary interval for }\tau+b &= [\hat{\tau}-cSE,\ \hat{\tau}+cSE],\\ \text{interval for }\tau\text{ at fixed }b &= [\hat{\tau}-cSE-b,\ \hat{\tau}+cSE-b],\\ \text{sensitivity interval for }b\in[-M,M] &= [\hat{\tau}-cSE-M,\ \hat{\tau}+cSE+M]. \end{aligned}

This is a conservative interval for the target across the allowed biases: if the sampling interval covers $\tau+b$ and the true $b$ lies within the stated bound, the sensitivity interval covers $\tau$. Its coverage depends on valid sampling inference and on the bound actually containing the bias. It is not a probability statement that the fixed parameter lies in the computed interval.

A bound need not be symmetric. If the plausible bias is $b\in[L,U]$, the corresponding interval is $[\hat{\tau}-cSE-U,\ \hat{\tau}+cSE-L]$. This allows a researcher to represent, for example, a setting where background trends are believed more likely to bias the estimate upward than downward. The chosen range and its justification should be reported, rather than hidden in a single adjusted estimate.

Worked Example: A Two-Period DiD Estimate

In a hypothetical wage analysis, the treated group’s mean rises from 10.0 to 11.2 hundred dollars per week, while the comparison group’s mean rises from 10.1 to 10.8 hundred dollars per week. Suppose the DiD estimate has a valid standard error of 0.20 hundred dollars per week. Use a symmetric bias bound of 0.25 hundred dollars per week and a 95% normal critical value of 1.96.

Compute the estimate. The treated group changes by $11.2-10.0=1.2$ hundred dollars per week, and the comparison group changes by $10.8-10.1=0.7$ hundred dollars per week. Therefore,

\hat{\tau}=(11.2-10.0)-(10.8-10.1)=1.2-0.7=0.5

The estimate is 0.5 hundred dollars, or $50, per week. This arithmetic is consistent with the four group means. The conventional 95% interval has sampling margin $1.96(0.20)=0.392$ hundred dollars, so it is $[0.5-0.392,\ 0.5+0.392]=[0.108,\ 0.892]$ hundred dollars per week.

Apply the bound. With $M=0.25$, the sensitivity interval is $[0.5-0.392-0.25,\ 0.5+0.392+0.25]=[-0.142,\ 1.142]$ hundred dollars per week. In dollars, that is approximately $[-$14.20,\ $114.20]$ per week. Under this bound, zero is inside the interval. Thus the positive conventional estimate is not robust, at the 95% interval level, to every violation allowed by the chosen bound. This conclusion assumes the stated standard error is valid for the design and that 0.25 hundred dollars is a defensible maximum trend difference; the arithmetic itself cannot validate either assumption.

Sensitivity for Event-Study Effects

An event study reports effects at different times relative to treatment, often with one pre-treatment period omitted as the reference period. For an event-time effect, the parallel-trends departure can be expressed as a change in the untreated treated-versus-comparison outcome gap between the event period and the reference period. Let $\hat{\beta}_k$ estimate the effect $\tau_k$ at event time $k$, and let $b_k$ denote this change in the untreated gap. For a correctly specified event-study estimand and valid inference, the relevant sensitivity representation is $\hat{\beta}_k=\tau_k+b_k+e_k$. A period-specific bound $|b_k|\leq M_k$ gives

[\hat{\beta}_k-c_kSE_k-M_k,\ \hat{\beta}_k+c_kSE_k+M_k].

The bound concerns the difference in untreated trends relative to the chosen reference period, not the observed event-study coefficient itself. It can vary across event times: a longer horizon may plausibly allow more drift in the untreated gap. Each $M_k$ should therefore be explained in the outcome’s units. The interval above is pointwise. If the goal is to make a simultaneous claim about several event times, the critical values must come from an inference procedure that provides simultaneous coverage; separate pointwise intervals do not automatically do so.

Worked Example: Bounded Drift in Two Event-Study Periods

Suppose a hypothetical event study measures employment in percentage points and uses the last pre-treatment period as its reference. For event time $k=0$, the estimated effect is 0.60 percentage points with $SE_0=0.18$, and a defensible bound on the change in the untreated gap is $M_0=0.20$ percentage points. For $k=1$, the estimate is 0.35 percentage points, $SE_1=0.20$, and $M_1=0.30$. Use pointwise 95% normal intervals with $c=1.96$.

For $k=0$, the sampling margin is $1.96(0.18)=0.3528$ percentage points. The sensitivity interval is $[0.60-0.3528-0.20,\ 0.60+0.3528+0.20]=[0.0472,\ 1.1528]$ percentage points. It remains above zero for this specified bound.

For $k=1$, the sampling margin is $1.96(0.20)=0.392$ percentage points. The interval is $[0.35-0.392-0.30,\ 0.35+0.392+0.30]=[-0.342,\ 1.042]$ percentage points. It includes zero. These are pointwise conclusions: the first period’s interval excludes zero under its bound, while the second does not. The calculation assumes that the supplied standard errors support the stated pointwise inference and that each bound plausibly covers the untreated-gap change from the reference period. The bounds are sensitivity choices, not quantities proven by the event-study estimates.

Choosing Bounds and Finding a Tipping Threshold

A sensitivity analysis is informative only if readers can judge the bound. State it in outcome units and explain its substantive basis. One transparent approach is to report results over a range of values, including a benchmark tied to the scale of pre-treatment changes. For example, a researcher might ask what happens if the post-period untreated-gap drift is no greater than the largest observed pre-period change, or no greater than a specified multiple of it. Such a benchmark is a calibration choice, not a guarantee: pre-period data are noisy, and stable-looking pre-trends do not prove that post-period untreated trends would remain parallel.

For a positive estimate, the sensitivity interval excludes zero only when its lower endpoint is strictly positive. Solving $\hat{\tau}-cSE-M>0$ gives the boundary $M=\hat{\tau}-cSE$, when that quantity is positive. This boundary is the tipping threshold (or supremum): every bound strictly below it leaves the interval above zero, while at the boundary the interval touches zero and no longer excludes it. For a negative estimate, the analogous boundary is $M=-\hat{\tau}-cSE$, provided it is positive. These thresholds summarize robustness to a specified class of biases; they do not select which bias is plausible.

Worked Example: Calculating the Tipping Threshold

A hypothetical program evaluation estimates an employment effect of 0.80 percentage points, with standard error 0.25 percentage points. Use a 95% normal critical value of 1.96 and consider symmetric bias bounds $M$.

The sampling margin is $1.96(0.25)=0.49$ percentage points. With no sensitivity adjustment, the lower endpoint is $0.80-0.49=0.31$ percentage points. For a general bound, the lower endpoint is $0.80-0.49-M=0.31-M$. It is strictly positive exactly when $M<0.31$.

Thus the tipping threshold is 0.31 percentage points. Every bound strictly below 0.31 leaves the 95% interval excluding zero. At $M=0.31$, the interval is $[0,\ 1.60]$ percentage points and includes zero; for larger bounds, it also includes zero. The threshold is not a largest bound that preserves exclusion, because exclusion fails at the boundary itself. It is the boundary between bounds that preserve exclusion and bounds that do not.

Why the Distinction Matters

Sensitivity analysis clarifies what a causal conclusion depends on. “The estimate is positive under parallel trends” and “the interval remains positive for any violation up to 0.2 outcome units” are different statements, and both are more informative when the units and uncertainty are explicit. Reporting intervals across several plausible bounds also shows whether the conclusion changes gradually or only after a substantial departure.

Do not treat pre-trend diagnostics as proof. A failure to detect pre-period differences may reflect limited precision; it does not establish that future untreated trends would match.
Do not call a bias bound an estimated bias. Unless the design supplies additional identifying information, the bound is a sensitivity assumption about an unobserved counterfactual departure.
Do not adjust only the point estimate. A bounded-bias interval widens the sampling interval to account for the range of possible bias; reporting only $\hat{\tau}\pm M$ omits sampling uncertainty.
Do not overstate coverage. The stated confidence level relies on valid sampling inference and the chosen bound containing the true bias. If the bound is implausible or the standard error is unsuitable, the interval does not fix those problems.

For an event study, report the reference period, the effect and standard error at each event time, the bound used for each period, and whether intervals are pointwise or simultaneous. This makes the sensitivity analysis reproducible and keeps the interpretation tied to the relevant event-time estimand.

Check Your Understanding

Answer each question using the bounded-bias framework.

A DiD estimate is 2.0 outcome units, its standard error is 0.4, the critical value is 1.96, and the symmetric bias bound is 0.5. Calculate the sensitivity interval.
In the model $\hat{\tau}=\tau+b+e$, why does a bound $b\in[L,U]$ produce a lower endpoint involving $U$ and an upper endpoint involving $L$?
What does an event-time bound $M_k$ describe relative to the omitted reference period?
A positive estimate has a sampling lower endpoint of 0.12 before adding a symmetric bias bound. State the tipping threshold and whether the interval excludes zero at the threshold.
Why does a nonsignificant pre-trend diagnostic not establish that the parallel-trends assumption holds after treatment?