Deriving The First-Difference Estimator

Why Difference a Panel Model?

A static panel model allows each unit to have an unobserved, time-constant component that may be related to its regressors. If that component is left in a pooled regression, it can confound the slope estimates. The fixed-effects within estimator removes it by subtracting unit means. First differencing removes the same component by a different transformation: subtract the equation for one period from the equation for the adjacent period.

The subtraction is simple, but its implications are important. It determines which variation identifies the slope, changes the error term, and requires an exogeneity condition on changes in regressors and errors. This tutorial derives the first-difference (FD) estimator and shows the conditions under which it consistently estimates the slope.

From the Panel Model to First Differences

For units \(i=1,\ldots,n\) observed in periods \(t=1,\ldots,T\), consider the static linear panel model

y_{it}=\alpha_i+x_{it}'\beta+u_{it}.

Here \(y_{it}\) is the outcome, \(x_{it}\) is a \(K\)-vector of regressors, \(\beta\) is a \(K\)-vector of slopes, \(\alpha_i\) is an unrestricted unit-specific effect that does not vary over time, and \(u_{it}\) is the time-varying error. The regressors may be related to \(\alpha_i\); removing that effect is one reason to use a transformation rather than pooled OLS.

For \(t=2,\ldots,T\), define a first difference by \(\Delta y_{it}=y_{it}-y_{i,t-1}\), \(\Delta x_{it}=x_{it}-x_{i,t-1}\), and \(\Delta u_{it}=u_{it}-u_{i,t-1}\). Subtracting the equation at \(t-1\) from the equation at \(t\) gives

\begin{aligned} y_{it}-y_{i,t-1} &=(\alpha_i-\alpha_i)+(x_{it}-x_{i,t-1})'\beta+(u_{it}-u_{i,t-1}),\\ \Delta y_{it}&=\Delta x_{it}'\beta+\Delta u_{it}. \end{aligned}

The unit effect cancels exactly. The resulting regression has \(n(T-1)\) differenced observations when the panel is balanced, and it has no separate unit intercept. It uses changes within each unit rather than differences in levels across units.

Definition: The first-difference estimator is the OLS estimator obtained by regressing \(\Delta y_{it}\) on \(\Delta x_{it}\), using the differenced equations for \(t=2,\ldots,T\). It estimates the slope vector in the original static panel model after eliminating the time-invariant unit effect.

Worked Example: Cancelling a Unit Effect

Consider one hypothetical unit with \(\alpha_i=5\), slope \(\beta=2\), and regressor values \(x_{i1}=1\), \(x_{i2}=3\), and \(x_{i3}=4\). Suppose the errors are \(u_{i1}=0.5\), \(u_{i2}=-0.5\), and \(u_{i3}=1\). The model implies outcomes \(y_{it}=5+2x_{it}+u_{it}\).

The levels are \(y_{i1}=5+2(1)+0.5=7.5\), \(y_{i2}=5+2(3)-0.5=10.5\), and \(y_{i3}=5+2(4)+1=14\). For the second period,

\Delta y_{i2}=10.5-7.5=3,\qquad \Delta x_{i2}=3-1=2,\qquad \Delta u_{i2}=-0.5-0.5=-1.

The differenced equation checks: \(\Delta y_{i2}=2(2)+(-1)=3\). For the third period, \(\Delta y_{i3}=14-10.5=3.5\), \(\Delta x_{i3}=4-3=1\), and \(\Delta u_{i3}=1-(-0.5)=1.5\); again, \(3.5=2(1)+1.5\). In both equations the value \(\alpha_i=5\) disappears.

The FD OLS Estimator

Stack the differenced observations over units and periods. If the differenced regressor matrix has full column rank, minimizing the sum of squared differenced residuals yields the OLS estimator

\widehat{\beta}_{FD} = \left(\sum_{i=1}^{n}\sum_{t=2}^{T}\Delta x_{it}\Delta x_{it}'\right)^{-1} \left(\sum_{i=1}^{n}\sum_{t=2}^{T}\Delta x_{it}\Delta y_{it}\right).

For one regressor, this becomes the sum of cross-products of changes divided by the sum of squared regressor changes. There is no intercept in this regression when the original model contains only the unit effect \(\alpha_i\) and the regressors shown. A common time effect or an additional time-varying control would require its own treatment in the model; differencing does not automatically remove every omitted component.

Worked Example: Computing a Differenced Slope

Suppose three hypothetical units are each observed in two periods. Their changes are recorded as follows. The outcome is measured in points and the regressor in hours of training.

Unit	\(\Delta x_i\) (hours)	\(\Delta y_i\) (points)
1	1	3
2	2	5
3	-1	-2

For this one-regressor model, the FD estimate is

\widehat{\beta}_{FD} =\frac{\sum_i\Delta x_i\Delta y_i}{\sum_i(\Delta x_i)^2} =\frac{(1)(3)+(2)(5)+(-1)(-2)}{1^2+2^2+(-1)^2} =\frac{15}{6}=2.5.

The estimated slope is 2.5 outcome points per additional hour of training change. It describes the fitted relationship between within-unit changes, not the cross-sectional relationship between units' levels. For instance, the first unit's fitted outcome change is \(2.5(1)=2.5\) points, and its differenced residual is \(3-2.5=0.5\) points.

Conditions for Consistency

Removing \(\alpha_i\) is not enough by itself to make the slope estimate consistent. The differenced regressors must contain identifying variation, and their variation must be appropriately unrelated to the differenced errors. A standard sufficient exogeneity assumption for the static panel model is strict exogeneity conditional on the unit effect:

E\!\left[u_{it}\mid x_{i1},\ldots,x_{iT},\alpha_i\right]=0 \quad\text{for every }i,t.

This condition allows the entire path of regressors for unit \(i\) to be related to \(\alpha_i\), but requires the error in every period to have conditional mean zero given that path and the unit effect. It implies the orthogonality needed after differencing. In particular, because both \(x_{it}\) and \(x_{i,t-1}\) are included in that path,

E\!\left[\Delta x_{it}\Delta u_{it}\right] = E\!\left[(x_{it}-x_{i,t-1})(u_{it}-u_{i,t-1})\right]=0.

Under this condition, independent sampling across units (or suitable weak dependence across units), finite moments, and a nonsingular limiting second-moment matrix for \(\Delta x_{it}\), the FD OLS estimator is consistent as the number of units grows with \(T\) fixed. The rank condition means that no nonzero linear combination of the regressors has zero variation in the differenced data. If a regressor does not change over time, it contributes no identifying variation and its slope cannot be estimated from first differences.

Transform the model.
Subtract each unit's equation in period \(t-1\) from its equation in period \(t\), giving \(\Delta y_{it}=\Delta x_{it}'\beta+\Delta u_{it}\).

Check identification.
Verify that changes in the regressors provide full-rank variation; level variation alone is not enough.

Check exogeneity.
Require orthogonality between regressor changes and error changes. Strict exogeneity in levels is a standard sufficient condition.

Estimate and interpret.
Apply OLS to the differenced equations and interpret each coefficient as the slope relating changes in the regressor to changes in the outcome.

Worked Example: Why Contemporaneous Exogeneity Can Fail

Consider a hypothetical single-regressor panel in which \(u_{it}\) is independent over time, has mean zero and variance \(\sigma^2\), and the regressor responds to the previous period's shock: \(x_{it}=u_{i,t-1}\). This setup makes the current regressor uncorrelated with the current error, since \(E[x_{it}u_{it}]=E[u_{i,t-1}u_{it}]=0\). But strict exogeneity fails: a future regressor \(x_{i,t+1}=u_{it}\) contains the current error.

The differenced variables are \(\Delta x_{it}=u_{i,t-1}-u_{i,t-2}\) and \(\Delta u_{it}=u_{it}-u_{i,t-1}\). Independence over time gives

\begin{aligned} E[\Delta x_{it}\Delta u_{it}] &=E[(u_{i,t-1}-u_{i,t-2})(u_{it}-u_{i,t-1})]\\ &=-E[u_{i,t-1}^{2}] =-\sigma^2. \end{aligned}

The covariance is not zero, even though the current regressor and current error are uncorrelated. The differenced regression therefore violates the required orthogonality condition. In this example, \(E[(\Delta x_{it})^2]=2\sigma^2\), so the ratio of the error-regressor cross-moment to the regressor second moment is \(-\sigma^2/(2\sigma^2)=-0.5\). With the stated exogeneity failure, this component shifts the large-sample slope away from \(\beta\) by \(-0.5\). The example shows why checking only contemporaneous level exogeneity is insufficient.

What Differencing Removes—and What It Does Not

Any regressor that is constant over time for a unit also disappears: if \(z_{it}=z_i\) for all \(t\), then \(\Delta z_{it}=0\). Its separate slope cannot be identified by FD estimation. This is the same basic limitation as in fixed-effects estimation: the transformation removes unit-level constants along with \(\alpha_i\).

First differences also change the error structure. Even if \(u_{it}\) is serially uncorrelated, adjacent differenced errors share a level error with opposite signs. For example, \(\Delta u_{it}=u_{it}-u_{i,t-1}\) and \(\Delta u_{i,t+1}=u_{i,t+1}-u_{it}\) both contain \(u_{it}\). Thus one should not assume differenced errors are serially uncorrelated merely because the level errors are. This matters for standard errors and inference, though it does not alter the algebraic FD point estimate.

The transformation eliminates time-invariant unit heterogeneity; it does not eliminate time-varying omitted variables or repair endogeneity. A regressor may vary over time and still be correlated with the changing error. The maintained exogeneity assumptions must therefore fit the substantive timing of the panel process.

Common Mistake

Wrong reasoning: “The unit effect is gone after subtraction, so first-difference OLS must be consistent.”
Correction: Cancellation removes \(\alpha_i\), but consistency additionally requires identifying variation in \(\Delta x_{it}\) and orthogonality between \(\Delta x_{it}\) and \(\Delta u_{it}\). Strict exogeneity is a standard sufficient assumption; contemporaneous zero correlation alone need not suffice.

Key takeaway: First differencing changes \(y_{it}=\alpha_i+x_{it}'\beta+u_{it}\) into \(\Delta y_{it}=\Delta x_{it}'\beta+\Delta u_{it}\), exactly eliminating a time-invariant unit effect. The resulting OLS estimator is consistent only when the differenced regressors have full-rank variation and are exogenous to the differenced errors.

Check Your Understanding

Answer each question using the definitions and conditions developed above.

Starting from the static panel model, show which term cancels when the period-\(t-1\) equation is subtracted from the period-\(t\) equation.
For one regressor and differenced observations \((\Delta x,\Delta y)=(1,2),(2,3)\), calculate the FD OLS slope using the no-intercept formula.
Why can a time-invariant regressor's slope not be estimated from first differences?
State the strict exogeneity condition in terms of the full regressor path and explain why it implies orthogonality between regressor changes and error changes.
In the lagged-shock example, how can \(x_{it}\) be uncorrelated with \(u_{it}\) while \(\Delta x_{it}\) is correlated with \(\Delta u_{it}\)?