Why Differencing a Stationary Series Changes Its Dependence
Differencing is designed to remove changing levels or trends. If a series is already stationary, however, a difference is not a neutral transformation: it subtracts adjacent observations, which are generally related to one another. Even when the original series is white noise, the differenced values share an observation and therefore become dependent.
The central case is especially useful. Differencing white noise produces an MA(1) process whose MA coefficient is \(-1\), exactly on the boundary of invertibility. More generally, differencing an already stationary ARMA process adds a moving-average factor \(1-B\), where \(B\) is the lag operator. This is the basic mechanism behind overdifferencing.
In this tutorial, an MA(1) is written \(Y_t=\varepsilon_t+\theta\varepsilon_{t-1}\), where \(\varepsilon_t\) is white noise. Under this convention, its MA polynomial is \(1+\theta B\), and invertibility requires its zero to lie outside the unit circle. In particular, an MA(1) is invertible when \(|\theta|<1\); \(\theta=-1\) is a boundary case, not an invertible case.
The General Covariance Effect of a Difference
If \(X_t\) is weakly stationary, then its difference is also weakly stationary. Its mean is zero, since the mean of \(X_t-X_{t-1}\) is the difference of two equal means. The covariance at lag \(k\) follows by expanding the two differences:
At lag zero, this gives \(\operatorname{Var}(\Delta X_t)=2\{\gamma_X(0)-\gamma_X(1)\}\). The formula makes the key point precise: differencing changes dependence according to the covariances between adjacent observations. It does not, in general, turn every stationary process into a finite-order MA process. Rather, when an ARMA representation is available, the differencing operator contributes an MA factor.
Use \(\Delta X_t=X_t-X_{t-1}\), keeping both terms in each covariance.
There are four covariance terms; stationarity expresses each using \(\gamma_X\) at its corresponding lag.
The difference has covariance determined by neighboring covariances of the original series; for white noise, only the shared observation creates nonzero lag-one covariance.
Why White Noise Becomes a Boundary MA(1)
Suppose \(X_t=\mu+\varepsilon_t\), where \(\varepsilon_t\) is independent white noise with mean zero and variance \(\sigma^2\). Then
This is an MA(1) with \(\theta=-1\). Its variance is \(2\sigma^2\), because the two white-noise terms are independent. Its lag-one covariance is \(-\sigma^2\), because \(\Delta X_t\) and \(\Delta X_{t-1}\) contain \(-\varepsilon_{t-1}\) and \(+\varepsilon_{t-1}\), respectively. At lags greater than one, the differences share no white-noise term, so their covariance is zero. The lag-one correlation is therefore \(-1/2\).
The MA polynomial is \(1-B\), whose zero is \(B=1\), on the unit circle. Thus this representation is noninvertible at the boundary. Informally, recovering the original white-noise sequence from the differences would require undoing \(1-B\), which cannot be done with a stable, convergent inverse filter at this unit root.
Worked Example: Differencing a White-Noise Level
A hypothetical monthly index is \(X_t=40+\varepsilon_t\), where the disturbances are independent with \(\varepsilon_t\sim N(0,9)\). Find the model, mean, variance, lag-one correlation, and invertibility status of the first difference.
Method. Subtract adjacent observations, then use independence of the disturbances. The common mistake is to treat consecutive differences as independent just because the original disturbances are independent.
Difference and model. \(\Delta X_t=(40+\varepsilon_t)-(40+\varepsilon_{t-1})=\varepsilon_t-\varepsilon_{t-1}\). In the stated MA convention, this is MA(1) with \(\theta=-1\).
Mean and variance. The mean is \(0-0=0\). Independence gives \(\operatorname{Var}(\Delta X_t)=9+9=18\).
Lag-one covariance and correlation. The only shared disturbance in consecutive differences is \(\varepsilon_{t-1}\), with opposite signs, so \(\operatorname{Cov}(\Delta X_t,\Delta X_{t-1})=-9\). Hence the correlation is \(-9/18=-0.5\). Covariances at lags greater than one are zero.
Check. The variance and covariance give \(-9/\sqrt{18\cdot18}=-0.5\), as calculated. The MA polynomial \(1-B\) has root \(1\), so this is the noninvertible boundary case.
Stationary ARMA Series: The Added MA Factor
The same idea applies beyond white noise. Suppose a stationary ARMA series has the lag-operator equation \(\phi(B)X_t=\theta(B)\varepsilon_t\), with a stable stationary AR part. Differencing both sides gives
The added factor \(1-B\) is an MA factor with a unit-circle zero. This identifies the structural effect of unnecessary differencing. Depending on the original ARMA orders and whether any factors cancel, the differenced series need not be a pure MA model. In the absence of cancellation, the added factor makes the resulting MA representation noninvertible at the boundary.
Worked Example: Differencing a Stationary AR(1)
Let \(X_t=0.6X_{t-1}+\varepsilon_t\), where \(\varepsilon_t\) is independent white noise with mean zero and variance \(1\). Since \(|0.6|<1\), \(X_t\) is stationary. Find the equation for \(Y_t=\Delta X_t\), its variance and lag-one correlation, and the status of its MA factor.
Method. Apply the difference operator to the AR equation and use the stationary AR(1) autocovariances to calculate the moments. The common mistake is to call \(Y_t\) a pure MA(1); the differenced equation still has an AR term.
Lag-operator equation. The original model is \((1-0.6B)X_t=\varepsilon_t\). Since \(Y_t=(1-B)X_t\), applying \(1-B\) to both sides gives \((1-0.6B)Y_t=(1-B)\varepsilon_t\). Thus \(Y_t\) has an ARMA representation with an AR(1) factor and the added MA factor \(1-B\), whose coefficient is \(-1\).
Original autocovariances. For a stationary AR(1), \(\gamma_X(0)=1/(1-0.6^2)=1/0.64=25/16\). Therefore \(\gamma_X(1)=0.6(25/16)=15/16\), and \(\gamma_X(2)=0.6^2(25/16)=9/16\).
Variance of the difference. \(\gamma_Y(0)=2\{\gamma_X(0)-\gamma_X(1)\}=2(25/16-15/16)=20/16=1.25\).
Lag-one covariance and correlation. \(\gamma_Y(1)=2\gamma_X(1)-\gamma_X(0)-\gamma_X(2)=30/16-25/16-9/16=-4/16=-0.25\). Thus \(\rho_Y(1)=-0.25/1.25=-0.2\).
Check. The variance is positive, and the calculated correlation is the covariance divided by the variance because both variables in the covariance have variance \(1.25\). The MA factor \(1-B\) has a root at \(1\), so the differenced representation is on the noninvertibility boundary.
Repeated Differencing and the Boundary Pattern
If a stationary series is differenced twice, the added factor is \((1-B)^2\). For white noise, the result is an MA(2) with coefficients \(1,-2,1\). The repeated unit root remains on the unit circle. This is not evidence that a second difference was needed; it is the algebraic consequence of applying the difference operator again.
Worked Example: A Second Difference of White Noise
Let \(X_t=\varepsilon_t\), where the disturbances are independent with mean zero and variance \(2\). Calculate the variance, the covariances and correlations at lags one and two, and the MA polynomial of \(W_t=\Delta^2X_t\).
Method. Expand the second difference into white-noise terms, then multiply only coefficients attached to shared disturbances. The common mistake is to assume that a second difference removes more dependence; here it creates additional overlapping terms.
Expansion. \(W_t=\Delta(\Delta X_t)=X_t-2X_{t-1}+X_{t-2}=\varepsilon_t-2\varepsilon_{t-1}+\varepsilon_{t-2}\). The MA polynomial is \(1-2B+B^2=(1-B)^2\).
Variance. The three disturbances are independent, so \(\operatorname{Var}(W_t)=(1^2+(-2)^2+1^2)2=(1+4+1)2=12\).
Lag-one covariance. \(W_t\) and \(W_{t-1}=\varepsilon_{t-1}-2\varepsilon_{t-2}+\varepsilon_{t-3}\) share two disturbances. Their products of coefficients are \((-2)(1)=-2\) for \(\varepsilon_{t-1}\) and \((1)(-2)=-2\) for \(\varepsilon_{t-2}\). Thus the covariance is \((-2-2)2=-8\), and the correlation is \(-8/12=-2/3\).
Lag-two covariance. The only shared disturbance is \(\varepsilon_{t-2}\), whose coefficient is \(1\) in both terms. The covariance is \(1\cdot1\cdot2=2\), giving correlation \(2/12=1/6\). At greater lags there are no shared disturbances, so the covariance is zero.
Check. The MA polynomial \((1-B)^2\) has a repeated root at \(B=1\). Both the first and second differences are stationary here, but neither extra difference was needed to make the original white noise stationary.
Exam Traps and Practical Interpretation
An unnecessary difference can make a model appear to need MA structure, even though that structure was introduced by the transformation rather than by the original process. It can also put an MA root at, or near, the noninvertibility boundary. A fitted model may then have difficulty distinguishing a genuine near-boundary MA effect from an excessive difference. The algebra explains the risk; whether differencing is warranted must be decided from the series and its modeling assumptions.
- Trap: assuming differenced white noise is still white noise. Consecutive differences share one disturbance with opposite signs, so their lag-one covariance is negative.
- Trap: reporting \(\theta=1\) for \(\varepsilon_t-\varepsilon_{t-1}\). Under \(Y_t=\varepsilon_t+\theta\varepsilon_{t-1}\), the coefficient is \(\theta=-1\).
- Trap: calling an AR(1) series differenced into a pure MA(1). Its equation retains the AR factor: \((1-\phi B)\Delta X_t=(1-B)\varepsilon_t\).
- Trap: calling a unit-circle root invertible. Invertibility requires MA polynomial roots strictly outside the unit circle; a root at \(1\) is on the boundary.
Check Your Understanding
For each problem, use the MA convention \(Y_t=\varepsilon_t+\theta\varepsilon_{t-1}\) when applicable.
- Let \(X_t=12+\varepsilon_t\), with independent \(\varepsilon_t\) of mean zero and variance \(3\). For \(Y_t=\Delta X_t\), find the mean, variance, lag-one covariance, and lag-one correlation. (Answer: mean \(0\), variance \(6\), lag-one covariance \(-3\), correlation \(-0.5000\).)
- A stationary AR(1) satisfies \(X_t=0.5X_{t-1}+\varepsilon_t\), with independent white-noise innovations of variance \(4\). For \(Y_t=\Delta X_t\), find its variance and lag-one correlation. (Answer: variance \(16/3\), lag-one correlation \(-0.2500\).)
- Let \(X_t\) be independent white noise of variance \(5\), and set \(W_t=\Delta^2X_t\). Find \(\operatorname{Var}(W_t)\), \(\operatorname{Cov}(W_t,W_{t-1})\), and \(\operatorname{Cov}(W_t,W_{t-2})\). (Answer: \(30,-20,5\), respectively.)
- A weakly stationary zero-mean series has \(\gamma_X(0)=10\), \(\gamma_X(1)=6\), and \(\gamma_X(2)=2\). Find the variance and lag-one covariance of \(\Delta X_t\). (Answer: variance \(8\), lag-one covariance \(0\).)
- A stationary AR(1) has \(X_t=-0.4X_{t-1}+\varepsilon_t\), with independent innovations of variance \(3\). For \(Y_t=\Delta X_t\), state the added MA factor and find \(\operatorname{Var}(Y_t)\). (Answer: added factor \(1-B\), with MA coefficient \(-1\) and root \(1\); variance \(10\).)