Why the Same Difference Can Mean Different Things
A time series can change over time for different reasons. It may fluctuate around a constant level, follow a deterministic trend, or accumulate random shocks as a random walk. In each case, first differencing subtracts one observation from the previous one. But the resulting series has a different interpretation depending on how the original series was generated.
The key exam skill is not just calculating differences. It is identifying what they remove: a constant level disappears, a deterministic linear trend becomes a constant mean, and a random walk becomes its increments. A second difference repeats the subtraction, so it can remove additional structure. The algebra is simple; the modeling interpretation requires care.
A weakly stationary series has a constant finite mean, a finite variance that does not change over time, and a covariance between two observations that depends only on their time separation. Differencing is often used when the level of a series is nonstationary, but the fact that a difference can be calculated does not by itself show that differencing was needed.
Constant Levels and Fluctuations Around a Level
A fixed level \(Y_t=c\) has no change from one time to the next, so \(\Delta Y_t=0\). If a series fluctuates around a fixed level, write \(Y_t=\alpha+X_t\), where \(\alpha\) is constant and \(X_t\) is weakly stationary. Then \(\Delta Y_t=X_t-X_{t-1}\). A constant shift does not affect stationarity, and a difference of a weakly stationary series is also weakly stationary. Thus, differencing removes the fixed level, but it is not generally needed to make a stationary series stationary.
For example, if \(X_t\) is independent and identically distributed with mean zero and variance \(\sigma^2\), then \(\Delta Y_t=X_t-X_{t-1}\) has mean zero and variance \(2\sigma^2\). Adjacent differences share an observation, so they need not be independent. The important point here is that the differences are stationary; differencing a stationary level is not evidence that the original series had a trend.
Worked Example: A Fixed Level With Random Fluctuations
A hypothetical monthly index follows \(Y_t=50+X_t\), where \(X_t\) are independent observations from \(N(0,4)\). Thus, the fluctuation has mean 0 and variance 4, and the level shift is 50. For one possible realization, let \(X_t\) be \(1,-1,0,2,-2,1\), in that order. Find the levels, the first differences, and the second differences. Does the model require differencing to remove a trend?
Method. Add 50 to each fluctuation to get the observed level. Then subtract successive levels to get first differences, and subtract successive first differences to get second differences.
The levels are \(51,49,50,52,48,51\). For example, the first level is \(50+1=51\), and the fourth is \(50+2=52\).
The first differences are \(49-51=-2\), \(50-49=1\), \(52-50=2\), \(48-52=-4\), and \(51-48=3\). Thus, they are \(-2,1,2,-4,3\).
The second differences are \(1-(-2)=3\), \(2-1=1\), \(-4-2=-6\), and \(3-(-4)=7\). Thus, they are \(3,1,-6,7\).
The model has a constant mean \(E[Y_t]=50\) and constant variance \(\operatorname{Var}(Y_t)=4\), so the level is already stationary. Differencing removes the fixed 50, but it is not needed to remove a trend. The calculated values are checked directly against the level sequence.
Common mistake avoided: Treating any visible movement in a short sequence as a trend. Random fluctuations around a constant mean are compatible with a stationary-level model.
Deterministic Linear Trend
A deterministic linear trend with stationary fluctuations can be written \(Y_t=\alpha+\beta t+X_t\), where \(\alpha\) and \(\beta\) are fixed constants and \(X_t\) is weakly stationary. The expected level is \(\alpha+\beta t+E[X_t]\), which changes with \(t\) if \(\beta\ne0\). Its first difference is \(\beta+\Delta X_t\). The linearly changing part becomes the constant \(\beta\); the remaining variation comes from differences of the stationary component.
Here is a direct justification. Let \(E[X_t]=m\) for every \(t\). Then \(E[\Delta X_t]=m-m=0\). If \(\gamma(h)=\operatorname{Cov}(X_t,X_{t-h})\) is the stationary autocovariance function, then
and the covariance between \(\Delta X_t\) and \(\Delta X_{t-h}\) can be written using \(\gamma\) at lags \(h\), \(h-1\), and \(h+1\). It therefore depends on \(h\), not on the calendar time \(t\). Adding the constant \(\beta\) changes the mean of the differenced series but not its variance or covariance structure. This is why the first difference of a deterministic linear trend with stationary fluctuations is stationary.
Worked Example: Differencing a Linear Trend With Observed Deviations
A hypothetical quarterly workload is modeled as \(Y_t=10+2t+X_t\), with \(t=1,\ldots,6\). The deterministic slope is 2 units per quarter. Assume \(X_t\) is a weakly stationary, mean-zero process; the observed deviations are \(0,1,-1,2,0,-2\). Calculate the levels and their first and second differences, and interpret the first difference.
Method. First calculate the trend component \(10+2t\), then add the stated deviation for each quarter.
For \(t=1,\ldots,6\), the trend components are \(12,14,16,18,20,22\). Adding the deviations gives levels \(12,15,15,20,20,20\).
Subtracting consecutive levels gives the first differences: \(15-12=3\), \(15-15=0\), \(20-15=5\), \(20-20=0\), and \(20-20=0\). Hence \(\Delta Y_t\) is \(3,0,5,0,0\).
Subtracting consecutive first differences gives the second differences: \(0-3=-3\), \(5-0=5\), \(0-5=-5\), and \(0-0=0\). Hence \(\Delta^2Y_t\) is \(-3,5,-5,0\).
Algebraically, the first difference is \(2+\Delta X_t\): the deterministic increase of 2 per quarter becomes a constant, while the deviations contribute their changes. The observed first differences need not each equal 2 because \(X_t\) varies. Under the stated assumption, the first-differenced series is stationary with mean 2. The second difference removes that constant as well, leaving \(\Delta^2X_t\); it is not required to remove the linear trend.
The calculations check against the levels: for instance, the first difference from quarter 3 to quarter 4 is \(20-15=5\), and the matching second difference is \(5-0=5\). Common mistake avoided: Claiming that every first difference must equal the trend slope. That is true only when there are no fluctuations.
Random Walks and Stochastic Trends
A random walk with drift \(\mu\) is defined by \(Y_t=Y_{t-1}+\mu+\varepsilon_t\), where \(\varepsilon_t\) are independent and identically distributed with mean zero and variance \(\sigma^2\), and \(Y_0\) is fixed. Rearranging the defining equation gives \(\Delta Y_t=\mu+\varepsilon_t\). Thus, first differences remove the accumulated level and expose the drift plus the new random shock.
The mean and variance follow by repeatedly substituting the recursion: \(Y_t=Y_0+t\mu+\sum_{j=1}^{t}\varepsilon_j\). Taking expectations gives \(Y_0+t\mu\). Since the shocks are independent, the variance of their sum is the sum of their variances, \(t\sigma^2\). This derivation also shows why the random walk is a stochastic trend: shocks do not merely cause temporary deviations from a fixed path; each shock remains in all later levels.
Worked Example: Recovering Innovations From a Random Walk
A hypothetical quarterly index follows \(Y_t=Y_{t-1}+3+\varepsilon_t\), with \(Y_0=100\). Assume the shocks are independent and identically distributed as \(N(0,9)\), so their mean is zero and variance is 9. For the observed shocks \(2,-1,0,4,-2\), calculate the levels, first differences, and second differences. State the mean and variance of the level at time 5.
Method. Add the drift and the shock to the preceding level at each step. The increment is \(3+\varepsilon_t\), and each second difference is the change between consecutive increments.
Starting at \(Y_0=100\), the increments are \(3+2=5\), \(3-1=2\), \(3+0=3\), \(3+4=7\), and \(3-2=1\). Adding these successively gives \(Y_1=105\), \(Y_2=107\), \(Y_3=110\), \(Y_4=117\), and \(Y_5=118\).
The first differences are \(105-100=5\), \(107-105=2\), \(110-107=3\), \(117-110=7\), and \(118-117=1\), as required by \(3+\varepsilon_t\). The second differences are \(2-5=-3\), \(3-2=1\), \(7-3=4\), and \(1-7=-6\). They also equal the successive changes in the shocks: \(-1-2=-3\), \(0-(-1)=1\), \(4-0=4\), and \(-2-4=-6\).
At time 5, \(E[Y_5]=100+3(5)=115\), and \(\operatorname{Var}(Y_5)=5(9)=45\). The observed value 118 differs from the expected value because the observed shocks sum to \(2-1+0+4-2=3\), so \(100+15+3=118\). The model’s level variance grows over time because \(\sigma^2=9>0\); its mean changes because \(\mu=3\ne0\).
Common mistake avoided: Calling the random-walk mean time-varying without checking the drift. With zero drift, its mean is constant even though its variance still grows when the shock variance is positive.
Choosing the Interpretation, Not Just the Order
The three cases can be compared directly. For a fixed level, differencing yields zero; for stationary fluctuations around a fixed level, it yields stationary changes but may be unnecessary. For a deterministic linear trend with stationary deviations, the first difference converts the slope into a constant mean. For a random walk, the first difference is the drift plus innovations. These outcomes all follow from the same subtraction, but they describe different data-generating mechanisms.
| Original series | First difference | Interpretation |
|---|---|---|
| \(Y_t=c\) | 0 | The fixed level is removed. |
| \(Y_t=\alpha+X_t\), with \(X_t\) stationary | \(\Delta X_t\) | A stationary series is differenced; this is not needed to remove a trend. |
| \(Y_t=\alpha+\beta t+X_t\), with \(X_t\) stationary | \(\beta+\Delta X_t\) | The linear trend becomes a constant mean. |
| \(Y_t=Y_{t-1}+\mu+\varepsilon_t\) | \(\mu+\varepsilon_t\) | The accumulated stochastic level becomes increments. |
A second difference is the first difference of the first-differenced series. For a deterministic linear trend with stationary deviations, it removes the constant slope as well, giving \(\Delta^2X_t\). For a random walk, it gives \(\Delta\varepsilon_t\), the change in consecutive shocks. These calculations do not establish that a second difference is appropriate: the first difference may already have removed the nonstationarity of interest. Choosing a difference order requires interpreting the original model and the resulting series, not automatically differencing again.
- Check the model form. A linear time term and a lagged level in a recursion represent distinct trend mechanisms.
- Check what the mean does. A nonzero deterministic slope changes the expected level; a zero-drift random walk has a constant mean despite a growing variance.
- Check the variance and dependence. A random walk with positive shock variance has a time-varying level variance, even when its drift is zero.
- Check what the difference represents. Do not interpret the first differences as pure shocks when the model also includes drift; they equal drift plus shocks.
- Check whether more differencing is justified. A second difference removes more structure and is not automatically preferable.
Check Your Understanding
For each problem, calculate the requested differences and use the stated model to interpret them.
- A fixed sensor baseline is recorded as \(7,7,7,7,7\). Give the first and second differences, and state whether a trend has been removed. (Answer: First differences \(0,0,0,0\); second differences \(0,0,0\); no trend was present to remove.)
- A deterministic trend is \(Y_t=4-0.5t\), for \(t=1,2,3,4\), with no random component. List the levels and first differences. (Answer: Levels \(3.5,3,2.5,2\); first differences \(-0.5,-0.5,-0.5\), equal to the slope.)
- A series has observed levels \(40,43,41,46\). Calculate both difference orders. If it is modeled as a random walk with drift \(\mu=1\), calculate the corresponding innovations. (Answer: First differences \(3,-2,5\); second differences \(-5,7\); innovations \(2,-3,4\).)
- For \(Y_t=7+2t+3t^2\), calculate \(\Delta Y_t\) and \(\Delta^2Y_t\) as functions of \(t\). (Answer: \(\Delta Y_t=6t-1\); \(\Delta^2Y_t=6\).)
- A random walk has fixed \(Y_0=20\), drift \(\mu=0\), and independent innovations with mean zero and variance 4. State \(E[Y_t]\) and \(\operatorname{Var}(Y_t)\), and describe what first differencing yields. (Answer: \(E[Y_t]=20\), \(\operatorname{Var}(Y_t)=4t\); first differences are the independent innovations, with mean 0 and variance 4.)