How a Filter Changes Covariance
A linear filter forms each output value by adding weighted values of an input process at different times. Because the input values can be correlated across time, the output covariance depends not only on the filter weights but also on the input’s covariance at several lags. Keeping track of both time indices is the central step in the calculation.
Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((X_t)_{t\in\mathbb{Z}}\) be a real-valued process with state space \(S=\mathbb{R}\), finite second moments, and weak stationarity: its mean is a constant \(m\), and its covariance depends only on the lag. Use the convention
For a real-valued weakly stationary process, \(\gamma_X(-h)=\gamma_X(h)\). A filter with real coefficients \((a_j)_{j\ge0}\) produces the output \(Y_t=\sum_{j\ge0}a_jX_{t-j}\), when this sum is well-defined. We will assume \(\sum_{j\ge0}|a_j|<\infty\). This condition is sufficient for mean-square convergence when the input has finite variance; it also makes the covariance double sum below absolutely convergent.
The indices in the formula come directly from pairing \(X_{t+h-j}\), the input value in \(Y_{t+h}\), with \(X_{t-k}\), the input value in \(Y_t\). Their time difference is \(h-j+k\). The order of the two output times and this sign convention matter: changing the lag convention changes how the formula is written.
Proof. Put \(Z_t=X_t-m\). For \(r\ge0\), form the finite partial sum \(Y_t^{(r)}=\sum_{j=0}^{r}a_jX_{t-j}\). Since \(\mathbb{E}[Z_t^2]=\gamma_X(0)\), the triangle inequality in mean square gives, for \(s>r\), $$ \left(\mathbb{E}\left[\left|\sum_{j=r+1}^{s}a_jZ_{t-j}\right|^2\right]\right)^{1/2} \le \sqrt{\gamma_X(0)}\sum_{j=r+1}^{s}|a_j|. $$ The right side tends to zero as \(r\) tends to infinity, uniformly over \(s>r\). Thus the centered partial sums converge in mean square. The deterministic means \(m\sum_{j=0}^{r}a_j\) also converge, so \(Y_t^{(r)}\) converges in mean square to \(Y_t\), and \(\mathbb{E}[Y_t]=m\sum_{j\ge0}a_j\).
For finite partial sums, bilinearity of covariance and weak stationarity give $$ \operatorname{Cov}(Y_{t+h}^{(r)},Y_t^{(r)}) =\sum_{j=0}^{r}\sum_{k=0}^{r}a_ja_k\gamma_X(h-j+k). $$ The covariance satisfies \(|\gamma_X(u)|\le\gamma_X(0)\) for every integer \(u\), by the Cauchy–Schwarz inequality applied to \(Z_{t+u}\) and \(Z_t\). Hence $$ \sum_{j\ge0}\sum_{k\ge0}|a_ja_k\gamma_X(h-j+k)| \le \gamma_X(0)\left(\sum_{j\ge0}|a_j|\right)^2<\infty. $$ The finite double sums therefore converge to the stated double sum. Mean-square convergence also implies convergence of these covariances: for example, the difference between \(\operatorname{Cov}(Y_{t+h}^{(r)},Y_t^{(r)})\) and \(\operatorname{Cov}(Y_{t+h},Y_t)\) tends to zero by Cauchy–Schwarz and convergence in mean square. The resulting covariance depends on \(h\), not on \(t\); the mean is constant. These are precisely the conditions for weak stationarity. \(\square\)
White-Noise Input
A real-valued weak white-noise process \((\varepsilon_t)_{t\in\mathbb{Z}}\) has constant mean, variance \(\sigma^2\), and zero covariance between distinct times. Independence is not required for the covariance calculation. For this input, a term in the double sum vanishes unless the two input times match: \(\operatorname{Cov}(\varepsilon_{t+h-j},\varepsilon_{t-k})\ne0\) only when \(j-k=h\). Thus, for \(h\ge0\), the surviving pairs have \(j=k+h\).
Proof. In the double sum of the theorem, \(\gamma_X(h-j+k)\) equals \(\sigma^2\) when \(h-j+k=0\), and equals zero otherwise. For \(h\ge0\), the equality \(h-j+k=0\) is equivalent to \(j=k+h\). Substituting these surviving indices gives \(\sigma^2\sum_{k\ge0}a_{k+h}a_k\). A finite filter has \(a_j=0\) for \(j>q\), so there are no surviving terms when \(h>q\), and for \(0\le h\le q\) the sum ends at \(k=q-h\). Finally, covariance symmetry gives \(\gamma_Y(-h)=\gamma_Y(h)\). \(\square\)
This finite-filter formula makes the overlap of coefficients visible. At lag zero, each coefficient pairs with itself, giving the output variance. At a positive lag \(h\), each coefficient pairs with the coefficient \(h\) positions away. A finite filter of length \(q+1\) therefore has zero output covariance at lags whose absolute value exceeds \(q\), when its input is white noise.
Worked Example: A Two-Term Filter of White Noise
Let \((\varepsilon_t)\) be mean-zero weak white noise with variance \(\sigma^2=3\), and define \(Y_t=\varepsilon_t+\tfrac12\varepsilon_{t-1}\). The coefficients are \(a_0=1\) and \(a_1=\tfrac12\); all later coefficients are zero. The white-noise formula gives
At lag one, only \(a_1a_0\) contributes, so $$ \gamma_Y(1)=3\left(\tfrac12\cdot1\right)=\tfrac32. $$ At lag two or greater, there are no coefficient pairs separated by that lag, so \(\gamma_Y(h)=0\) for \(h\ge2\). By covariance symmetry, \(\gamma_Y(-1)=\tfrac32\) and \(\gamma_Y(h)=0\) for \(h\le-2\).
Check by expansion. The output variance is the sum of the two individual variances and twice their covariance: \(\operatorname{Var}(\varepsilon_t+\tfrac12\varepsilon_{t-1})=3+\tfrac14(3)+2(\tfrac12)(0)=\tfrac{15}{4}\). For lag one, expanding \(\operatorname{Cov}(Y_{t+1},Y_t)\) leaves just \(\tfrac12\operatorname{Cov}(\varepsilon_t,\varepsilon_t)=\tfrac12(3)=\tfrac32\); all other pairs are at distinct times and have zero covariance. Both checks agree with the formula.
Worked Example: Filtering a Geometric Covariance
Suppose \((X_t)\) has mean zero and autocovariance \(\gamma_X(h)=(\tfrac23)^{|h|}\). This is the standard geometric covariance sequence. Apply \(Y_t=X_t+\tfrac12X_{t-1}\). In the double sum, the four coefficient pairs give $$ \gamma_Y(h)=\gamma_X(h)+\tfrac12\gamma_X(h+1) +\tfrac12\gamma_X(h-1)+\tfrac14\gamma_X(h) =\tfrac54\gamma_X(h)+\tfrac12\{\gamma_X(h+1)+\gamma_X(h-1)\}. $$
At lag zero, \(\gamma_X(0)=1\) and \(\gamma_X(1)=\gamma_X(-1)=\tfrac23\), so $$ \gamma_Y(0)=\tfrac54+\tfrac12\left(\tfrac23+\tfrac23\right) =\tfrac54+\tfrac23=\tfrac{15}{12}+\tfrac{8}{12}=\tfrac{23}{12}. $$ At lag one, \(\gamma_X(1)=\tfrac23\), \(\gamma_X(2)=\tfrac49\), and \(\gamma_X(0)=1\). Therefore $$ \gamma_Y(1)=\tfrac54\cdot\tfrac23+\tfrac12\left(\tfrac49+1\right) =\tfrac56+\tfrac{13}{18}=\tfrac{15}{18}+\tfrac{13}{18}=\tfrac{14}{9}. $$ At lag two, the required input covariances are \(\gamma_X(2)=\tfrac49\), \(\gamma_X(3)=\tfrac8{27}\), and \(\gamma_X(1)=\tfrac23\), giving $$ \gamma_Y(2)=\tfrac54\cdot\tfrac49+\tfrac12\left(\tfrac8{27}+\tfrac23\right) =\tfrac59+\tfrac{13}{27}=\tfrac{15}{27}+\tfrac{13}{27}=\tfrac{28}{27}. $$
Check at lag one by direct expansion. Write \(Y_{t+1}=X_{t+1}+\tfrac12X_t\) and \(Y_t=X_t+\tfrac12X_{t-1}\). The four covariance terms are \(\tfrac23\), \(\tfrac12\cdot\tfrac49=\tfrac29\), \(\tfrac12\cdot1=\tfrac12\), and \(\tfrac14\cdot\tfrac23=\tfrac16\). Their sum is \(\tfrac{12}{18}+\tfrac4{18}+\tfrac9{18}+\tfrac3{18}=\tfrac{28}{18}=\tfrac{14}{9}\), as obtained above.
Worked Example: A Three-Term Filter of White Noise
Let \((\varepsilon_t)\) be mean-zero weak white noise with variance \(2\), and set \(Y_t=\varepsilon_t+\tfrac12\varepsilon_{t-1}-\tfrac12\varepsilon_{t-2}\). The coefficient list is \(a_0=1\), \(a_1=\tfrac12\), \(a_2=-\tfrac12\). At lag zero, the sum of squared coefficients is \(1+\tfrac14+\tfrac14=\tfrac32\), so \(\gamma_Y(0)=2\cdot\tfrac32=3\).
At lag one, the coefficient products are \(a_1a_0=\tfrac12\) and \(a_2a_1=-\tfrac14\). Their sum is \(\tfrac14\), so \(\gamma_Y(1)=2\cdot\tfrac14=\tfrac12\). At lag two, the only product is \(a_2a_0=-\tfrac12\), so \(\gamma_Y(2)=2(-\tfrac12)=-1\). There are no coefficient pairs at larger lags. Thus \(\gamma_Y(h)=0\) for \(h\ge3\), and negative lags follow by symmetry.
Check. Expanding \(Y_t^2\) in the variance calculation, all cross-time covariances vanish; the remaining variance terms are \(2+\tfrac14(2)+\tfrac14(2)=3\). For lag one, only the matching-time products contribute: the coefficient pairs have total \(\tfrac12-\tfrac14=\tfrac14\), multiplied by the white-noise variance \(2\), giving \(\tfrac12\). For lag two, the one matching-time product is \(-\tfrac12\), giving \(-1\) after multiplication by \(2\). These checks reproduce all three nonzero lag values.
Common Pitfall: Losing a Cross-Term
When calculating an output variance, it is incorrect to add only the variances of the filtered input terms and ignore their covariance. For the two-term filter in the first example, that mistake gives \(3+\tfrac14(3)=\tfrac{15}{4}\) only because the distinct white-noise terms are uncorrelated. If the same filter is applied to the geometric-covariance input in the second example, omitting the cross-terms would give \(1+\tfrac14=\tfrac54\), not the correct \(\tfrac{23}{12}\). The missing contribution is \(2(\tfrac12)\gamma_X(1)=\tfrac23\), and \(\tfrac54+\tfrac23=\tfrac{23}{12}\).
A related indexing error is to pair the output at \(t+h\) with the output at \(t\) but use \(\gamma_X(h+j-k)\) without changing which coefficient belongs to which input time. Starting from the two input times avoids this: they are \(t+h-j\) and \(t-k\), so their difference is \(h-j+k\). For real-valued stationary processes the final covariance is symmetric in the lag, but the derivation should still use a consistent convention. The double-sum calculation gives a direct way to check each term before simplifying.
Check Your Understanding
In each question, state the lag convention you use and verify the covariance calculation from the input terms.
- For \(Y_t=\varepsilon_t-\tfrac13\varepsilon_{t-1}\), where \((\varepsilon_t)\) is mean-zero weak white noise with variance \(9\), compute \(\gamma_Y(0)\), \(\gamma_Y(1)\), and \(\gamma_Y(2)\).
- For a filter with coefficients \(a_0,a_1,a_2\) applied to weak white noise, write the formulas for \(\gamma_Y(0)\), \(\gamma_Y(1)\), and \(\gamma_Y(2)\).
- For the filter \(Y_t=X_t+cX_{t-1}\), derive \(\gamma_Y(h)\) in terms of \(\gamma_X\) for an arbitrary integer lag \(h\).
- Why is absolute summability of the filter coefficients sufficient to make the covariance double sum converge absolutely when the input variance is finite?
- For the geometric input covariance \(\gamma_X(h)=(\tfrac12)^{|h|}\) and filter \(Y_t=X_t+\tfrac12X_{t-1}\), calculate the output variance, including the cross-covariance terms.