Why These Three Processes Are Different
White noise, autoregressive processes, and moving-average processes are building blocks for time-series models. They can all have a constant mean and variance, yet differ in how observations depend on one another. The distinction is visible in their equations and in their theoretical autocorrelation function (ACF) and partial autocorrelation function (PACF).
The word pure matters here: a pure AR process has autoregressive terms and no moving-average terms, while a pure MA process has moving-average terms and no autoregressive terms. Their characteristic ACF and PACF patterns describe population correlations under the stated model, not guaranteed patterns in every finite data sample.
For a weakly stationary process \(\{X_t\}\), the mean is constant and its autocovariance at lag \(h\), \(\gamma_h=\operatorname{Cov}(X_t,X_{t-h})\), depends only on \(h\). Its ACF is \(\rho_h=\gamma_h/\gamma_0\), where \(\gamma_0=\operatorname{Var}(X_t)>0\). Thus, white noise has \(\rho_0=1\) and \(\rho_h=0\) for all \(h\ne 0\).
Pure AR and Pure MA Models
A pure autoregressive model relates the current centered observation to earlier centered observations and a new innovation. For an AR(\(p\)) process,
Here \(\mu\) is the process mean, \(\phi_1,\ldots,\phi_p\) are autoregressive coefficients, and \(\varepsilon_t\) is white noise with variance \(\sigma^2\). The innovations are assumed uncorrelated across time and uncorrelated with the past values needed to form the process. For a stationary AR process, the characteristic polynomial \(1-\phi_1z-\cdots-\phi_pz^p\) must have no roots with \(|z|\leq 1\). This root condition ensures a stationary causal solution; an AR equation outside that condition cannot be treated as a stationary AR process using the results below.
A pure moving-average process of order \(q\) is a finite weighted sum of current and past innovations:
With white-noise innovations of finite variance, this finite sum is weakly stationary. Each observation uses only \(q+1\) innovations, so observations separated by more than \(q\) time steps share no innovation and have zero covariance. No invertibility condition is needed for this stationarity result. Invertibility concerns whether the innovations can be recovered from current and past observations, a different property.
| Process | Equation uses | Theoretical ACF | Theoretical PACF |
|---|---|---|---|
| White noise | Current innovation only | Zero at every nonzero lag | Zero at every positive lag |
| Stationary pure AR(\(p\)) | Previous observations and current innovation | Typically tails off | Zero after lag \(p\) |
| Pure MA(\(q\)) | Current and previous \(q\) innovations | Zero after lag \(q\) | Typically tails off |
The PACF at lag \(k\) is the correlation between \(X_t\) and \(X_{t-k}\) after accounting linearly for the intervening values \(X_{t-1},\ldots,X_{t-k+1}\). At lag 1 there are no intervening values, so the PACF equals the ACF at lag 1. The theoretical cutoff statements assume a correctly specified, nondegenerate model. Particular coefficient values can create additional zero correlations, and sample estimates will not usually be exactly zero.
Worked Example: Recognizing White Noise Without Assuming Independence
Worked Example: Recognizing White Noise Without Assuming Independence
Let \(Z_t\) be independent standard normal random variables, and define \(X_t=Z_tZ_{t-1}\). Determine whether \(\{X_t\}\) is weak white noise, and whether its observations are independent.
Step 1: Find the mean. Independence of \(Z_t\) and \(Z_{t-1}\) gives \(E[X_t]=E[Z_t]E[Z_{t-1}]=0\).
Step 2: Find the variance. Since \(E[X_t]=0\), \(\operatorname{Var}(X_t)=E[Z_t^2Z_{t-1}^2]=E[Z_t^2]E[Z_{t-1}^2]=1\).
Step 3: Check nonzero-lag covariances. At lag 1, \(E[X_tX_{t-1}]=E[Z_tZ_{t-1}^2Z_{t-2}]=0\), because \(Z_t\) and \(Z_{t-2}\) each have mean zero and are independent of the squared term. At any lag \(h\geq2\), the products \(X_t\) and \(X_{t-h}\) contain distinct independent mean-zero factors, so \(E[X_tX_{t-h}]=0\). Thus all nonzero-lag covariances vanish.
Step 4: Check independence separately. Adjacent observations are not independent: \(X_t^2=Z_t^2Z_{t-1}^2\) and \(X_{t+1}^2=Z_{t+1}^2Z_t^2\) share \(Z_t^2\). In fact, \(E[X_t^2X_{t+1}^2]=E[Z_{t-1}^2]E[Z_t^4]E[Z_{t+1}^2]=1\cdot3\cdot1=3\), whereas \(E[X_t^2]E[X_{t+1}^2]=1\). The unequal expectations show dependence.
Conclusion and check. The process has mean zero, variance 1, and zero covariance at every nonzero lag, so it is weak white noise. It is not an independent sequence. The common mistake avoided here is treating “white noise” as synonymous with “independent noise.”
Worked Example: A Stationary AR(1) Process
Worked Example: A Stationary AR(1) Process
Suppose \(X_t-10=0.5(X_{t-1}-10)+\varepsilon_t\), where \(\varepsilon_t\) is white noise with variance 9 and is uncorrelated with past values. Find the stationary variance, the ACF at lags 1 and 3, and the PACF cutoff.
Step 1: Check stationarity. For an AR(1), the stationarity condition is \(|\phi_1|<1\). Here \(|0.5|<1\), so the stationary formulas apply.
Step 2: Calculate the variance. Let \(Y_t=X_t-10\). Then \(Y_t=0.5Y_{t-1}+\varepsilon_t\). Taking variances and using the uncorrelatedness of the innovation and the past, \(\operatorname{Var}(Y_t)=0.5^2\operatorname{Var}(Y_{t-1})+9\). Stationarity makes both variances equal to \(\gamma_0\), so \(\gamma_0=0.25\gamma_0+9\). Therefore \(0.75\gamma_0=9\) and \(\gamma_0=12\).
Step 3: Calculate the ACF. The AR(1) autocorrelation at lag \(h\) is \(\rho_h=\phi_1^h\). Thus \(\rho_1=0.5\), and \(\rho_3=0.5^3=0.125\). The ACF decays geometrically rather than cutting off after lag 1.
Step 4: State the PACF pattern. A pure AR(1) has PACF equal to zero at every lag greater than 1. Its lag-1 PACF is 0.5.
Conclusion and check. The stationary mean is 10 and variance is 12; the ACF values are 0.5 and 0.125, while the PACF cuts off after lag 1. The variance is greater than the innovation variance 9, as expected when past shocks also contribute to \(X_t\).
Why the AR(1) Pattern Holds
The AR(1) ACF result follows directly from the model. Write the centered process as \(Y_t=X_t-\mu=\phi Y_{t-1}+\varepsilon_t\), with \(|\phi|<1\). For \(h\geq1\), take covariance with \(Y_{t-h}\). The innovation is uncorrelated with the past, so
Repeated application gives \(\gamma_h=\phi^h\gamma_0\), and division by \(\gamma_0\) gives \(\rho_h=\phi^h\). This is why the AR(1) ACF tails off. The PACF cutoff has a different interpretation: once \(X_{t-1}\) is included, the AR(1) equation has no additional direct dependence on \(X_{t-2},X_{t-3},\ldots\). Consequently, the partial correlations beyond lag 1 are zero.
Worked Example: A Pure MA(2) Process
Worked Example: A Pure MA(2) Process
Let \(X_t=\varepsilon_t+0.5\varepsilon_{t-1}-0.25\varepsilon_{t-2}\), where the innovations are white noise with variance 4. Find \(\gamma_0,\gamma_1,\gamma_2\), the corresponding ACF values, and the ACF cutoff.
Step 1: Identify shared innovations. The current observation uses innovations at times \(t,t-1,t-2\). The lag-1 observation is \(X_{t-1}=\varepsilon_{t-1}+0.5\varepsilon_{t-2}-0.25\varepsilon_{t-3}\). Only the innovations at times \(t-1\) and \(t-2\) are shared.
Step 2: Calculate the variance. Uncorrelated innovations contribute their variances separately, so \(\gamma_0=4(1^2+0.5^2+(-0.25)^2)=4(1+0.25+0.0625)=5.25\).
Step 3: Calculate lag 1. Multiply the coefficients of each shared innovation across \(X_t\) and \(X_{t-1}\). This gives \(\gamma_1=4[(0.5)(1)+(-0.25)(0.5)]=4(0.375)=1.5\).
Step 4: Calculate lag 2. The only shared innovation between \(X_t\) and \(X_{t-2}\) is \(\varepsilon_{t-2}\). Its coefficients are \(-0.25\) and 1, so \(\gamma_2=4(-0.25)(1)=-1\). At lags greater than 2 there are no shared innovations, so the covariance is zero.
Step 5: Convert to correlations. \(\rho_1=1.5/5.25=2/7\), and \(\rho_2=-1/5.25=-4/21\). For all \(h>2\), \(\rho_h=0\).
Conclusion and check. The ACF cuts off after lag 2, as expected for an MA(2). The variance is positive, and the lag-2 correlation is negative because the shared innovation has coefficients with opposite signs. The calculation avoids the mistake of multiplying all coefficients in the two observations, including coefficients on innovations they do not share.
Interpreting the Patterns Carefully
For a nondegenerate pure MA(\(q\)), the finite innovation window implies an ACF cutoff after lag \(q\). For a stationary pure AR(\(p\)), the model relates observations recursively, so dependence can persist over many lags; its ACF typically tails off, while its PACF cuts off after \(p\). For white noise, both ACF and PACF are zero at positive lags. “Tails off” does not prescribe one exact shape: an AR ACF can decay gradually or oscillate while diminishing.
A cutoff is an exact statement about theoretical correlations under the model. In observed data, estimated correlations fluctuate. An estimated ACF value beyond an MA order may be nonzero by chance, and an estimated PACF beyond an AR order may also be nonzero. Conversely, a small estimated value at an early lag does not prove that the true correlation is zero. Use the model equations, assumptions, and the overall pattern together rather than treating one sample bar as a definitive test.
Check Your Understanding
For each problem, identify the relevant assumptions and distinguish exact model properties from sample-based impressions.
- A zero-mean process has variance 16 and autocovariance zero at every nonzero lag. State its ACF at lags 0 and 4, and its PACF at lag 2. Does this information establish independence? (Answer: \(\rho_0=1\), \(\rho_4=0\), and lag-2 PACF \(=0\); independence is not established.)
- A stationary AR(1) model has coefficient \(-0.4\) and innovation variance 9. Find its variance and ACF at lag 3, and state where its PACF cuts off. (Answer: variance \(9/(1-0.16)=10.7143\), \(\rho_3=(-0.4)^3=-0.064\), and the PACF cuts off after lag 1.)
- For \(X_t=\varepsilon_t+0.75\varepsilon_{t-1}\) with white-noise innovation variance 4, calculate \(\gamma_0,\gamma_1,\rho_1\), and \(\rho_2\). (Answer: \(\gamma_0=6.25\), \(\gamma_1=3\), \(\rho_1=0.48\), and \(\rho_2=0\).)
- A stationary pure AR(2) has both autoregressive coefficients nonzero and satisfies the stationarity root condition. State the characteristic theoretical cutoff and which function typically tails off. (Answer: PACF is zero after lag 2; the ACF typically tails off.)
- A pure MA(2) is \(X_t=\varepsilon_t-0.5\varepsilon_{t-1}+0.25\varepsilon_{t-2}\), with innovation variance 4. Calculate its variance and ACF at lags 1 and 2, and state the ACF cutoff. (Answer: \(\gamma_0=5.25\), \(\rho_1=-10/21\), \(\rho_2=4/21\), and the ACF is zero after lag 2.)