Why Residual Shape and Dependence Are Different Questions
After fitting a time-series model, residuals are checked to determine whether the model has captured the structure needed for its intended use. Two questions are often confused: Are residuals normally distributed? and Are residuals serially dependent? The first concerns their marginal distributional shape; the second asks whether residual values are related across time. A model can have non-normal residuals with little evidence of serial correlation, or nearly normal residuals that remain autocorrelated.
That distinction matters. Residual autocorrelation can indicate that the model has left predictable time structure unexplained, weakening its mean forecasts. Non-normality alone does not establish that the conditional mean is wrong. It can, however, affect the accuracy of normal-theory likelihood inference and forecast intervals. The appropriate response depends on which feature is present and what the model will be used for.
In practice, fitted residuals are used to assess the unobserved innovations. The earlier material on residual ACFs and Ljung–Box tests addresses evidence of serial correlation. Distributional diagnostics, such as a histogram or normal Q–Q plot, address whether the residual values look compatible with a normal distribution. Neither diagnostic substitutes for the other.
A few large residuals or visibly asymmetric tails may suggest non-normality, but plots and tests are uncertain, especially in short series. A normality test can reject because of a small number of extreme observations, while a failure to reject does not establish normality. Interpret distributional evidence alongside the sample size, residual dependence, and purpose of the model.
Worked Example: Separate a Skewed Residual Sample From Lag-One Correlation
A fitted model produces the ordered residuals \(-3, 0, 1, 0, 2\). Assess their sample mean, moment skewness, and lag-one sample autocorrelation. What can these calculations establish?
Method. Calculate the distributional summary and the lag-one correlation separately. For this illustration, use the moment skewness \(m_3/m_2^{3/2}\), where \(m_j\) is the average \(j\)th power of the deviations from the sample mean. Use the common residual-ACF convention with the sum of squared residuals in the denominator.
The residual sum is \(-3+0+1+0+2=0\), so the sample mean is \(0/5=0\). The sum of squares is \[ (-3)^2+0^2+1^2+0^2+2^2=9+0+1+0+4=14. \] Thus \(m_2=14/5=2.8\). The sum of cubed deviations is \(-27+0+1+0+8=-18\), so \(m_3=-18/5=-3.6\). The moment skewness is \[ \frac{-3.6}{(2.8)^{3/2}}\approx \frac{-3.6}{4.685}=-0.768. \] This small sample has a negative third-moment skewness; that is a description of these five values, not proof that the underlying innovations have a particular distribution.
For lag one, the numerator is the sum of adjacent products: \[ (-3)(0)+(0)(1)+(1)(0)+(0)(2)=0. \] The denominator, the sum of squares, is \(14\). Therefore the lag-one residual autocorrelation is \(0/14=0\) under the stated convention.
Conclusion and check. The sample has a negative skewness summary and zero lag-one autocorrelation. This does not show that all residual autocorrelations are zero: lags two and higher have not been checked. Nor does five observations provide reliable evidence for the population distribution. The common mistake is to treat a skewness finding as evidence of autocorrelation, or a zero lag-one value as proof of normality; these calculations answer distinct, limited questions.
What Non-Normality Means for Likelihood-Based Inference
Gaussian likelihood methods assume that, conditional on the past and the model parameters, the innovations have a normal distribution. Under that assumption, the Gaussian likelihood gives the probability model used for estimation and the associated normal-theory standard errors, tests, and confidence statements. If the innovations are not normal, the Gaussian likelihood is not the exact likelihood for their distribution. In finite samples, inferences based directly on the Gaussian assumption may therefore be poorly calibrated.
This does not mean that every estimate of the mean dynamics is automatically unusable. Under suitable regularity conditions, a correctly specified conditional mean and stable dependence structure can allow Gaussian likelihood estimation to function as a quasi-likelihood method: estimates of mean parameters may still be consistent, and may be asymptotically normal, even when the innovations are non-normal. Those conclusions require conditions such as a correctly specified mean, suitable stationarity and moment conditions, and an appropriate estimation procedure. They do not guarantee that ordinary Gaussian standard errors or small-sample tests are accurate.
When normality is doubtful, the analyst should distinguish the mean-model question from the inference question. A mean model may still capture the predictable component, while uncertainty estimates based on Gaussian assumptions need additional scrutiny. Depending on the setting, robust covariance estimates, a distribution better suited to the residual shape, or a bootstrap designed to respect the time-series structure may be considered. Resampling residuals as if they were independent is not justified when dependence remains.
Why Normal-Theory Forecast Intervals Can Mislead
A point forecast is commonly the conditional mean of a future observation given the data available at the forecast origin. If the conditional mean is correctly specified and future innovations have conditional mean zero, the point forecast can remain appropriate without normally distributed errors. A forecast interval makes a stronger claim: it describes the distribution of future forecast errors. A normal-theory interval therefore needs a suitably normal predictive distribution, not merely a plausible point forecast.
For a one-step forecast with conditional mean \(\hat{y}_{T+1|T}\) and innovation standard deviation \(\sigma\), a familiar approximate 95% interval is the forecast mean plus or minus \(1.96\sigma\). This uses the standard normal 97.5th percentile. If the innovation distribution is skewed or heavy-tailed, its actual quantiles need not be \(-1.96\) and \(1.96\) standard deviations. A symmetric interval can then miss too many values on one side or fail to include sufficiently extreme values.
At horizons beyond one step, the forecast error can combine several future innovations. Its distribution depends on the model’s propagation of those innovations and on their joint distribution. Normal innovations in a linear model give normal forecast errors under the usual assumptions. With non-normal innovations, the forecast-error distribution need not be normal, even when its mean and variance have been calculated correctly. The normal interval may be a useful approximation in some cases, but its coverage should not be assumed.
Worked Example: A Skewed Innovation Distribution Changes Interval Coverage
A one-step model forecast for a hypothetical monthly account value is \(100\) thousand dollars. The next innovation is \(-5\) thousand dollars with probability \(0.8\), and \(20\) thousand dollars with probability \(0.2\). Calculate its mean and variance, then assess the Gaussian 95% interval that uses the same mean and variance.
Method. First calculate the innovation moments from the stated discrete distribution. Then compare the normal-theory interval with the two possible forecast outcomes. All amounts below are in thousands of dollars.
The innovation mean is \[ E[\varepsilon]=0.8(-5)+0.2(20)=-4+4=0. \] The variance is the expected squared deviation from zero: \[ \operatorname{Var}(\varepsilon)=0.8(-5)^2+0.2(20)^2 =0.8(25)+0.2(400)=20+80=100. \] The standard deviation is \(\sqrt{100}=10\). The Gaussian 95% interval is therefore \[ 100\pm1.96(10)=100\pm19.6, \] or \([80.4,119.6]\) thousand dollars.
Under the stated innovation distribution, the actual next value is \(100-5=95\) with probability \(0.8\), or \(100+20=120\) with probability \(0.2\). The Gaussian interval includes \(95\) but excludes \(120\), so its actual coverage under this distribution is \(0.8\), not \(0.95\). The mean forecast of \(100\) remains correct because the innovation mean is zero, but the interval’s Gaussian calibration fails.
Check and common mistake. The innovation values have mean zero and variance \(100\), as required by the calculation. The mistake is to infer that matching the mean and variance makes a normal interval valid. It does not: the distribution here is asymmetric, and the upper outcome alone has probability \(0.2\), far greater than a normal upper-tail probability of \(0.025\) beyond \(1.96\) standard deviations.
A Short Derivation: Why the Mean Forecast Need Not Require Normality
Consider the AR(1) model \[ y_t-\mu=\phi(y_{t-1}-\mu)+\varepsilon_t, \] where \(\mu\) is the long-run mean, \(|\phi|<1\), and the innovation has conditional mean zero given information available through time \(t-1\). Normality is not needed for the following conditional-mean calculation.
At time \(T\), the observed \(y_T\) is known. Taking conditional expectation given information through \(T\), and using \(E[\varepsilon_{T+1}\mid\mathcal{F}_T]=0\), gives \[ E[y_{T+1}\mid\mathcal{F}_T] =\mu+\phi(y_T-\mu). \] Apply the same model one step further. The term involving \(\varepsilon_{T+2}\) has conditional expectation zero given \(\mathcal{F}_T\), by iterated conditional expectation, so \[ E[y_{T+2}\mid\mathcal{F}_T] =\mu+\phi\big(E[y_{T+1}\mid\mathcal{F}_T]-\mu\big) =\mu+\phi^2(y_T-\mu). \] Repeating the recursion yields \(\mu+\phi^h(y_T-\mu)\) at horizon \(h\). This derivation uses the zero conditional mean of future innovations and the specified model recursion. It does not use a normal distribution. Normality matters for the shape of the predictive distribution and for exact normal-theory inference, not for this conditional-mean identity.
Worked Example: Retain a Mean Forecast While Qualifying Its Uncertainty
A hypothetical stationary AR(1) model for monthly claim counts has long-run mean \(\mu=40\), coefficient \(\phi=0.5\), and a final observed value \(y_T=48\). The estimated innovation variance is \(9\), but residual diagnostics suggest skewness and show no material residual autocorrelation at the examined lags. Calculate the one- and two-step conditional mean forecasts and the two-step forecast-error variance under the model. State what can and cannot be concluded.
Method. Use the AR(1) conditional-mean recursion and the usual variance recursion for uncorrelated, constant-variance innovations. The model is stationary because \(|0.5|<1\). The variance calculation assumes innovations are uncorrelated with variance \(9\); normality is not needed for the variance itself, but is needed for an exact normal predictive distribution.
The final deviation from the long-run mean is \(48-40=8\). The one-step forecast is \[ 40+0.5(8)=44. \] The two-step forecast is \[ 40+0.5(44-40)=40+0.5(4)=42. \] Equivalently, it is \(40+(0.5)^2(8)=42\).
The two-step forecast error contains the next innovation and one-half of the following innovation: \[ \varepsilon_{T+2}+0.5\varepsilon_{T+1}. \] If the innovations are uncorrelated and each has variance \(9\), the variance is \[ 9+(0.5)^2(9)=9+2.25=11.25. \] The standard deviation is \(\sqrt{11.25}\approx3.354\). A Gaussian 95% interval would be \(42\pm1.96(3.354)\), approximately \([35.43,48.57]\) claim counts, rounded to two decimals.
Conclusion and check. The point forecasts follow from the mean recursion and remain justified if the conditional mean is correct and future innovations have conditional mean zero. The variance follows from the stated uncorrelated-error assumption. The interval is only a Gaussian approximation if the innovations are non-normal; its coverage is not established by the mean and variance calculations. The common mistake is to discard the mean model solely because its residuals are skewed, or to report the normal interval as exactly calibrated despite that skewness.
A Practical Decision Rule and Common Traps
The purpose of residual analysis is to understand what model features remain in the errors, not to enforce normality as a universal pass-or-fail requirement. For a model intended to forecast a conditional mean, residual autocorrelation is usually more direct evidence of remaining predictable time structure than non-normality alone. If dependence remains, revisit the mean dynamics and the residual dependence diagnostics. If residuals appear uncorrelated but non-normal, preserve that distinction and assess whether the model’s intended inference or interval use relies strongly on Gaussian assumptions.
- Do not equate white noise with Gaussian noise. White noise requires zero autocorrelation, not normality or independence.
- Do not equate a normal-looking histogram with adequate dynamics. Check serial dependence separately.
- Do not treat a normality-test rejection as automatic grounds to reject the mean model. Consider sample size, influential residuals, dependence, and the forecasting objective.
- Do not promise normal-theory interval coverage from a correct point forecast. The predictive distribution requires its own assumptions.
- Do not assume that Gaussian likelihood inference is exact under non-normal innovations. Quasi-likelihood results require conditions, and finite-sample calibration can still be affected.
For a forecasting task, the model should be judged on whether its mean structure is adequate and whether its uncertainty statements are defensible for the observed residual behavior. If interval calibration is important and normality is doubtful, a suitable alternative predictive distribution or simulation approach may be needed. Any method must also account for the time-series dependence structure rather than treating dependent errors as independent observations.
Check Your Understanding
For each situation, distinguish what the diagnostics imply from what they do not establish.
- A fitted monthly model has residual skewness, while its residual ACF has no notable spikes at the checked lags. Is non-normality evidence by itself that the conditional mean is wrong? (Answer: No; it raises a distributional concern but does not by itself establish mean-model misspecification.)
- A one-step forecast is \(50\). Its innovation is \(-1\) with probability \(0.9\) and \(9\) with probability \(0.1\). Find the innovation mean and variance, then the Gaussian 95% interval using the same moments. What is that interval’s actual coverage under the stated distribution? (Answer: Mean \(0\), variance \(9\), Gaussian interval \([44.12,55.88]\), actual coverage \(0.9000\).)
- A model’s residual Q–Q plot appears close to a straight line, but its residual ACF has a clear lag-one spike. Which concern should be prioritized? (Answer: Residual autocorrelation; normal-looking marginal shape does not remove evidence of serial dependence.)
- For a stationary AR(1) model with \(\mu=20\), \(\phi=0.4\), and \(y_T=25\), calculate the one- and two-step conditional mean forecasts. (Answer: One-step \(22\); two-step \(20.8\).)
- In the model in Question 4, suppose the innovation variance is \(16\), and innovations are uncorrelated. Calculate the two-step forecast-error variance and state whether it alone validates a Gaussian 95% interval. (Answer: Variance \(16(1+0.4^2)=18.56\); no, the variance alone does not establish Gaussian predictive errors or interval coverage.)