Collinearity in Predictors That Share a Trend

Why Shared Trends Can Complicate Regression

Two measurements can move together over time even when neither provides much information that is distinct from the other. For example, a hypothetical organization might record annual spending on two related activities; both amounts may rise as the organization grows. Including both predictors in a regression can make it difficult to estimate their separate relationships with an outcome, because the observed years provide few comparisons in which one amount changes while the other stays similar.

This is a particular way that collinearity can arise. The pattern is not diagnosed just by noticing that variables increase over time: the relevant question is whether their values carry largely overlapping information in the observations used to fit the model. Plots against time can reveal a shared pattern, and regression summaries can show whether separate coefficient estimates are correspondingly uncertain.

Model and Meaning of the Coefficients

Consider observations indexed by time \(i\), with an outcome \(Y_i\) and two quantitative predictors \(X_{1i}\) and \(X_{2i}\). The multiple linear regression model is

Y_i=\beta_0+\beta_1X_{1i}+\beta_2X_{2i}+\varepsilon_i.

Here, \(\beta_0\) is the intercept, and each slope describes the modeled change in the mean outcome for a one-unit increase in that predictor holding the other predictor fixed. For usual least-squares standard errors and \(t\)-based intervals and tests, assume the errors have mean zero, constant variance \(\sigma^2\), are independent across observations, and are normally distributed. The predictors must not be perfectly linearly dependent; otherwise, the separate coefficients cannot be uniquely estimated. These assumptions concern the errors given the predictors. A shared trend among predictors is not, by itself, a violation of the error assumptions, but it can make the separate slopes imprecise.

Definition: Predictors share a trend when their values vary systematically with time in similar directions over the observed period. If that common pattern makes one predictor largely predictable from the others, the predictors are redundant for estimating their separate regression coefficients. A trend plot is a diagnostic clue, not proof of collinearity on its own.

The conditional meaning of a slope matters here. If \(X_1\) and \(X_2\) almost always rise together, the data contain few observed comparisons that increase \(X_1\) while holding \(X_2\) fixed, or vice versa. The fitted model may therefore have limited information for distinguishing \(\beta_1\) from \(\beta_2\). This is different from asking whether the outcome rises over time or whether the predictors are associated with the outcome.

Why a Common Trend Can Produce Collinearity

A simple way to describe the pattern is to write each predictor as a linear trend plus a departure from that trend:

X_{1i}=a_1+b_1t_i+u_{1i},\qquad X_{2i}=a_2+b_2t_i+u_{2i}.

Here \(t_i\) is time, \(b_1\) and \(b_2\) are the trends, and \(u_{1i}\) and \(u_{2i}\) are departures from them. If both trends move in the same direction and the departures are small, the predictors will tend to be strongly associated. To see why, suppose the departures have mean zero and are uncorrelated with time. Then

\operatorname{Cov}(X_1,X_2) =b_1b_2\operatorname{Var}(t)+\operatorname{Cov}(u_1,u_2).

The first term is the covariance contributed by the shared time pattern. When it is large relative to the variation in the departures, much of the information in one predictor overlaps with the other. This explains the diagnostic pattern; it does not establish that time caused either predictor to change.

Worked Example: Spotting a Shared Trend in Hypothetical Data

Suppose a hypothetical organization records two annual spending measures, \(X_1\) and \(X_2\), in thousands of dollars. The values are:

Year index \(t\)	\(X_1\)	\(X_2\)
1	2	1.0
2	4	2.2
3	6	2.8
4	8	4.2
5	10	4.8
6	12	6.2

Both measures generally rise with year. To summarize their pairwise association, their means are \(\bar{x}_1=7\) and \(\bar{x}_2=3.533\). The sum of cross-products of centered values is \(35.2\); the sums of squared centered values are \(70\) for \(X_1\) and \(17.9\) for \(X_2\). Therefore,

r_{X_1,X_2} =\frac{35.2}{\sqrt{70(17.9)}} \approx 0.994.

The high positive correlation is consistent with strong redundancy in this small set of observations. A plot of each measure against year would show both increasing; a scatterplot of \(X_1\) against \(X_2\) would show points close to an increasing line. Those displays provide complementary evidence. They do not tell us whether either spending measure affects an outcome, and six observations are not enough to establish a general pattern.

Reading the Regression Summary

A useful diagnosis connects the predictor plots to the fitted model. If predictors share a trend, inspect their scatterplot and their plots against time, then examine the estimated slopes, standard errors, and intervals in the regression summary. Large standard errors relative to the estimated slopes indicate that the data do not pin down the corresponding conditional comparisons precisely. Do not infer that the model is useless from this fact alone: the overall fitted pattern or predictions for predictor combinations like those observed may still be informative.

The standard error of a slope measures the estimated sampling variability of its coefficient estimate under the model assumptions. Collinearity reduces the distinct predictor variation available for separating slopes; as a result, standard errors can be large. This is about precision, not automatic bias. If the error assumptions and model are appropriate, collinearity alone does not make least-squares coefficient estimates biased.

Worked Example: Interpreting Uncertain Separate Slopes

Suppose a hypothetical regression uses 12 annual observations to relate an outcome, measured in units, to two spending predictors, each measured in thousands of dollars. Both predictors increase over time, and their scatterplot is close to a line. Assume the regression error conditions stated above are reasonable and there is no perfect linear dependence. A summary reports \(\hat{\beta}_1=2.4\) with \(SE=1.8\), and \(\hat{\beta}_2=1.1\) with \(SE=1.5\). With an intercept and two slopes, \(df=12-3=9\). For a 95% \(t\)-interval, use \(t^*=2.262\).

For the first slope, the interval is

2.4\pm 2.262(1.8) =2.4\pm4.072 =(-1.672,\ 6.472).

For the second slope, it is

1.1\pm 2.262(1.5) =1.1\pm3.393 =(-2.293,\ 4.493).

Both intervals include zero and are broad relative to their point estimates. In this fitted model, the data do not estimate either separate conditional slope precisely. For example, \(\hat{\beta}_1=2.4\) describes an estimated increase of 2.4 outcome units per additional thousand dollars in \(X_1\), for a fixed value of \(X_2\); it is not the simple association between \(X_1\) and the outcome over time. The intervals are not evidence that the true slopes equal zero. Their width, together with the shared-trend plots, is consistent with limited information for separating the effects. The calculation relies on the stated error assumptions; if residuals show dependence over time, these standard errors and intervals may not be appropriate.

Separate Coefficients and Predictions

When observed predictor combinations lie close to a line, the model may estimate the outcome along that line more precisely than it estimates what would happen by changing one predictor while holding the other fixed. The following arithmetic illustrates why. It is a hypothetical comparison of fitted coefficient pairs, not a claim that these particular summaries came from one data set.

Worked Example: Different Slopes, Similar Prediction Along the Trend

Suppose two plausible fitted summaries for predictors measured in thousands of dollars give these coefficient pairs: Model A has slopes \(3.2\) and \(1.0\), while Model B has slopes \(1.2\) and \(5.0\). In both, the predictors commonly occur near \(X_2=0.5X_1\). At the representative observed combination \(X_1=10\), \(X_2=5\), the contribution of the predictors to the fitted outcome in Model A is

$$ 3.2(10)+1.0(5)=37. $$

In Model B it is

$$ 1.2(10)+5.0(5)=37. $$

The slope estimates differ substantially, yet these contributions agree at this predictor combination. Now consider a comparison that holds \(X_1=10\) but changes \(X_2\) from 5.0 to 5.2. Model A changes its predicted outcome by \(1.0(0.2)=0.2\) units, whereas Model B changes it by \(5.0(0.2)=1.0\) unit. The models disagree more for this separate change because the data pattern provides less support for such a comparison. These calculations illustrate the distinction; assessing actual prediction uncertainty requires the fitted model and the relevant prediction interval.

A Practical Diagnostic Sequence

A focused review of predictors that share a trend can be organized as follows:

Plot each predictor against time.
Look for similar directions and shapes, and note whether the pattern is driven by a few observations or is visible throughout the observed period.

Compare the predictors directly.
Use a scatterplot and, as a first numerical screen, their pairwise correlation. A tight pattern suggests overlapping information, but pairwise correlation alone cannot diagnose every form of redundancy.

Connect the plots to coefficient uncertainty.
Review coefficient estimates and standard errors, and consider whether intervals are wide. A variance inflation factor (VIF), interpreted in context rather than against a rigid cutoff, can provide a further measure of redundancy.

Check the outcome model’s time pattern.
Examine residuals against time. Systematic residual structure or dependence can indicate that the error assumptions or the way the mean outcome is modeled need attention; shared predictor trends do not resolve that issue.

The displays should be interpreted together. Two predictors can each trend over time without being nearly redundant if their departures from their trends contain enough distinct variation. Conversely, a modest pairwise correlation does not rule out more complex redundancy among several predictors. The central issue is whether the observed data contain useful variation for the particular conditional comparisons represented by the separate coefficients.

Common Mistake

Wrong reasoning: “Both predictors rise over time, so they are collinear, and their regression coefficients must be biased.”

Correction: Similar trends are a reason to investigate, not a complete diagnosis. Check the predictor scatterplot and regression summary to see whether the predictors are redundant in the fitted data. Collinearity can inflate standard errors and destabilize separate coefficient estimates, but it does not by itself bias least-squares estimates. Also, a slope is a conditional association under the model, not automatically a causal effect. If residuals have a temporal pattern, that is a separate concern about the error assumptions or model specification.

Key takeaway: Shared trends can make predictors move together and leave little information for estimating their separate conditional slopes. Use time plots, predictor association, and regression uncertainty together; distinguish imprecise coefficients from poor predictions along the observed pattern, and check the regression assumptions separately.

Check Your Understanding

Use the definitions and diagnostic reasoning in this tutorial to answer the following questions.

Two predictors both rise steadily over the observed years. What additional plot would help assess whether they are redundant with one another?
In a multiple regression, what does the coefficient of \(X_1\) describe when \(X_2\) is also included?
Why can strong predictor association make separate coefficient intervals wide even if fitted values along the observed predictor pattern are similar?
Does collinearity alone show that least-squares coefficient estimates are biased? Explain.
Why should residuals be plotted against time even after shared trends among predictors have been investigated?