Covariance Implied by Random Intercepts and Slopes

Why measurement times matter

Two biomarker measurements from the same person are related because they share that person’s underlying trajectory. With a random intercept and a random slope, the trajectory can differ across people both in its level and in its rate of change. The covariance between measurements therefore depends on where both measurements fall in time, not just on how far apart they are.

The aim here is to derive that covariance and interpret its components. The formula is conditional on the measurement times and covariates being considered. It describes the part of dependence induced by the subject-specific intercept and slope; it does not, by itself, specify every possible source of within-subject dependence.

Model and assumptions

Let $Y_i(t)$ denote the biomarker value for subject $i$ at time $t$. Write a linear mixed model as

Y_i(t)=\mathbf{x}_i(t)^\top\boldsymbol{\beta}+b_{0i}+b_{1i}t+\varepsilon_i(t).

Here $\mathbf{x}_i(t)^\top\boldsymbol{\beta}$ is the fixed-effects mean, $b_{0i}$ is the subject-specific random intercept, and $b_{1i}$ is the subject-specific random slope per unit of time. For the derivation, take the measurement times and covariates as fixed. Assume the random effects have mean zero and finite second moments, with covariance matrix

\begin{pmatrix}b_{0i}\\b_{1i}\end{pmatrix} \sim \text{a distribution with mean } \begin{pmatrix}0\\0\end{pmatrix} \text{ and covariance } \begin{pmatrix} \sigma_0^2 & \sigma_{01}\\ \sigma_{01} & \sigma_1^2 \end{pmatrix}.

The notation $\sigma_0^2$ and $\sigma_1^2$ denotes the random-intercept and random-slope variances, and $\sigma_{01}$ is their covariance. Normality is not needed for the covariance calculation. If the random effects are parameterized by a correlation $\rho$, then $\sigma_{01}=\rho\sigma_0\sigma_1$, where $\sigma_0$ and $\sigma_1$ are the nonnegative square roots of the corresponding variances.

The fixed-effects mean is treated as nonrandom when conditioning on the covariates. To isolate the covariance induced by the random effects, first omit $\varepsilon_i(t)$, or equivalently consider the random-effects part of the outcome. If measurement errors are also included, assume for this basic calculation that they have mean zero and are independent of the random effects. For two distinct visits, independently varying measurement errors add no cross-visit covariance under that assumption.

Definition: The random-effects-induced covariance between measurements at times $t$ and $s$ is the covariance of $b_{0i}+b_{1i}t$ and $b_{0i}+b_{1i}s$, conditional on those times and the fixed covariates. It is $$ \operatorname{Cov}\{b_{0i}+b_{1i}t,\ b_{0i}+b_{1i}s\} =\sigma_0^2+(t+s)\sigma_{01}+ts\sigma_1^2. $$

Deriving the covariance formula

Covariance is bilinear: constants can be taken outside, and the covariance of sums is the sum of the pairwise covariances. Expand the covariance of the two subject-specific deviations from the fixed mean:

\begin{aligned} \operatorname{Cov}(b_{0i}+b_{1i}t,\ b_{0i}+b_{1i}s) &=\operatorname{Cov}(b_{0i},b_{0i}) +s\operatorname{Cov}(b_{0i},b_{1i})\\ &\quad+t\operatorname{Cov}(b_{1i},b_{0i}) +ts\operatorname{Cov}(b_{1i},b_{1i})\\ &=\sigma_0^2+s\sigma_{01}+t\sigma_{01}+ts\sigma_1^2\\ &=\sigma_0^2+(t+s)\sigma_{01}+ts\sigma_1^2. \end{aligned}

This calculation uses $\operatorname{Cov}(b_{0i},b_{1i})=\operatorname{Cov}(b_{1i},b_{0i})=\sigma_{01}$. Setting $s=t$ gives the random-effects variance of a subject’s trajectory at time $t$:

\operatorname{Var}(b_{0i}+b_{1i}t) =\sigma_0^2+2t\sigma_{01}+t^2\sigma_1^2.

The two formulas make the time dependence explicit. Intercept variation contributes the same $\sigma_0^2$ at every pair of times. Slope variation contributes $ts\sigma_1^2$, which depends on both times. The intercept–slope covariance contributes $(t+s)\sigma_{01}$, also depending on both times. Thus, even if the time gap $s-t$ is unchanged, the covariance may differ when the pair of times is shifted along the follow-up scale.

For the random-effects covariance matrix to be valid, its variances must be nonnegative and its determinant must be nonnegative:

\sigma_0^2\geq 0,\qquad \sigma_1^2\geq 0,\qquad \sigma_{01}^2\leq \sigma_0^2\sigma_1^2.

These conditions ensure that every linear combination of the random effects has a nonnegative variance. They do not require every covariance between two different-time measurements to be positive: a valid model can imply negative covariance for some time pairs.

Worked Example: Covariance at Two Follow-Up Times

Suppose, hypothetically, that a biomarker is measured in units at times measured in months from baseline. Let the random-intercept variance be $9$ units squared, the random-slope variance be $0.25$ units squared per month squared, and the intercept–slope covariance be $-0.60$ units squared per month. Find the random-effects-induced covariance between measurements at 2 and 8 months.

The proposed random-effects covariance matrix is valid because $9\geq0$, $0.25\geq0$, and $(-0.60)^2=0.36\leq9(0.25)=2.25$. With $t=2$ and $s=8$, substitute into the formula:

\begin{aligned} \operatorname{Cov}\{Y_i(2),Y_i(8)\}_{\text{random effects}} &=9+(2+8)(-0.60)+(2)(8)(0.25)\\ &=9-6+4\\ &=7.00\text{ units}^2. \end{aligned}

The covariance is positive: in this model, subjects whose random-effects contribution is above its mean at 2 months tend, on average, also to have an above-mean contribution at 8 months. The negative intercept–slope covariance reduces the covariance between these measurements, while the positive slope-variance contribution increases it.

As a check, the random-effects variances at 2 and 8 months are $9+2(2)(-0.60)+2^2(0.25)=7.60$ and $9+2(8)(-0.60)+8^2(0.25)=15.4$ units squared. The implied random-effects correlation is $7.00/\sqrt{7.60(15.4)}\approx0.647$, a value between $-1$ and $1$.

Changing the time origin

The covariance between measurements at particular physical times does not change just because the time variable is recoded. However, the meaning and covariance of the random intercept do change: it is the subject-specific deviation at the chosen time origin. This matters when interpreting a fitted intercept–slope covariance or comparing model descriptions that use different time origins.

For a new time variable $u=t-c$, where $c$ is a fixed centering time, rewrite the random part:

b_{0i}+b_{1i}t=(b_{0i}+cb_{1i})+b_{1i}u.

Thus the random intercept on the centered time scale is $b_{0i}^{*}=b_{0i}+cb_{1i}$, and the random slope remains $b_{1i}$. Its variance and covariance with the slope are

\operatorname{Var}(b_{0i}^{*})=\sigma_0^2+2c\sigma_{01}+c^2\sigma_1^2, \qquad \operatorname{Cov}(b_{0i}^{*},b_{1i})=\sigma_{01}+c\sigma_1^2.

Worked Example: Centering Time at Four Days

Consider a hypothetical biomarker model with time in days, $\sigma_0^2=4$ units squared, $\sigma_1^2=0.09$ units squared per day squared, and $\sigma_{01}=0.30$ units squared per day. Compare the covariance between measurements at days 2 and 7 before and after centering time at day 4.

On the original scale, the covariance is

4+(2+7)(0.30)+(2)(7)(0.09) =4+2.70+1.26=7.96\text{ units}^2.

After centering, the times are $u=-2$ and $v=3$. The new intercept variance is $4+2(4)(0.30)+4^2(0.09)=7.84$ units squared, and the new intercept–slope covariance is $0.30+4(0.09)=0.66$ units squared per day. Therefore,

7.84+(-2+3)(0.66)+(-2)(3)(0.09) =7.84+0.66-0.54=7.96\text{ units}^2.

The covariance at the two physical measurement times is unchanged. The random-intercept variance and intercept–slope covariance changed because the intercept now refers to day 4 rather than day 0.

Why covariance can be negative

A negative covariance at two times is not, by itself, evidence that the model is invalid. It can arise when people with larger subject-specific levels tend to have more negative slopes, or from how the slope variation contributes at the chosen times. The validity conditions apply to the random-effects covariance matrix, not to the requirement that every pairwise covariance of measurements be positive.

Worked Example: A Valid Model With Negative Covariance

Suppose a hypothetical biomarker is measured in units at times in years, with $\sigma_0^2=1$ unit squared, $\sigma_1^2=1$ unit squared per year squared, and $\sigma_{01}=-0.80$ units squared per year. The matrix is valid because $0.80^2=0.64\leq1(1)$. Find the covariance between the measurements at years 0 and 2.

\operatorname{Cov}\{Y_i(0),Y_i(2)\}_{\text{random effects}} =1+(0+2)(-0.80)+(0)(2)(1) =1-1.60=-0.60\text{ units}^2.

The random-effects variance at year 0 is $1$ unit squared, and at year 2 it is $1+2(2)(-0.80)+2^2(1)=1.80$ units squared. The implied correlation is $-0.60/\sqrt{1(1.80)}\approx-0.447$, which is admissible. In this example, the negative association of intercept and slope makes subjects with higher deviations at baseline tend to have lower random-effects deviations at year 2.

Checking the implied pattern and choosing a model

The formula gives a useful way to assess whether a random-intercept, random-slope structure is plausible for the scientific setting and the observed follow-up schedule. Its implications can be checked against the data and against alternative structures, while recognizing that estimated covariances also reflect sampling uncertainty.

Check the time scale and origin. A slope variance is expressed in squared biomarker units per squared time unit, and $\sigma_{01}$ is expressed in squared biomarker units per time unit. Rescaling time changes these numerical values. Centering changes the random-intercept variance and intercept–slope covariance.
Check the covariance matrix. Estimates that violate $\sigma_{01}^2\leq\sigma_0^2\sigma_1^2$ cannot describe a valid covariance matrix. A fitted model must respect the nonnegative-variance and determinant conditions.
Compare implied and observed within-subject patterns. Large discrepancies across time pairs can suggest that a linear random trajectory is inadequate or that another source of dependence needs consideration. Sparse observations at particular times can make this assessment uncertain.
Consider simpler or richer structures. A random-intercept-only model implies a constant random-effects covariance $\sigma_0^2$ across all time pairs. A model with independent random intercepts and slopes sets $\sigma_{01}=0$, leaving $\sigma_0^2+ts\sigma_1^2$. Allowing $\sigma_{01}$ to be nonzero gives more flexibility, but requires enough information to estimate the additional covariance.

These structures are not interchangeable descriptions of the same assumption. For example, forcing $\sigma_{01}=0$ says that subject-specific baseline deviations and rates of change are uncorrelated under the model. It does not say that the random intercept and slope have no effect on measurements. Conversely, a good fit to a few observed covariance patterns does not establish that a linear subject-specific trajectory is correct at unobserved times.

The calculation also has limits. It describes covariance induced by the specified random intercept and linear random slope, conditional on the time scale and covariates. It does not accommodate nonlinear subject-specific trajectories unless the model is expanded, and it does not automatically account for serially correlated measurement errors. When visit times depend on a subject’s unobserved biomarker trajectory, conditioning on the observed times may also require care; a covariance formula alone does not resolve that observation-process issue.

Common mistake

A common mistake is to treat covariance as a function only of the time gap $s-t$, as if all pairs separated by the same interval must have the same covariance. That conclusion is generally false here: the terms $(t+s)\sigma_{01}$ and $ts\sigma_1^2$ depend on the locations of both times. Another mistake is to omit the intercept–slope contribution or to use $2t\sigma_{01}$ for two different times. The correct contribution is $(t+s)\sigma_{01}$; it becomes $2t\sigma_{01}$ only when the two times are equal.

Key takeaway: Under a random-intercept, random-slope model, the covariance between measurements at times $t$ and $s$ is $\sigma_0^2+(t+s)\sigma_{01}+ts\sigma_1^2$. Both times, the time origin, and the intercept–slope association matter.

Check Your Understanding

Answer each question, then compare with the solutions below.

If $\sigma_0^2=4$, $\sigma_1^2=0.25$, and $\sigma_{01}=0.50$, what is the random-effects covariance between measurements at times 1 and 3?
For the same parameters, what is the random-effects variance at time 2?
Why does centering time change the random-intercept variance but not the covariance between measurements at the same two physical times?
Does a negative covariance between two measurements necessarily imply an invalid random-effects covariance matrix? State the matrix condition that determines validity.

Solutions. (1) Substitute the times: $4+(1+3)(0.50)+(1)(3)(0.25)=4+2+0.75=6.75$ units squared. (2) Set both times to 2: $4+2(2)(0.50)+2^2(0.25)=4+2+1=7.00$ units squared. (3) Centering changes which physical time is represented by the intercept, so the random intercept becomes $b_{0i}+cb_{1i}$; the covariance at the original physical times is unchanged because the fitted random trajectory is simply rewritten using a different time coordinate. (4) No. The matrix is valid when $\sigma_0^2\geq0$, $\sigma_1^2\geq0$, and $\sigma_{01}^2\leq\sigma_0^2\sigma_1^2$; an individual pair of measurements may still have negative covariance.