Conditional Covariance Across Repeated Observations

Why Repeated Observations Can Move Together

A risk that tends to generate high claim counts in one period may also tend to generate high counts in another. That pattern does not require the individual periods to have large random fluctuations in common. It can arise because both observations are influenced by the same underlying risk characteristic. In credibility theory, the key is to separate that shared effect from variation that occurs from period to period.

Let $\Theta$ denote a risk's latent parameter, and let $X_1$ and $X_2$ be observations from two distinct periods for that risk. Write $m(\Theta)=E[X_t\mid\Theta]$ for the conditional mean in either period. The central question is whether $\operatorname{Cov}(X_1,X_2)$ equals the variance of these conditional means, and what assumptions make that equality true.

Definition: The conditional covariance of two observations given $\Theta$ is $\operatorname{Cov}(X_1,X_2\mid\Theta)$. It describes their remaining association when the risk parameter is held fixed. The variance of hypothetical means is $\operatorname{Var}(m(\Theta))$, the variance across risks of their conditional expected observations. The expected process variance is $E[\operatorname{Var}(X_t\mid\Theta)]$, the average within-risk variation across periods.

These quantities answer different questions. Conditional covariance is about association between periods after fixing the risk parameter. Unconditional covariance, $\operatorname{Cov}(X_1,X_2)$, is calculated across risks and their possible observations. Even if the periods have zero conditional covariance, their unconditional covariance can be positive because both conditional means vary from risk to risk.

The Covariance Decomposition

The general identity separates these two sources of covariance:

\operatorname{Cov}(X_1,X_2) = E[\operatorname{Cov}(X_1,X_2\mid\Theta)] + \operatorname{Cov}(E[X_1\mid\Theta],E[X_2\mid\Theta]).

For the standard repeated-period Bühlmann setup, the conditional means are the same function $m(\Theta)$. A further model assumption is that distinct period observations are conditionally uncorrelated given $\Theta$; conditional independence is a sufficient condition, but not necessary. Under that assumption, the first term is zero, and the second term is the variance of the hypothetical means.

Theorem: If $E[X_1\mid\Theta]=E[X_2\mid\Theta]=m(\Theta)$ and $E[\operatorname{Cov}(X_1,X_2\mid\Theta)]=0$, then $\operatorname{Cov}(X_1,X_2)=\operatorname{Var}(m(\Theta))$. If conditional covariance is zero for every possible $\Theta$, the expectation in the condition is automatically zero.

The result follows by adding and subtracting the conditional means inside each centered observation. Conditional on $\Theta$, write $X_t-\mu=(X_t-m(\Theta))+(m(\Theta)-\mu)$, where $\mu=E[X_t]=E[m(\Theta)]$. The cross terms have conditional expectation zero because $E[X_t-m(\Theta)\mid\Theta]=0$. Therefore:

\begin{aligned} E[(X_1-\mu)(X_2-\mu)\mid\Theta] &=\operatorname{Cov}(X_1,X_2\mid\Theta) +(m(\Theta)-\mu)^2,\\ \operatorname{Cov}(X_1,X_2) &=E[\operatorname{Cov}(X_1,X_2\mid\Theta)] +E[(m(\Theta)-\mu)^2]. \end{aligned}

The last expectation is $\operatorname{Var}(m(\Theta))$, which establishes the decomposition and the theorem. The derivation also shows exactly when the simplified result does not apply: a nonzero expected conditional covariance contributes an additional term.

This connects directly to the variance decomposition. If the periods have the same conditional variance structure, then $\operatorname{Var}(X_t)=E[\operatorname{Var}(X_t\mid\Theta)]+\operatorname{Var}(m(\Theta))$. Consequently, under zero conditional covariance, the covariance between periods equals the between-risk part of a single observation's variance, not its total variance.

\operatorname{Var}(X_t) = E[\operatorname{Var}(X_t\mid\Theta)] +\operatorname{Var}(m(\Theta)).

Worked Applications

Worked Example: Two Risk Classes With Normal Observations

A hypothetical insurer models a policy's annual claim amount in two distinct years. The latent risk type $\Theta$ is Low or High, each with probability $0.5$. Given Low, each annual amount has a normal distribution with mean $10 and variance 4; given High, each has a normal distribution with mean $20 and variance 4. The two annual amounts are conditionally independent given type. Find the unconditional covariance and correlation. There is no discounting or mortality assumption; the observations are annual claim amounts.

Method and assumptions. Use the covariance theorem because the conditional means are equal across years for a given type and conditional independence gives zero conditional covariance.

The unconditional mean is $0.5(10)+0.5(20)=15$. The hypothetical mean is 10 or 20, so its variance is \[ 0.5(10-15)^2+0.5(20-15)^2=0.5(25)+0.5(25)=25. \] Thus $\operatorname{Cov}(X_1,X_2)=25$.

As a direct check, conditional independence gives $E[X_1X_2\mid\Theta]=m(\Theta)^2$. Hence $E[X_1X_2]=0.5(10^2)+0.5(20^2)=50+200=250$, while $E[X_1]E[X_2]=15(15)=225$. The covariance is $250-225=25$, as before.

Each year's unconditional variance is the expected conditional variance plus the variance of hypothetical means: $4+25=29$. The correlation is therefore $25/\sqrt{29(29)}=25/29=0.8621$, to four decimal places. The check is consistent: covariance is positive and smaller than the common variance. The common mistake avoided here is setting unconditional covariance to zero merely because conditional covariance is zero.

Worked Example: Repeated Poisson Claim Counts

For a hypothetical portfolio segment, a risk's annual count in each of two years, conditional on its latent rate $\Theta$, has a Poisson distribution with mean $\Theta$. The rate equals 2 or 6, each with probability $0.5$, and the annual counts are conditionally independent. Find the covariance between the two annual counts and the variance of one annual count. No interest or mortality assumption is involved; the observations are annual counts.

Method and assumptions. The Poisson distribution with parameter $\Theta$ has conditional mean and variance $\Theta$. Conditional independence makes the conditional covariance zero, so the between-risk variance of $\Theta$ gives the covariance across years.

First, $E[\Theta]=0.5(2)+0.5(6)=4$. The variance of hypothetical means is \[ \operatorname{Var}(\Theta)=0.5(2-4)^2+0.5(6-4)^2 =0.5(4)+0.5(4)=4. \] Therefore, $\operatorname{Cov}(X_1,X_2)=4$.

For one year's count, the expected process variance is $E[\Theta]=4$. The variance decomposition gives $\operatorname{Var}(X_t)=4+4=8$. A direct covariance check uses $E[X_1X_2\mid\Theta]=\Theta^2$: $E[X_1X_2]=0.5(2^2)+0.5(6^2)=2+18=20$, and $E[X_1]E[X_2]=4(4)=16$. Thus covariance is $20-16=4$. The common mistake avoided is using the Poisson variance alone as the unconditional variance; that omits the variation in rates across risks.

Worked Example: When Conditional Covariance Is Not Zero

Consider a hypothetical paired annual measure with $\Theta=8$ or 12, each with probability $0.5$. Given $\Theta$, the pair $(X_1,X_2)$ has a bivariate normal distribution with mean vector $(\Theta,\Theta)$, variances 4 for both observations, and conditional covariance 1. Find the unconditional covariance and explain why it is not just the variance of hypothetical means. No discounting or mortality assumption applies.

Method and assumptions. Use the full covariance decomposition, retaining the conditional covariance term. The specified conditional covariance is 1 in either risk state, so its expectation is also 1.

The overall mean of each observation is $0.5(8)+0.5(12)=10$. The variance of hypothetical means is \[ 0.5(8-10)^2+0.5(12-10)^2=0.5(4)+0.5(4)=4. \] The covariance decomposition therefore gives $\operatorname{Cov}(X_1,X_2)=1+4=5$.

As a check, $E[X_1X_2\mid\Theta]=\Theta^2+1$, so $E[X_1X_2]=0.5(64+1)+0.5(144+1)=32.5+72.5=105$. Subtracting $E[X_1]E[X_2]=100$ gives 5. The common mistake avoided is applying the simplified theorem despite nonzero conditional covariance. Here the extra 1 reflects within-risk association, so the total covariance exceeds the variance of hypothetical means.

Interpretation and Exam Traps

Under the standard zero-conditional-covariance condition, the covariance between two periods from the same risk is a direct measure of how much the risks' expected outcomes vary. A risk's period-specific random deviations do not create covariance under that condition; the common conditional mean does. This interpretation is useful when separating within-risk process variation from between-risk heterogeneity in credibility models.

The conclusion is specific to repeated observations for the same risk under a shared risk parameter and the stated conditional covariance condition. It does not say that observations from different risks have this covariance, nor that all observations within one risk must have covariance equal to the variance of hypothetical means. If the periods have different conditional mean functions, or have residual dependence after conditioning on $\Theta$, use the general decomposition instead.

Identify the conditioning structure.
State the shared risk parameter and the two period-specific observations.

Find the conditional means.
Check whether both periods have the same conditional mean $m(\Theta)$.

Check conditional covariance.
If its expectation is zero, unconditional covariance reduces to $\operatorname{Var}(m(\Theta))$; otherwise retain the extra term.

Verify units and scale.
Covariance and variance of hypothetical means both have squared observation units. Do not confuse either with a standard deviation.

A frequent exam error is to calculate $\operatorname{Cov}(X_1,X_2\mid\Theta)=0$ and report zero as the answer for $\operatorname{Cov}(X_1,X_2)$. The incorrect step is treating a conditional covariance as though it were the unconditional covariance. Instead, add the covariance of the conditional means. In the normal example, the wrong answer would be 0; the correction is $0+\operatorname{Var}(m(\Theta))=25$.

Key takeaway: For two periods with the same conditional mean and zero expected conditional covariance, the unconditional covariance is the variance of hypothetical means. If conditional dependence remains after fixing the risk parameter, add its expected covariance rather than discarding it.

Check Your Understanding

For each problem, distinguish the conditional and unconditional quantities before calculating.

A hypothetical risk has $\Theta=5$ or 15 with equal probability. Given $\Theta$, two observations are independent normal variables with mean $\Theta$ and variance 9. Find their unconditional covariance and the variance of one observation. (Answer: covariance 25; variance 34.)
A risk's annual count is conditionally Poisson with mean $\Theta$, where $\Theta=1$ with probability 0.25 and $\Theta=5$ with probability 0.75. Annual counts are conditionally independent. Find the covariance between two years and the variance of one year's count. (Answer: covariance 3; variance 7.)
Two observations have the same conditional mean $m(\Theta)$, with $\operatorname{Var}(m(\Theta))=4$. Their expected conditional covariance is $-0.5$. Find their unconditional covariance. (Answer: 3.5.)
Repeated observations have identical conditional means and zero conditional covariance. The variance of one observation is 13 and its expected conditional variance is 5. Find the covariance between two distinct periods. (Answer: 8.)
A risk parameter takes values 0, 4, and 10 with probabilities 0.2, 0.5, and 0.3. Two observations have conditional mean $\Theta$ and zero conditional covariance. Calculate the variance of hypothetical means and the covariance between observations. (Answer: variance of hypothetical means 13; covariance 13.)

What You'll Learn