Why Conditional and Unconditional Variance Differ
A claim observation varies for two reasons. Even for a fixed risk, random claims produce process variation. In addition, risks may have different expected claim amounts or frequencies, so the risk-specific conditional mean also varies across the portfolio. Unconditional variance includes both sources.
The distinction matters in credibility calculations: repeated observations from the same risk share its underlying risk characteristics. They may be independent once those characteristics are fixed, yet still be dependent when viewed across the whole portfolio. Ignoring that shared risk can understate the variance of an average or miss covariance between periods.
Here, “hypothetical means” means the possible expected values for different risks, not observed sample averages. The risk parameter can be a category, such as a risk type, or a quantity that determines the conditional claim distribution. It is treated as fixed for observations from one risk, but variable across the population of risks.
The Law of Total Variance
The result applies whenever \(X\) has a finite second moment and the conditional mean and variance given \(\Theta\) are defined. No particular distribution is required.
The first term averages the variability remaining after the risk is known. The second term measures how much the conditional expectation changes from risk to risk. Both are nonnegative; the unconditional variance is at least as large as the expected conditional variance.
To justify the decomposition, write \(m(\Theta)=E[X\mid\Theta]\) and add and subtract this conditional mean inside the squared deviation from \(E[X]\). The cross-product has expectation zero because, conditional on \(\Theta\), \(X-m(\Theta)\) has mean zero.
For the cross-product, condition first on \(\Theta\). Since \(m(\Theta)-E[X]\) is then fixed, \(E[(X-m(\Theta))(m(\Theta)-E[X])\mid\Theta]\) equals \((m(\Theta)-E[X])E[X-m(\Theta)\mid\Theta]=0\). Taking expectations gives zero, establishing the last equality.
A useful companion result concerns two observations \(X_1\) and \(X_2\) from the same risk. If they are conditionally independent given \(\Theta\), then their conditional covariance is zero, but their unconditional covariance need not be.
The shared conditional mean creates the covariance: risks with higher expected outcomes tend to have higher outcomes in both observations. Conditional independence does not imply unconditional independence when the observations share a random risk parameter.
Worked Calculations
Worked Example: Variance of a Multi-Year Average Count
A hypothetical portfolio has two equally likely risk types. Conditional on type, each year's claim count is Poisson with mean 2 for the first type and 6 for the second. Annual counts are independent given the type. Find the unconditional variance of the four-year average count and the covariance of counts in two different years.
Method and assumptions. Let \(\Theta\) identify the type and let \(N_t\) be the count in year \(t\). A Poisson variable with mean \(\lambda\) has conditional variance \(\lambda\). Write \(\bar N=(N_1+N_2+N_3+N_4)/4\). Given the type, the annual counts are independent, so their conditional variances add.
The conditional mean of \(\bar N\) is the type's annual mean count, either 2 or 6. Thus \(E[\bar N]=(2+6)/2=4\), and \[ E[\operatorname{Var}(\bar N\mid\Theta)] =\frac12\left(\frac{2}{4}+\frac{6}{4}\right)=1. \] For the second component, \[ \operatorname{Var}(E[\bar N\mid\Theta]) =\frac12(2-4)^2+\frac12(6-4)^2=4. \] Therefore, \[ \operatorname{Var}(\bar N)=1+4=5. \]
For two distinct years, say years 1 and 2, conditional independence gives \(E[\operatorname{Cov}(N_1,N_2\mid\Theta)]=0\). Both conditional means equal the same type-specific count mean, so \[ \operatorname{Cov}(N_1,N_2)=\operatorname{Var}(\lambda_\Theta) =\frac12(2-4)^2+\frac12(6-4)^2=4. \] The results are consistent with the average variance: each annual count has unconditional variance \(E[\lambda_\Theta]+\operatorname{Var}(\lambda_\Theta)=4+4=8\), so \(\operatorname{Var}(\bar N)=(4(8)+12(4))/16=5\). The 12 covariance terms arise from the 12 ordered pairs of distinct years.
Common mistake avoided: Dividing the between-type variance by four would be wrong. Conditional on type, the four-year average has mean \(\lambda_\Theta\), not \(\lambda_\Theta/4\). Averaging reduces process variance, but it does not eliminate variation in the shared type-specific mean.
Worked Example: Unconditional Variance of Compound Poisson Loss
For a hypothetical policy group, the annual claim count \(N\), conditional on risk type, is Poisson with mean 1 or 3, with each type equally likely. Each claim amount \(Y\) is independently \$50 or \$150 with equal probability, independently of the count and type. Find the unconditional mean and variance of aggregate annual loss \(S=\sum_{j=1}^{N}Y_j\).
Method and assumptions. For a compound Poisson sum with Poisson frequency parameter \(\lambda\) and independent identically distributed severities, \(E[S\mid\Theta]=\lambda E[Y]\) and \(\operatorname{Var}(S\mid\Theta)=\lambda E[Y^2]\). The second formula uses the raw second moment of severity.
First calculate the severity moments: \[ E[Y]=\frac{50+150}{2}=100,\qquad E[Y^2]=\frac{50^2+150^2}{2}=\frac{2{,}500+22{,}500}{2}=12{,}500. \] Thus, for type 1, \(E[S\mid\Theta]=1(100)=100\) and \(\operatorname{Var}(S\mid\Theta)=1(12{,}500)=12{,}500\). For type 2, those values are 300 and 37,500.
The unconditional mean is \((100+300)/2=200\). The expected process variance is \[ E[\operatorname{Var}(S\mid\Theta)] =\frac12(12{,}500)+\frac12(37{,}500)=25{,}000. \] The variance of conditional means is \[ \operatorname{Var}(E[S\mid\Theta]) =\frac12(100-200)^2+\frac12(300-200)^2=10{,}000. \] Therefore, \(\operatorname{Var}(S)=25{,}000+10{,}000=\$35{,}000\). The final variance is in dollars squared.
Check. The overall mean frequency is 2. The unconditional frequency variance is \(E[\lambda]+\operatorname{Var}(\lambda)=2+1=3\). Using \(E[S^2]\) for a compound sum gives \(\operatorname{Var}(S)=E[N]\operatorname{Var}(Y)+\operatorname{Var}(N)(E[Y])^2=2(2{,}500)+3(10{,}000)=35{,}000\), matching the decomposition.
Common mistake avoided: Using \(\lambda\operatorname{Var}(Y)\) as the conditional aggregate variance omits the variation in the Poisson count. For a compound Poisson sum the correct expression is \(\lambda E[Y^2]\), which here equals \(\lambda(12{,}500)\).
Worked Example: Process Variation and Shared-Risk Covariance
A hypothetical population has two equally likely risk types. For each exposure, a claim-cost observation \(X\), measured in dollars, is \$80 or \$120 with equal probability for type A, and \$170 or \$230 with equal probability for type B. Two observations from the same risk are independent given its type. Find the unconditional variance of one observation and the covariance of the two observations.
Method. For type A, the conditional mean is \((80+120)/2=100\), and the conditional variance is \(((80-100)^2+(120-100)^2)/2=400\). For type B, the mean is \((170+230)/2=200\), and the variance is \(((170-200)^2+(230-200)^2)/2=900\).
The overall mean is \((100+200)/2=150\). The expected process variance is \((400+900)/2=650\), and the variance of hypothetical means is \[ \frac12(100-150)^2+\frac12(200-150)^2=2{,}500. \] Hence \(\operatorname{Var}(X)=650+2{,}500=3{,}150\) dollars squared.
For the two observations \(X_1,X_2\), conditional independence makes their conditional covariance zero. Their conditional means both equal 100 for type A or 200 for type B. Consequently, \[ \operatorname{Cov}(X_1,X_2)=\operatorname{Var}(m(\Theta))=2{,}500 \] dollars squared. The covariance is positive because the observations share the same risk type.
Check. The marginal distribution places probability \(1/4\) on each of \$80, \$120, \$170, and \$230. Its mean is \$150, and its variance is \((4{,}900+900+400+6{,}400)/4=3{,}150\), agreeing with the decomposition.
Interpreting the Components Correctly
The formulas can be summarized as follows. The conditional distribution describes outcomes for a known risk; averaging its variance gives the process component. The conditional mean describes the expected outcome for each risk; its variance gives the between-risk component.
| Quantity | What varies? | Interpretation |
|---|---|---|
| \(E[\operatorname{Var}(X\mid\Theta)]\) | Outcomes around a fixed risk's mean | Expected process variance |
| \(\operatorname{Var}(E[X\mid\Theta])\) | Conditional means across risks | Variance of hypothetical means |
| \(\operatorname{Var}(X)\) | Both sources together | Unconditional variance |
For \(n\) observations from one risk, conditionally independent with the same conditional mean and variance, the average \(\bar X\) satisfies \[ E[\bar X\mid\Theta]=m(\Theta),\qquad \operatorname{Var}(\bar X\mid\Theta)=\frac{s^2(\Theta)}{n}. \] Applying total variance gives \[ \operatorname{Var}(\bar X) =\frac{E[s^2(\Theta)]}{n}+\operatorname{Var}(m(\Theta)). \] Only the process component shrinks by \(n\). The shared risk-specific mean remains, so increasing the number of observations does not drive the unconditional variance to zero unless the variance of those means is zero.
This conclusion depends on the shared-risk and conditional-independence assumptions. If observations come from different risks, their conditional means need not be the same. If observations remain dependent even after conditioning on \(\Theta\), their conditional covariance must also be included; the simple \(s^2(\Theta)/n\) formula then does not apply.
Common Exam Traps
- Using conditional variance as the total variance. This drops variation in risk-specific means. Apply both terms of the law of total variance.
- Dividing every component by the number of observations. Averaging reduces conditional process variance under conditional independence; it does not reduce the variance of a shared conditional mean.
- Assuming conditional independence means zero unconditional covariance. Shared risk parameters can induce covariance, as the covariance formula shows.
- Using severity variance in place of the raw second moment for a compound Poisson sum. The conditional aggregate variance is \(\lambda E[Y^2]\), not \(\lambda\operatorname{Var}(Y)\).
These distinctions are aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus. In a calculation, identify what is random conditional on the risk, state the conditional mean and variance, and then average those quantities over the risk distribution. This prevents process variation and variation between risks from being combined incorrectly.
Check Your Understanding
For each problem, identify the conditional process component and the variation in conditional means before combining them.
- A risk type is equally likely to have a Poisson annual count mean of 2 or 6. Find the unconditional variance of one year's count. (Answer: 8)
- A risk has an equally likely chance of having Poisson annual count mean 1 or 4. Conditional annual counts are independent. Find the variance of the average count over three years. (Answer: \(37/12\), or approximately 3.0833)
- Conditional on risk type, annual aggregate loss is compound Poisson with frequency mean 2 or 5, equally likely. Severity is independently \$10 or \$30 with equal probability. Find the unconditional mean and variance of annual aggregate loss. (Answer: Mean \$70; variance 2,650 dollars squared)
- Two conditionally independent observations from one risk have conditional means \$40 or \$100, equally likely, and conditional variances 100 or 400 dollars squared, respectively. Find the unconditional variance of one observation and the covariance of the two observations. (Answer: Variance 1,150 dollars squared; covariance 900 dollars squared)
- A hypothetical risk group has Poisson frequency mean 1 with probability \(1/4\) and 3 with probability \(3/4\). Severity is \$20 or \$40 with equal probability, independently of frequency and risk type. Find the unconditional mean and variance of annual aggregate loss. (Answer: Mean \$75; variance 3,175 dollars squared)