Dependence Between Observations and Credibility

Why Dependence Matters in Credibility

A class with several years of experience can appear to have a large volume of data. But if unusually high or low outcomes tend to persist across years for reasons not represented by the risk parameter, those observations repeat some of the same information. Treating them as independent can make the observed class mean seem more precise than it is, and can produce excessive credibility.

The key question is not whether observations from the same class are correlated overall. In the Bühlmann model, they generally are correlated overall because they share an unknown risk parameter. The question is whether they remain dependent after conditioning on that parameter. This tutorial examines that distinction and shows how within-class dependence changes the information in experience.

Conditional independence does not mean unconditional independence. Even when the standard model holds, the observations share \(\Theta\). For distinct observations, the law of total covariance gives \(\operatorname{Cov}(X_j,X_k)=a\) under conditional independence. Thus, a nonzero correlation calculated from all observations together does not by itself show a violation. A violation occurs when process errors remain associated after conditioning on the common risk parameter.

How Conditional Dependence Changes the Variance of the Mean

Suppose the observations have the same conditional mean \(m(\Theta)\), the same conditional variance \(s^2(\Theta)\), and a common conditional covariance between distinct observations. Define \(c=E[\operatorname{Cov}(X_j,X_k\mid\Theta)]\) for \(j\ne k\). Then the unconditional variance of one observation is \(a+v\), and the covariance of two distinct observations is \(a+c\).

For a clear effective-sample-size adjustment, consider the homogeneous exchangeable case: conditional variance is the constant \(\sigma^2\), and each pair of distinct observations has conditional covariance \(\rho\sigma^2\). Here \(\rho\) is the within-class conditional correlation. The covariance matrix must be valid; for \(n\) exchangeable observations, \(-1/(n-1)\leq \rho\leq 1\). In the common positive-dependence case, \(0<\rho\leq1\).

The factor \(1+(n-1)\rho\) is the inflation in process variance relative to independent observations. Positive conditional correlation increases the variance of the mean, so the mean carries less information about \(m(\Theta)\) than \(n\) independent observations would.

Derivation of the Effective Sample Size and Credibility

The adjustment follows directly from the variance of a sum and the linear projection used in Bühlmann credibility. Conditional on \(\Theta\), each observation has variance \(\sigma^2\), and each of the \(n(n-1)\) ordered pairs of distinct observations contributes covariance \(\rho\sigma^2\). Therefore,

Because \(E[\bar X\mid\Theta]=m(\Theta)\), the law of total variance adds \(\operatorname{Var}(m(\Theta))=a\), giving the stated unconditional variance. Also, \(\operatorname{Cov}(m(\Theta),\bar X)=a\): conditional process errors have mean zero, so they contribute no covariance with \(m(\Theta)\). The best linear predictor of \(m(\Theta)\) from \(\bar X\) has slope equal to covariance divided by variance. Centering at \(\mu\), that slope is

Rearranging the denominator gives the effective-sample-size form. When \(\rho=0\), \(n_{\mathrm{eff}}=n\), so the standard Bühlmann factor is recovered. When \(\rho>0\), \(n_{\mathrm{eff}}<n\); as \(n\) grows with fixed positive \(\rho\), \(n_{\mathrm{eff}}\) approaches \(1/\rho\), rather than growing without bound.

Estimating a Risk Mean Versus Predicting a Future Observation

The credibility adjustment above concerns \(m(\Theta)\), the risk’s expected outcome. It also applies to predicting a future observation if that future observation’s process error is conditionally independent of the historical process errors. But conditional dependence may extend from the observed periods to the future period. In that case, a future observation can contain information from the historical process errors as well as from \(\Theta\). The target and the dependence structure must be specified before choosing a formula.

For the homogeneous exchangeable model, if a future observation \(X_{n+1}\) has the same conditional covariance \(\rho\sigma^2\) with every historical observation, then \(\operatorname{Cov}(X_{n+1},\bar X)=a+\rho\sigma^2\). The best linear predictor based on \(\bar X\) is \(\mu+B(\bar X-\mu)\), where \(B=\operatorname{Cov}(X_{n+1},\bar X)/\operatorname{Var}(\bar X)\). This is not the same as the credibility factor for \(m(\Theta)\): part of the predictive relationship comes from persistent process errors.

Distinguishing Shared Risk From Residual Dependence

A practical diagnostic must distinguish between two sources of dependence. Shared variation in \(\Theta\) is part of the Bühlmann model; persistent association after conditioning on \(\Theta\) is the issue addressed here. For example, repeated observations may share a temporary environmental or operational effect that is not included in the latent risk parameter. If that effect persists across periods, the conditional errors can be correlated. A raw correlation across all risks or periods combines both sources and cannot isolate the violation by itself.

Common Mistakes and Practical Limits

• Applying the standard formula to conditionally dependent observations. The standard \(Z=na/(na+v)\) relies on conditional independence. With positive exchangeable dependence, replace the independent process variance contribution by \(\sigma^2[1+(n-1)\rho]/n\).
• Calling any observed correlation a model failure. Observations can be unconditionally correlated because they share \(\Theta\). The assumption to assess is conditional independence, not unconditional independence.
• Using the adjusted factor for the wrong target. The effective-sample-size formula estimates \(m(\Theta)\). If the future process error is correlated with past errors, prediction of that future outcome requires its covariance with the observed data, as in the preceding example.
• Assuming exchangeability without justification. A single \(\rho\) for every pair is a simplifying model, not a universal description. For serial or unequal dependence, use the appropriate covariance structure; a single effective sample size may not adequately describe it.

For unequal weights or non-exchangeable observations, the variance of a weighted mean must include every relevant covariance term. If weights are \(w_j\), the conditional process variance contribution is \( \operatorname{Var}(\sum_j w_jX_j\mid\Theta)\), with the weights normalized to sum to one. Ignoring off-diagonal covariance terms can overstate the information just as it does in the equal-weight case. The exchangeable formulas are useful when their assumptions are credible, but should not be applied mechanically.

Dependence matters because credibility converts information into a weight. If repeated observations largely echo a shared process disturbance, treating each as new independent evidence overweights the class experience. A sound assessment identifies the target, separates shared-risk variation from conditional process variation, and uses a covariance model consistent with the data and the intended prediction.