Worked Comparison of Common Working Correlations

Why Compare Working Correlations by Calculation?

In generalized estimating equations, a working correlation summarizes how responses from the same cluster are related. Independence, exchangeable, AR(1), and unstructured choices impose different patterns on that relationship. Their consequences are easiest to see by applying each choice to the same residuals, rather than treating the names as interchangeable options.

The calculations below use standardized residuals from a fixed mean-model fit and a common visit schedule. They illustrate method-of-moments estimates; in an actual alternating fitting procedure, the residuals may be updated after the mean model is refitted. A residual-based estimate is a candidate working structure, not proof that the population correlation has exactly that form.

Definition: A working correlation matrix \(R(\alpha)\) is a specified correlation matrix for the repeated responses within a cluster, indexed by one or more correlation parameters \(\alpha\). For a cluster with marginal variance scale \(\phi\) and diagonal variance matrix \(A\), its working covariance is commonly written \(V=\phi A^{1/2}R(\alpha)A^{1/2}\). The matrix \(R(\alpha)\) must be symmetric, have diagonal entries equal to 1, and be positive semidefinite; positive definiteness is generally required for the usual inverse-based GEE calculations.

One Residual Dataset, Four Estimates

Suppose there are four independent clusters, each observed at three equally spaced visits. Let \(e_{ij}\) denote the standardized residual for cluster \(i\) at visit \(j\). For this example, estimate each visit-pair correlation by the pooled residual cross-product

\hat r_{jk}= \frac{\sum_{i=1}^{4} e_{ij}e_{ik}} {\sqrt{\left(\sum_{i=1}^{4}e_{ij}^{2}\right) \left(\sum_{i=1}^{4}e_{ik}^{2}\right)}}.

This normalization makes the pairwise estimates ordinary pooled residual correlations. Consider the following residuals, where \(q=\sqrt{7}/4\), approximately \(0.6614\).

Cluster	Visit 1	Visit 2	Visit 3
1	1	1.4	0.75 + q
2	1	-0.2	-0.75 - q
3	-1	0.2	0.75 - q
4	-1	-1.4	-0.75 + q

Worked Example: Calculate All Three Pairwise Residual Correlations

The visit-1 residual vector is \(u=(1,1,-1,-1)^\mathsf{T}\). The visit-2 vector is \(0.6u+0.8v\), where \(v=(1,-1,1,-1)^\mathsf{T}\). The visit-3 vector is \(0.75v+qw\), where \(w=(1,-1,-1,1)^\mathsf{T}\). The vectors \(u,v,w\) are mutually orthogonal, and each has squared length 4. Since \(q^2=7/16\), the visit-3 squared length is \(4(0.75^2+q^2)=4(9/16+7/16)=4\). The visit-2 squared length is \(4(0.6^2+0.8^2)=4\); visit 1 also has squared length 4.

The visit-1 and visit-2 cross-product is \(u^\mathsf{T}(0.6u+0.8v)=0.6(4)+0.8(0)=2.4\). The visit-1 and visit-3 cross-product is \(u^\mathsf{T}(0.75v+qw)=0\). The visit-2 and visit-3 cross-product is \((0.6u+0.8v)^\mathsf{T}(0.75v+qw)=0.8(0.75)(4)=2.4\). Each denominator in the correlation formula is \(\sqrt{4\cdot4}=4\).

Therefore, \(\hat r_{12}=0.6\), \(\hat r_{13}=0\), and \(\hat r_{23}=0.6\). The full unstructured estimate uses all three pair-specific values; the other structures summarize or constrain them.

Worked Example: Obtain the Independence, Exchangeable, and AR(1) Estimates

Independence. The independence working structure sets every off-diagonal correlation to zero, so its estimated working matrix is the identity. It does not claim that the empirical residual correlations calculated above are all zero; it chooses not to represent within-cluster association in the working matrix.

Exchangeable. With three visits and equal numbers of observations for every pair, the pooled exchangeable method-of-moments estimate is the average of the three pairwise correlations: \[ \hat\rho_{\mathrm{ex}}=(0.6+0+0.6)/3=0.4. \] It assigns the same value to every off-diagonal entry.

AR(1). For equally spaced visits, an AR(1) working structure assigns correlation \(\rho^h\) to visits \(h\) steps apart. Using the two adjacent-visit correlations in this example gives \(\hat\rho_{\mathrm{AR(1)}}=(\hat r_{12}+\hat r_{23})/2=(0.6+0.6)/2=0.6\). The resulting model-implied visit-1 to visit-3 correlation is \(\hat\rho_{\mathrm{AR(1)}}^2=0.36\), not the empirical pairwise value 0. The difference is a visible consequence of imposing the AR(1) pattern.

The four resulting matrices are

\hat R_{\mathrm{ind}}= \begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix},\qquad \hat R_{\mathrm{ex}}= \begin{pmatrix}1&0.4&0.4\\0.4&1&0.4\\0.4&0.4&1\end{pmatrix},

\hat R_{\mathrm{AR(1)}}= \begin{pmatrix}1&0.6&0.36\\0.6&1&0.6\\0.36&0.6&1\end{pmatrix},\qquad \hat R_{\mathrm{un}}= \begin{pmatrix}1&0.6&0\\0.6&1&0.6\\0&0.6&1\end{pmatrix}.

Check Matrix Validity, Not Just Individual Entries

A collection of pairwise correlations is not automatically a valid joint correlation matrix. Every pairwise value may lie between \(-1\) and 1, yet the assembled matrix can fail to be positive semidefinite. For a symmetric matrix, positive definiteness can be checked by Sylvester’s criterion: all leading principal minors must be positive. The following checks establish positive definiteness for the four matrices in this example.

Worked Example: Verify the Four Estimated Matrices

The independence matrix has all eigenvalues equal to 1, so it is positive definite. The eigenvalues of a three-visit exchangeable matrix with off-diagonal value \(\rho\) are \(1-\rho\), \(1-\rho\), and \(1+2\rho\). At \(\rho=0.4\), these are \(0.6,0.6,1.8\), all positive.

For an AR(1) matrix with three visits, the leading principal minors are \(1\), \(1-\rho^2\), and \((1-\rho^2)^2\). At \(\rho=0.6\), they are \(1\), \(0.64\), and \(0.4096\), so the matrix is positive definite.

For the unstructured matrix, the first two leading principal minors are \(1\) and \(1-0.6^2=0.64\). Its determinant is \[ 1-0.6^2-0.6^2=0.28. \] All three leading principal minors are positive, so this unstructured estimate is also positive definite. In particular, the pairwise estimates \(0.6,0,0.6\) do form a valid matrix here—but that conclusion comes from the joint check, not merely from each value being in the allowable pairwise range.

Result: For three visits, the exchangeable matrix is positive definite exactly when \(-1/2<\rho<1\). The three-visit AR(1) matrix is positive definite when \(|\rho|<1\). An unstructured matrix has no single scalar bound that guarantees validity: its joint principal minors must be checked.

For the exchangeable condition, the eigenvalues given above are positive precisely when \(1-\rho>0\) and \(1+2\rho>0\), which together yield \(-1/2<\rho<1\). For the three-visit AR(1) case, Sylvester’s criterion gives \(1>0\), \(1-\rho^2>0\), and \((1-\rho^2)^2>0\), equivalent to \(|\rho|<1\). These checks concern the estimated matrices themselves; they do not establish that any one structure is the true population correlation.

What the Structures Imply for Precision

Changing the working correlation changes the weighting of observations within a cluster. When the working covariance is close to the true covariance, that weighting can improve the efficiency of regression estimation. When it is misspecified, it may be less efficient than another valid choice. Thus, there is no general rule that independence, a positive correlation, or a more flexible structure must always be best.

The basic efficiency result is clearest in a linear model with a fixed design. Let \(Y=X\beta+\varepsilon\), where \(X\) has full column rank, \(E[\varepsilon]=0\), and \(\operatorname{Cov}(\varepsilon)=\Sigma\) is known and positive definite. Among linear unbiased estimators, generalized least squares uses the covariance weighting.

Theorem: Under those conditions, the generalized least-squares estimator \[ \hat\beta_{\mathrm{GLS}}=(X^\mathsf{T}\Sigma^{-1}X)^{-1}X^\mathsf{T}\Sigma^{-1}Y \] has covariance no larger than that of any other linear unbiased estimator, in the positive-semidefinite ordering. No normality assumption is needed for this covariance comparison.

Write a general linear unbiased estimator.
It has the form \(AY\) for a matrix \(A\), and unbiasedness for every \(\beta\) requires \(AX=I\).

Separate the GLS estimator.
Let \(A_0=(X^\mathsf{T}\Sigma^{-1}X)^{-1}X^\mathsf{T}\Sigma^{-1}\), and write \(C=A-A_0\). Since both estimators are unbiased, \(CX=0\).

Compare their covariances.
Expanding \(A\Sigma A^\mathsf{T}\), the cross terms vanish because \(A_0\Sigma C^\mathsf{T}=(X^\mathsf{T}\Sigma^{-1}X)^{-1}X^\mathsf{T}C^\mathsf{T}=0\), and likewise \(C\Sigma A_0^\mathsf{T}=0\). Thus \(A\Sigma A^\mathsf{T}-A_0\Sigma A_0^\mathsf{T}=C\Sigma C^\mathsf{T}\), which is positive semidefinite because \(\Sigma\) is positive definite. Therefore GLS has covariance no larger than that of \(AY\). \(\square\)

In a GEE setting, the working covariance plays the role of \(\Sigma\) in this weighting. If it matches the true covariance, the weighting can attain the corresponding efficiency advantage in the regular linear-model setting. Under misspecification, the GEE coefficient estimator still has a variance determined by both the working weights and the true covariance; the GLS comparison does not say that a misspecified working structure is optimal.

Worked Example: Correlation Can Help One Contrast and Hurt Another

Suppose a cluster has three responses with common variance \(\sigma^2\) and true exchangeable correlation \(\rho=0.4\). For the within-cluster average \(\bar Y=(Y_1+Y_2+Y_3)/3\), the variance is \[ \operatorname{Var}(\bar Y)=\frac{1}{9}\left(3\sigma^2+6(0.4\sigma^2)\right) =\frac{5.4}{9}\sigma^2=0.6\sigma^2. \] Under independence, the corresponding variance would be \(3\sigma^2/9=\sigma^2/3\). Thus the independence covariance would understate the variance of this average if the true correlation were exchangeable with \(\rho=0.4\).

Now consider the visit contrast \(C=Y_1-Y_2\). Its variance is \[ \operatorname{Var}(C)=\operatorname{Var}(Y_1)+\operatorname{Var}(Y_2) -2\operatorname{Cov}(Y_1,Y_2) =2\sigma^2(1-0.4)=1.2\sigma^2. \] Under independence it would be \(2\sigma^2\). Positive exchangeable correlation therefore increases the variance of the common average relative to independence, but decreases the variance of this difference. Precision depends on the regression coefficient or contrast being estimated, not only on whether correlation is positive.

Choosing What to Compare

The residual calculations provide a practical comparison, not an automatic selection rule. In this example, independence discards the observed pairwise pattern; exchangeable compresses it to one average value; AR(1) imposes a decay pattern that predicts a lag-two value of \(0.36\); and unstructured retains all three pairwise estimates. The unstructured option describes the residual pattern most directly here, but it requires estimating more correlations and still needs a valid matrix.

A useful comparison keeps three questions separate:

What pattern does the data suggest? Inspect pairwise residual moments and consider visit spacing and study design.
Does the proposed structure define a valid matrix? Check positive definiteness or positive semidefiniteness, rather than checking entries one at a time.
What does the choice imply for the target coefficient? Correlation changes within-cluster weighting, and its efficiency effect depends on the mean-model design and the estimand.

Common Pitfall

A tempting but incorrect argument is: “Every estimated pairwise correlation is between \(-1\) and 1, so the unstructured matrix is valid.” The range check is necessary for each pair, but it is not sufficient for the joint matrix. The correction is to assemble the matrix and check its eigenvalues or principal minors; here the determinant \(0.28\) confirms validity for this particular estimate.

A second mistake is to treat the model-implied AR(1) lag-two correlation as another free estimate. In an AR(1) structure it is constrained to be \(\rho^2\); here it is \(0.36\), even though the empirical lag-two residual correlation is zero. Keeping the observed moments distinct from the imposed structure makes the comparison interpretable.

Key takeaway: Calculate the residual moments first, apply each structure’s constraints explicitly, and then verify that the resulting matrix is valid. Working-correlation choice affects regression weighting and precision, but its efficiency consequences depend on the covariance truth and the coefficient being estimated.

Check Your Understanding

Use the calculations and distinctions in this tutorial to answer the following questions.

For the residual moments \(\hat r_{12}=0.6\), \(\hat r_{13}=0\), and \(\hat r_{23}=0.6\), calculate the exchangeable estimate when all three pair types have equal weight.
Under the three-visit AR(1) estimate based on adjacent pairs, what correlation is implied between visits 1 and 3? Why need it differ from the empirical lag-two residual correlation?
Why is checking that each unstructured pairwise estimate lies between \(-1\) and 1 insufficient to establish that the matrix is valid?
For an exchangeable correlation of \(0.4\), compare the variance of the three-visit average with its variance under independence, assuming equal marginal variances.
Does positive within-cluster correlation necessarily increase the variance of every contrast? Use a difference between two visits to explain.