Exchangeable Correlation and Its Validity Bounds

Why Exchangeable Correlation Has a Size Constraint

In a working-correlation model, the exchangeable choice treats every pair of distinct responses in a cluster alike: each pair has the same working correlation. This simple assumption gives a matrix with only one off-diagonal parameter, but not every numerical value of that parameter is possible. The admissible range depends on the number of responses in the cluster. With unequal cluster sizes, a common parameter must satisfy the constraints imposed by the largest cluster.

The key calculation is an eigenvalue calculation. It separates the direction in which all entries move together from the directions that contrast observations within the cluster. Positive definiteness requires positive variance in both kinds of directions.

From Permutation Symmetry to the Matrix

Consider a random response vector $Y=(Y_1,\ldots,Y_m)^\mathsf{T}$ with finite second moments. A permutation of its coordinates can be represented by a permutation matrix $P$. If the distribution of $Y$ is unchanged by every permutation, then its covariance matrix is unchanged as well: $P\operatorname{Cov}(Y)P^\mathsf{T}=\operatorname{Cov}(Y)$. The same covariance conclusion follows directly if covariance invariance is assumed, without requiring full distributional exchangeability.

Permuting coordinates can move any diagonal entry of the covariance matrix to any other diagonal position, so all marginal variances are equal. It can also move any ordered pair of distinct coordinates to any other such pair, so all covariances between distinct coordinates are equal. Write the common marginal variance as $\sigma^2$ and the common off-diagonal covariance as $c$. To standardize and obtain a correlation matrix, we must additionally assume $\sigma^2>0$.

Definition: For a cluster of size $m$, an exchangeable correlation matrix with common off-diagonal correlation $\rho$ is the $m\times m$ matrix $$ R_m(\rho)= \begin{pmatrix} 1 & \rho & \cdots & \rho\\ \rho & 1 & \cdots & \rho\\ \vdots & \vdots & \ddots & \vdots\\ \rho & \rho & \cdots & 1 \end{pmatrix} =(1-\rho)I_m+\rho J_m, $$ where $I_m$ is the identity matrix and $J_m=\mathbf{1}_m\mathbf{1}_m^\mathsf{T}$ is the matrix of ones. If all marginal variances equal a positive $\sigma^2$, standardizing the covariance matrix gives this form with $\rho=c/\sigma^2$.

The formula shows the symmetry explicitly: every diagonal entry is $(1-\rho)+\rho=1$, and every off-diagonal entry is $\rho$. The parameter describes the working correlation between any two distinct responses in the same cluster. It does not require responses in different clusters to share this correlation.

Worked Example: Building the Matrix From a Common Correlation

Suppose a cluster contains four responses and the working correlation between each pair of distinct responses is $0.2$. Then $m=4$, $\rho=0.2$, and $$ R_4(0.2)= \begin{pmatrix} 1&0.2&0.2&0.2\\ 0.2&1&0.2&0.2\\ 0.2&0.2&1&0.2\\ 0.2&0.2&0.2&1 \end{pmatrix}. $$

Equivalently, $R_4(0.2)=0.8I_4+0.2J_4$. The diagonal entries are $0.8+0.2=1$; the off-diagonal entries are $0+0.2=0.2$. This verifies that the expression produces the required diagonal and common pairwise correlation.

Positive-Definiteness Bounds for One Cluster

A correlation matrix used as a nonsingular working correlation must be positive definite. For $R_m(\rho)$, its eigenvalues can be read from two types of vectors. The vector $\mathbf{1}_m$ points in the direction where all coordinates are equal. Any vector orthogonal to $\mathbf{1}_m$ represents a contrast whose coordinates sum to zero.

Theorem: For $m\geq 2$, the exchangeable matrix $R_m(\rho)$ is positive definite if and only if $$ -\frac{1}{m-1}<\rho<1. $$ It is positive semidefinite if and only if $$ -\frac{1}{m-1}\leq\rho\leq 1. $$

Proof. Since $J_m\mathbf{1}_m=m\mathbf{1}_m$, $$ R_m(\rho)\mathbf{1}_m =\bigl((1-\rho)+m\rho\bigr)\mathbf{1}_m =\bigl(1+(m-1)\rho\bigr)\mathbf{1}_m. $$ Thus the eigenvalue in the all-equal direction is $1+(m-1)\rho$. If $z^\mathsf{T}\mathbf{1}_m=0$, then $J_mz=0$, so $$ R_m(\rho)z=(1-\rho)z. $$ The subspace orthogonal to $\mathbf{1}_m$ has dimension $m-1$, so $1-\rho$ is an eigenvalue with multiplicity $m-1$. These account for all $m$ eigenvalues.

A real symmetric matrix is positive definite exactly when all its eigenvalues are positive. Here that means both $1+(m-1)\rho>0$ and $1-\rho>0$. Rearranging gives $\rho>-1/(m-1)$ and $\rho<1$. Replacing strict positivity by nonnegativity gives the positive-semidefiniteness bounds. $\square$

Worked Example: Checking a Negative Correlation at the Lower Bound

For clusters of size $m=5$, the positive-definiteness interval is $$ -\frac{1}{5-1}<\rho<1, \qquad\text{or}\qquad -0.25<\rho<1. $$ Take $\rho=-0.24$, which lies inside this interval. The eigenvalue in the all-equal direction is $$ 1+(5-1)(-0.24)=1-0.96=0.04. $$ The other four eigenvalues are $$ 1-(-0.24)=1.24. $$ All five eigenvalues are positive, so $R_5(-0.24)$ is positive definite.

At the lower boundary, $\rho=-0.25$, the all-equal eigenvalue is $1+4(-0.25)=0$, while the other four eigenvalues are $1.25$. The matrix is positive semidefinite but singular, so it cannot be inverted as a positive-definite working correlation. A value below $-0.25$ makes the all-equal eigenvalue negative and the matrix is not positive semidefinite.

The bounds also explain why the allowable negative correlation depends on cluster size. As $m$ grows, the lower limit $-1/(m-1)$ moves closer to zero. A single strongly negative pairwise correlation cannot be assigned to every pair in an arbitrarily large cluster while keeping the full matrix positive definite.

Worked Example: A Positive Correlation and a Singular Upper Boundary

For a cluster of size $m=3$, consider $\rho=0.6$. Its eigenvalues are $$ 1+(3-1)(0.6)=2.2 \quad\text{and}\quad 1-0.6=0.4 $$ with the second eigenvalue repeated twice. Each is positive, so $R_3(0.6)$ is positive definite.

Now set $\rho=1$. The eigenvalue in the all-equal direction is $1+2(1)=3$, and each contrast eigenvalue is $1-1=0$. The matrix is positive semidefinite but singular. In fact, all rows are identical: every response is assigned correlation one with every other response. Thus $\rho=1$ is a valid semidefinite boundary, not a positive-definite value for any cluster of size at least two.

One Common Correlation Across Unequal Cluster Sizes

Suppose clusters have sizes $m_1,\ldots,m_K$, and a single working parameter $\rho$ is used for every within-cluster pair. For each cluster with $m_k\geq2$, positive definiteness requires $$ -\frac{1}{m_k-1}<\rho<1. $$ The parameter must satisfy all these intervals at once. If at least one cluster has size two or more, let $M=\max_k m_k$. Since $-1/(m-1)$ increases as $m$ increases, the most restrictive lower bound comes from the largest cluster. Therefore the common parameter must obey $$ -\frac{1}{M-1}<\rho<1. $$ For positive semidefiniteness, the corresponding inclusive bounds are $-1/(M-1)\leq\rho\leq1$.

A cluster of size one contributes only the matrix $(1)$; it places no restriction on $\rho$, because it contains no pair of distinct responses. If every cluster has size one, the working matrices do not involve $\rho$ at all, so these matrices alone impose no bound on it and cannot identify it.

Worked Example: Finding the Common Bound for Unequal Sizes

Suppose the observed cluster sizes are $2$, $4$, and $7$, and the same $\rho$ is to be used in every cluster. The separate positive-definiteness intervals are $$ \begin{aligned} m=2 &: \quad -1<\rho<1,\\ m=4 &: \quad -\frac{1}{3}<\rho<1,\\ m=7 &: \quad -\frac{1}{6}<\rho<1. \end{aligned} $$ Their intersection is $-1/6<\rho<1$, determined at the lower end by the size-seven cluster.

For example, $\rho=-0.15$ satisfies this common interval. In a size-seven cluster, the all-equal eigenvalue is $1+6(-0.15)=0.10$, and each contrast eigenvalue is $1.15$. In a size-four cluster, the corresponding eigenvalues are $1+3(-0.15)=0.55$ and $1.15$ (the latter repeated three times). For size two they are $0.85$ and $1.15$. All are positive, verifying positive definiteness for every listed size.

By contrast, $\rho=-0.20$ passes the size-four lower bound because $-0.20>-1/3$, but fails for size seven because $-0.20<-1/6$. Indeed, its size-seven all-equal eigenvalue is $1+6(-0.20)=-0.20$, so that cluster's matrix is not positive semidefinite.

Common Mistakes and Why the Bounds Matter

Checking only that each correlation lies between $-1$ and $1$. Pairwise bounds are necessary but not sufficient for a whole matrix. For $m=5$, $\rho=-0.3$ is between $-1$ and $1$, yet $1+4(-0.3)=-0.2$, so the matrix is not positive semidefinite.
Using the smallest cluster to set the common lower bound. The largest cluster imposes the strictest lower bound. With sizes $2$ and $7$, the relevant lower bound is $-1/6$, not $-1$.
Treating a boundary value as nonsingular. At either endpoint for $m\geq2$, at least one eigenvalue is zero. A boundary matrix may be positive semidefinite, but it is not positive definite and cannot be inverted.
Standardizing when the common marginal variance is zero. Permutation-invariant covariances imply equal variances and equal off-diagonal covariances, but a correlation matrix is obtained by standardization only if the common marginal variance is positive.

These checks matter in GEE because working-correlation matrices are used in cluster-level calculations, including matrix inversions. An invalid or singular choice can make those calculations impossible or unstable. The bounds are algebraic validity conditions; they do not establish that the exchangeable structure is a good description of the dependence in a particular study.

Key takeaway: For a cluster of size $m\geq2$, exchangeable positive definiteness requires $-1/(m-1)<\rho<1$. With one common parameter across unequal sizes, use the largest cluster size $M$, giving $-1/(M-1)<\rho<1$.

Check Your Understanding

Use the eigenvalue conditions to answer each question.

For $m=6$, what is the strict positive-definiteness interval for an exchangeable correlation matrix?
For $m=4$ and $\rho=-1/3$, is the matrix positive definite, positive semidefinite but singular, or not positive semidefinite? Verify using its eigenvalues.
Clusters have sizes $1$, $3$, and $8$. Find the interval required for a common $\rho$ to make every cluster matrix positive definite.
Why does permutation-invariant covariance alone not guarantee that standardizing produces a correlation matrix?
If every cluster has size one, what restrictions do the cluster working-correlation matrices place on $\rho$?