Independence Working Correlation for Unequal Cluster Sizes

What Independence Means in GEE

Repeated responses from one person, clinic, or other sampling unit may be correlated. A working correlation specifies how the GEE will represent that within-cluster dependence when constructing its estimating equation. The independence choice is the simplest: it sets the working correlation between distinct responses in a cluster to zero. It remains well-defined when clusters have different numbers of observations, because each cluster receives its own matrix dimension.

This is a choice about the working covariance used in estimation; it is not, by itself, evidence that the observed responses are actually independent. Correct specification of the marginal mean remains central. If within-cluster dependence is present, it can affect efficiency and uncertainty calculations, even when the mean parameters can still be estimated consistently.

Definition: For cluster $i$, let $m_i$ be its number of observed responses and let $R_i$ be its $m_i$-by-$m_i$ working correlation matrix. The independence working correlation is $R_i=I_{m_i}$, where $I_{m_i}$ is the identity matrix. Thus, its diagonal entries are 1 and all off-diagonal entries are 0. The matrix is constructed separately for each cluster; unequal cluster sizes do not require padding the matrices to a common dimension.

Write $Y_i=(Y_{i1},\ldots,Y_{im_i})^T$ for the response vector, $\mu_i(\beta)=E[Y_i]$ for the modeled marginal mean vector, and $D_i=\partial\mu_i(\beta)/\partial\beta^T$ for its derivative matrix. Let $A_i$ be the diagonal matrix of marginal variance functions, with entries $v\{\mu_{ij}(\beta)\}$. With dispersion parameter $\phi>0$, the usual working covariance is $V_i=\phi A_i^{1/2}R_iA_i^{1/2}$. For independence, it becomes diagonal.

Result: Under independence working correlation, $V_i=\phi A_i$, and the cluster contribution to the GEE is $D_i^T(\phi A_i)^{-1}\{Y_i-\mu_i(\beta)\}$. The full estimating equation is $$ \sum_{i=1}^{K}D_i^T(\phi A_i)^{-1}\{Y_i-\mu_i(\beta)\}=0, $$ where $K$ is the number of clusters. Each residual is weighted by the inverse of its marginal working variance, together with the mean derivative in $D_i$.

To verify the result, substitute $R_i=I_{m_i}$ into the working covariance:

V_i=\phi A_i^{1/2}I_{m_i}A_i^{1/2} =\phi A_i.

Because the diagonal entries of $A_i$ are positive marginal variance functions, $V_i^{-1}=\phi^{-1}A_i^{-1}$. Substitution into the general GEE contribution $D_i^TV_i^{-1}(Y_i-\mu_i)$ gives the stated expression. This also shows exactly what the independence choice removes: there are no off-diagonal working-covariance terms that mix residuals from different observations in the same cluster.

How Unequal Cluster Sizes Enter

The equation sums cluster contributions, but the number of terms inside a contribution depends on $m_i$. For example, a cluster with two observations has a two-by-two identity matrix; a cluster with four observations has a four-by-four identity matrix. No assumption that every $m_i$ is the same is needed. In practice, one computes each contribution using the observations actually available in that cluster.

Independence does not discard cluster membership altogether. It discards the modeled within-cluster covariance when forming $V_i^{-1}$. Clusters still determine which residuals are grouped in each contribution, and they remain the units used to form cluster-level scores for robust sandwich inference. That distinction matters when responses are correlated in reality: a diagonal working covariance can still be used, while the variance estimator must account for dependence within clusters under the conditions for valid sandwich inference.

Worked Example: Binary Responses in Clusters of Different Sizes

Consider an intercept-only logistic mean model, so every response has mean $\mu=\operatorname{logit}^{-1}(\beta)$. There are three clusters with observed responses $(1,0)$, $(1,1,0)$, and $(0)$, of sizes 2, 3, and 1.

For a Bernoulli response, the variance is $\mu(1-\mu)$, and the derivative of the mean with respect to $\beta$ is also $\mu(1-\mu)$. Thus, $A_i=\mu(1-\mu)I_{m_i}$ and $D_i$ is an $m_i$-vector all of whose entries are $\mu(1-\mu)$. The contribution for cluster $i$ is therefore $$ D_i^T(\phi A_i)^{-1}(Y_i-\mu\mathbf{1}) =\frac{1}{\phi}\sum_{j=1}^{m_i}(Y_{ij}-\mu). $$

The total number of successes is 3 and the total number of responses is $2+3+1=6$. Summing the contributions gives the estimating equation $$ \frac{1}{\phi}(3-6\mu)=0, $$ so $\hat{\mu}=3/6=0.5$, and hence $\hat{\beta}=\log\{0.5/(1-0.5)\}=0$. At this fitted value, the cluster residual sums are $0$, $0.5$, and $-0.5$; their sum is zero. The calculation accommodates the unequal cluster sizes directly, with no common-size matrix required.

What the Choice Discards—and What It Does Not

Under independence working correlation, the covariance used in estimation has no off-diagonal entries. Consequently, it does not use a working estimate of how two responses from the same cluster co-vary. This can be a substantial simplification: the working covariance is diagonal, and no correlation parameter needs to be estimated for that matrix.

The choice does not eliminate the marginal mean model or prevent estimation of its parameters. It also does not assert that the true covariance is diagonal. When the mean model is correct and clusters are independent sampling units, GEE theory allows a working covariance to differ from the true within-cluster covariance, subject to the regularity conditions for the relevant consistency and inference results. A robust sandwich covariance is commonly used to reflect actual within-cluster variation rather than relying on the working covariance as if it were the truth.

There is a possible efficiency cost: a working covariance that ignores real dependence may not use information as effectively as one that represents it well. But independence is not automatically inefficient. It can be efficient if it matches the relevant covariance structure, and in some models it can yield the same mean estimator as the correct-covariance procedure even when responses are correlated.

Worked Example: A Poisson GEE with Unequal Cluster Sizes

Suppose $Y_{ij}$ has a Poisson marginal mean $\mu_{ij}=\exp(\beta_0+\beta_1x_{ij})$. Cluster 1 has two observations with $x=(0,1)$ and $Y=(0,3)$; cluster 2 has three observations with $x=(0,1,1)$ and $Y=(2,1,4)$. Evaluate the independence GEE contribution at $(\beta_0,\beta_1)=(0,\log 2)$, taking $\phi=1$.

The fitted means are $(1,2)$ in cluster 1 and $(1,2,2)$ in cluster 2. For the log link, the derivative row for observation $ij$ is $\mu_{ij}(1,x_{ij})$, while its Poisson variance is $\mu_{ij}$. The factor $\mu_{ij}$ therefore cancels against the inverse variance in the contribution, leaving the row $ (1,x_{ij})(Y_{ij}-\mu_{ij})$.

In cluster 1 the residuals are $(-1,1)$. Its contribution is $$ \begin{pmatrix}1&0\\1&1\end{pmatrix}^{T} \begin{pmatrix}-1\\1\end{pmatrix} = \begin{pmatrix}0\\1\end{pmatrix}. $$ In cluster 2 the residuals are $(1,-1,2)$, so its contribution is $$ \begin{pmatrix}1&0\\1&1\\1&1\end{pmatrix}^{T} \begin{pmatrix}1\\-1\\2\end{pmatrix} = \begin{pmatrix}2\\1\end{pmatrix}. $$ The total GEE left-hand side at the stated parameter value is $(2,2)^T$, not zero. This is a contribution calculation, not a solution of the estimating equation. It exhibits separate cluster vectors of different lengths and shows how the inverse-variance weight combines with the mean derivative.

When Independence Can Be Efficient

A precise efficiency comparison is easiest to state for a Gaussian linear mean model with known covariance. Let $Y$ be a stacked response vector, $E[Y]=X\beta$, and $\operatorname{Cov}(Y)=\Sigma$, where $\Sigma$ is positive definite and $X$ has full column rank. The full-rank condition ensures that the parameter is estimable and that the GLS inverse below exists. Among linear unbiased estimators, generalized least squares (GLS) has covariance no larger in the positive-semidefinite ordering than any other such estimator.

Theorem: Under the preceding conditions, the GLS estimator is $$ \hat{\beta}_{\mathrm{GLS}}=(X^T\Sigma^{-1}X)^{-1}X^T\Sigma^{-1}Y. $$ If $\tilde{\beta}=LY$ is any other linear unbiased estimator, so that $LX=I$, then $$ \operatorname{Cov}(\tilde{\beta})-\operatorname{Cov}(\hat{\beta}_{\mathrm{GLS}}) $$ is positive semidefinite.

To prove the comparison, set $L_{\mathrm{GLS}}=(X^T\Sigma^{-1}X)^{-1}X^T\Sigma^{-1}$ and $C=L-L_{\mathrm{GLS}}$. Both estimators are unbiased, so $LX=I$ and $L_{\mathrm{GLS}}X=I$, giving $CX=0$. Also, $$ \Sigma L_{\mathrm{GLS}}^T=X(X^T\Sigma^{-1}X)^{-1}. $$ Thus $C\Sigma L_{\mathrm{GLS}}^T=0$. Expanding the covariance of $LY=(L_{\mathrm{GLS}}+C)Y$, the two cross terms vanish, and $$ \operatorname{Cov}(LY)-\operatorname{Cov}(L_{\mathrm{GLS}}Y)=C\Sigma C^T. $$ For any vector $a$, $a^TC\Sigma C^Ta=(C^Ta)^T\Sigma(C^Ta)\geq 0$, since $\Sigma$ is positive definite. This proves the claimed ordering. $\square$

If the true Gaussian covariance is diagonal with the marginal variances used by the independence working covariance, the independence estimating equation in a linear mean model is the GLS equation. It therefore attains the GLS covariance. Correlation need not always change the estimator, however. For example, in an intercept-only model with equal marginal variances, if every cluster covariance satisfies $\Sigma_i\mathbf{1}=c\mathbf{1}$ for the same positive $c$, then $\Sigma_i^{-1}\mathbf{1}=c^{-1}\mathbf{1}$. The GLS weights on observations are then all equal, just as for the independence estimator. This condition makes clear that efficiency depends on how the true covariance interacts with the mean design, not simply on whether any correlation exists.

Worked Example: Equal GLS and Independence Estimates Despite Correlation

Consider an intercept-only Gaussian model with two independent clusters of sizes 2 and 3. Suppose the true within-cluster covariance matrices are $$ \Sigma_1=\begin{pmatrix}1&0.2\\0.2&1\end{pmatrix}, \qquad \Sigma_2=\begin{pmatrix}1&0.1&0.1\\0.1&1&0.1\\0.1&0.1&1\end{pmatrix}. $$ Both are positive definite: the eigenvalues of $\Sigma_1$ are $1.2,0.8$, and those of $\Sigma_2$ are $1.2,0.9,0.9$. Each matrix has row sums equal to 1.2, so $\Sigma_i\mathbf{1}=1.2\mathbf{1}$ in both clusters.

Writing $\mathbf{1}$ for the five-vector of ones, GLS weights the response vector by $\Sigma^{-1}\mathbf{1}$, where $\Sigma$ is block diagonal with blocks $\Sigma_1,\Sigma_2$. The row-sum property gives $\Sigma^{-1}\mathbf{1}=(1/1.2)\mathbf{1}$. The common factor cancels in the intercept estimate, leaving the average of all five responses. Independence working correlation with equal marginal variances also gives that average. Thus, in this Gaussian model the two estimates coincide, and the independence estimate is GLS-efficient despite the nonzero within-cluster covariances. The equality follows from the particular mean design and covariance matrices; it is not a general guarantee.

Common Mistakes

Using the variance rather than its inverse. A variance of 4 gives an inverse-variance factor of $1/4$ in $V_i^{-1}$, not a factor of 4. The derivative matrix also enters the GEE contribution.
Treating working independence as a factual claim. Setting working off-diagonal entries to zero is a modeling choice for the estimating equation. Actual responses within a cluster may still be dependent.
Assuming all cluster matrices have the same size. Use $I_{m_i}$ for cluster $i$; its dimension is determined by that cluster’s observed responses.
Concluding that independence must be inefficient whenever correlation exists. The efficiency comparison depends on the mean design and true covariance. Independence can be efficient when it matches the covariance or when both procedures produce the same estimator, as in the special case above.

Key takeaway: Independence working correlation sets $R_i=I_{m_i}$, making each working covariance diagonal and each residual weighted by its inverse marginal variance. It simplifies the GEE without ruling out actual within-cluster dependence; whether it costs efficiency depends on the true covariance and the mean model.

Check Your Understanding

Use the definitions and derivations above to answer the following.

For a cluster of size 4, what is the dimension of its independence working correlation matrix, and what are its off-diagonal entries?
If one marginal working variance is 5, what inverse-variance factor appears in the GEE contribution for that response?
Does using $R_i=I_{m_i}$ establish that responses within cluster $i$ are actually independent? Explain.
Why does the independence GEE remain defined when cluster sizes differ?
Name one condition under which an independence-based linear mean estimator can be GLS-efficient, and explain why the condition matters.