Proximal Contamination in LMM Association Tests

Why a Relationship Matrix Can Weaken a Nearby Association Test

A genomic relationship matrix summarizes genetic similarity among individuals and supplies the covariance structure for a linear mixed model (LMM). If the matrix includes the same region as the variant being tested, part of the local genetic pattern can be represented as random-effect covariance. The test must then distinguish a fixed effect of the tested genotype from variation the model has already allowed along a similar direction.

This phenomenon is called proximal contamination. It is most important when the tested variant, or variants in linkage disequilibrium (LD) with it, contribute appreciably to the relationship matrix. The practical consequence is often reduced information and power. It is important to be precise: with a fixed, correctly specified covariance matrix, including a genotype-related covariance direction does not by itself bias the generalized least-squares estimate of the fixed effect. It can nevertheless make that estimate less precise, and variance-component estimation under a null model can further weaken the test.

Model and the Source of Information Loss

For one tested genotype vector \(x\), write the association model as

Here \(y\) is the phenotype vector, \(C\) is the fixed-covariate design matrix, \(\gamma\) is its coefficient vector, \(\beta\) is the tested variant effect, \(K\) is the relationship matrix, and \(\tau\) and \(\sigma_e^2\) are variance components. The model assumes that the random effect and residual have mean zero, are independent, and have the stated covariance matrices. Conditional on the genotypes and covariates, the marginal covariance of \(y\) is \(V=\tau K+\sigma_e^2I\). For the derivation below, take \(V\) as known and positive definite.

After accounting for covariates, the generalized least-squares (GLS) information for \(\beta\) is \(x'Px\), where

Thus the GLS variance is \(1/(x'Px)\), and the standard error is \(1/\sqrt{x'Px}\). A covariance contribution aligned with \(x\) makes the model treat variation in that direction as less informative about a fixed effect. This raises the standard error. It does not necessarily pull the GLS coefficient estimate toward zero.

A Rank-One Derivation

The simplest calculation isolates the tested genotype's contribution to the covariance. Suppose there are no covariates beyond an intercept, \(x\) is centered, \(x'x=n\), and

The local covariance term has \(x\) as an eigenvector: \(Vx=(\sigma_e^2+\tau)x\). Therefore \(V^{-1}x=x/(\sigma_e^2+\tau)\), and the information for \(\beta\) is

Without that local covariance term, the information would be \(n/\sigma_e^2\). The information ratio is \(\sigma_e^2/(\sigma_e^2+\tau)\), and the standard error is multiplied by \(\sqrt{(\sigma_e^2+\tau)/\sigma_e^2}\). This comparison assumes the same residual variance and known covariance parameters; real fitted models may also change their variance-component estimates.

How Nearby Markers Can Contaminate a Test

The tested variant need not appear explicitly in the matrix for the problem to arise. A nearby marker in LD can have a genotype vector strongly aligned with the tested genotype. For a simple illustration, suppose the relationship contribution is \(K_R=zz'/n\), where \(z'z=n\) is a centered nearby-marker vector, and the tested vector also satisfies \(x'x=n\). Let their sample correlation be \(r=x'z/n\). With \(V=I+\tau K_R\), the Sherman–Morrison identity gives

The identity applies because \(I\) is invertible and \(1+\tau>0\), as required for this covariance model. The formula shows how LD transfers the effect of a nearby marker's covariance onto the tested variant's information. The size of the reduction depends on both the alignment \(r\) and the covariance contribution \(\tau\); proximity in physical distance alone does not determine it.

Exclusion Strategies and Their Trade-Offs

The standard remedy is to construct the relationship matrix without markers from the tested region, then use that matrix in the association test. The leave-one-chromosome-out (LOCO) approach excludes the tested chromosome. A leave-region-out approach excludes a smaller window around the variant. Both aim to reduce local alignment between the tested genotype and the random-effect covariance while retaining information about genome-wide relatedness.

The appropriate exclusion depends on the genomic structure and analysis goal. A narrow window may leave correlated markers outside the window; a whole-chromosome exclusion removes more local contribution but can make the covariance estimate less representative of genome-wide relatedness. Recomputing the matrix can also change estimated variance components and standard errors. Therefore, differences between approaches should be assessed as matched analyses, not interpreted as a simple correction factor.

Diagnostics, Assumptions, and Failure Modes

A useful diagnostic is to compare association results for the same variants using the all-marker matrix and a matrix that excludes the tested region. Keep phenotype processing, covariates, genotype coding, sample, and test definition fixed. Inspect changes in effect estimates, standard errors, and p-values together. A pattern of similar estimates but larger all-marker standard errors is consistent with an information loss. A substantial estimate change deserves further investigation, because it may reflect variance-component refitting, model instability, or a change in the effective analysis rather than proximal contamination alone.

The derivations assume a correct mean model, a positive-definite covariance matrix, and (for the displayed Wald calculations) known variance parameters and normally distributed errors. In practice, variance components are estimated. Under a null association fit, a strong local signal may be represented partly as random-effect variance, which can make a test conservative. The magnitude and direction of the resulting test distortion are not determined by the rank-one calculation; they depend on the fitting procedure and model. Relatedness, population structure, residual variance, and genotyping quality still need appropriate treatment.

Look for markers with high LD to the tested genotype that remain in the matrix, and check which markers were actually used to construct each matrix. An exclusion window that stops before relevant LD decays may provide little protection. Conversely, unusually large changes after exclusion can signal that the two fitted models differ in more than local covariance. Neither a single changed p-value nor a comparison of significance labels is an adequate diagnostic; effect estimates and standard errors are more informative.

A common mistake is to say, “The relationship matrix contains the tested SNP, so the fixed-effect estimate must be shrunk toward zero.” That conclusion does not follow. With known \(V\), the GLS estimate remains unbiased under the stated model; its variance can increase because \(x'Px\) decreases. A different mistake is to switch from a Wald test to a score test and assume the problem is gone. If both tests use a covariance matrix containing the same local contribution, changing the test statistic does not remove that contribution.

Proximal exclusion also has limits. LOCO or leave-region-out matrices reduce one source of overlap; they do not guarantee a well-calibrated test, correct variance components, or absence of population stratification. The region definition should be reported, and sensitivity to reasonable exclusion choices should be examined when local signal is important.