Collinearity And Weakly Identified Marker Associations

Why Collinearity Matters for Marker Associations

In a multimarker joint model, two biomarkers may follow very similar trajectories across subjects and time. The survival component can then have difficulty distinguishing which marker is associated with the event hazard. This is not necessarily a problem with estimating each marker’s longitudinal trajectory: it is a problem of learning their separate contributions to the hazard from data in which their values tend to move together.

Suppose the hazard for subject $i$ at time $t$, conditional on the latent marker trajectories, is

\lambda_i(t)=\lambda_0(t)\exp\{\alpha_1 m_{i1}(t)+\alpha_2 m_{i2}(t)+\boldsymbol{\gamma}^{\mathsf T}\mathbf{w}_i\}.

Here $m_{i1}(t)$ and $m_{i2}(t)$ are the two latent marker values, $\alpha_1$ and $\alpha_2$ are their hazard association coefficients, $\lambda_0(t)$ is the baseline hazard, and $\mathbf{w}_i$ denotes other covariates with coefficients $\boldsymbol{\gamma}$. Holding the other terms fixed, a one-unit increase in marker 1 multiplies the hazard by $\exp(\alpha_1)$, with marker 2 held fixed. If the two markers are highly correlated, such a contrast may be uncommon in the observed data. The coefficient can therefore be poorly estimated even when the model estimates the markers’ combined contribution more precisely.

Definition: Collinearity occurs when marker values or trajectory features used together in the hazard model are strongly linearly related. Weak identification of separate marker associations means that the likelihood changes little for some changes in the individual coefficients, so the data provide limited information about those coefficients separately. It does not, by itself, mean that the model has no maximum or that the markers have no association with the hazard.

In a joint model, the marker values in the hazard are latent and estimated through the longitudinal and survival likelihood together. The concern here is not simply correlation between observed measurements at one visit. It is whether the subjects’ estimated marker trajectories provide enough independent variation to distinguish $\alpha_1$ from $\alpha_2$. The earlier material on identifying marker effects separately from covariance is relevant to the distinction: covariance parameters describe dependence in the marker process, whereas collinearity can also weaken information about the hazard coefficients.

Information, Uncertainty, and Coefficient Directions

Near a regular maximum-likelihood estimate $\widehat{\boldsymbol{\alpha}}$, the log likelihood is often approximated by a quadratic. The negative Hessian of the log likelihood, evaluated at the estimate, is the observed information matrix. Under the usual large-sample approximation, its inverse estimates the covariance matrix of the coefficient estimates. If that matrix has a small eigenvalue, the likelihood is relatively flat in the corresponding direction and the associated coefficient combination is uncertain.

With two markers, the relevant directions are often the sum and difference of the coefficients. If the trajectories are similar, increasing $\alpha_1$ while decreasing $\alpha_2$ can leave the linear predictor nearly unchanged. The difference $\alpha_1-\alpha_2$ may then be poorly determined, while a sum-like combination is more stable. Which combination is precise depends on the information in the actual data; it should be checked rather than assumed.

Local likelihood approximation: For a coefficient vector $\boldsymbol{\alpha}$ near its estimate, the log likelihood is approximated by $$ \ell(\boldsymbol{\alpha})\approx \ell(\widehat{\boldsymbol{\alpha}})-\frac{1}{2}(\boldsymbol{\alpha}-\widehat{\boldsymbol{\alpha}})^{\mathsf T}\mathbf{I}(\boldsymbol{\alpha}-\widehat{\boldsymbol{\alpha}}), $$ where $\mathbf{I}$ is the observed information matrix. The approximation is useful for diagnosing local curvature, but it may be inaccurate when the likelihood is strongly nonquadratic or nearly flat over a broad range.

Worked Example: Reading Uncertainty From an Information Matrix

Consider hypothetical fitted coefficients $\widehat{\alpha}_1=0.80$ and $\widehat{\alpha}_2=-0.30$. Suppose the observed information for these two coefficients is

\mathbf{I}= \begin{pmatrix} 100 & 98\\ 98 & 100 \end{pmatrix}.

This positive off-diagonal information is consistent with strong collinearity in the marker predictors. The determinant is $100(100)-98(98)=396$, so the inverse information matrix is

\widehat{\operatorname{Var}}(\widehat{\boldsymbol{\alpha}}) =\mathbf{I}^{-1} =\frac{1}{396} \begin{pmatrix} 100 & -98\\ -98 & 100 \end{pmatrix}.

Thus each coefficient has estimated variance $100/396=1/3.96\approx0.2525$, and its standard error is $\sqrt{1/3.96}\approx0.5025$. The estimated covariance is $-98/396=-0.98/3.96\approx-0.2475$. The strong negative covariance means that the two estimates can move in opposite directions while fitting similarly.

For the sum, use the unrounded covariance entries:

\widehat{\operatorname{Var}}(\widehat{\alpha}_1+\widehat{\alpha}_2) =\frac{1}{3.96}+\frac{1}{3.96}+2\left(\frac{-0.98}{3.96}\right) =\frac{2(1-0.98)}{3.96} \approx0.01010.

Its standard error is approximately $0.1005$, and the estimated sum is $0.50$. A 95% Wald interval is $0.50\pm1.96(0.1005)$, or approximately $(0.3030,0.6970)$. By contrast, the estimated difference is $1.10$, with variance $2(1+0.98)/3.96=1.000$ and standard error $1.000$. This example shows why uncertainty about each marker’s separate coefficient need not imply comparable uncertainty about every combined effect.

Profiling a Marker Coefficient

A likelihood profile examines the likelihood for a coefficient of interest while allowing nuisance parameters to adjust. For example, to study $\alpha_1$, fix it at a candidate value and maximize the likelihood over $\alpha_2$, the baseline-hazard parameters, and other nuisance parameters. Denote this maximized log likelihood by $\ell_p(\alpha_1)$, and let $\ell_{\max}$ be the unrestricted maximum. The profile likelihood-ratio statistic is $2\{\ell_{\max}-\ell_p(\alpha_1)\}$.

Under regularity conditions—including an interior true parameter, a correctly specified likelihood, adequate information, and a sufficiently large sample—the statistic has an approximate chi-square distribution with one degree of freedom when testing a fixed value of one coefficient. A 95% likelihood-ratio confidence set contains values for which the statistic is no greater than $\chi^2_{1,0.95}\approx3.84$. For a joint region for two coefficients, the corresponding reference cutoff is $\chi^2_{2,0.95}\approx5.99$, not 3.84.

Why profiling helps: Fixing $\alpha_1$ and re-optimizing the other parameters shows how well the model can compensate for a change in $\alpha_1$. A broad, shallow profile indicates that many values of $\alpha_1$ fit nearly as well after the nuisance parameters adjust. A profile interval can reveal asymmetry or flatness that a symmetric Wald interval based on the local curvature may obscure.

Worked Example: A Broad Profile for an Individual Marker

Continue with the hypothetical information matrix and estimate above. Under the local quadratic approximation, the variance of $\widehat{\alpha}_1$ is $1/3.96$. The approximate profile likelihood-ratio statistic for a candidate $\alpha_1$ is therefore

2\{\ell_{\max}-\ell_p(\alpha_1)\} \approx \frac{(\alpha_1-0.80)^2}{1/3.96} =3.96(\alpha_1-0.80)^2.

The one-coefficient 95% profile cutoff gives a half-width of $\sqrt{3.84/3.96}=\sqrt{0.969697}\approx0.9847$. The approximate profile interval for $\alpha_1$ is consequently $(0.80-0.9847,\,0.80+0.9847)=(-0.1847,1.7847)$. For $\alpha_2$, centered at $-0.30$ with the same variance, it is $(-1.2847,0.6847)$. These wide intervals include zero and illustrate that the separate coefficients are imprecise under this approximation. They are not proof that either association is exactly zero.

The interval is based on a quadratic approximation; an actual profile should be calculated by refitting the nuisance parameters at each fixed coefficient value. A real profile may be asymmetric or depart from this interval, particularly if the likelihood is weakly identified or the sample is not large.

Deriving the Profile Curvature

The relationship between profile curvature and the inverse information matrix explains why uncertainty in the nuisance coefficient must be included. Partition the information for $\alpha_1$ and a nuisance coefficient $\alpha_2$ as

\mathbf{I}= \begin{pmatrix} I_{11} & I_{12}\\ I_{12} & I_{22} \end{pmatrix}.

For a fixed displacement $\delta_1=\alpha_1-\widehat{\alpha}_1$, the quadratic log-likelihood loss is minimized over $\delta_2=\alpha_2-\widehat{\alpha}_2$. Differentiating the quadratic form with respect to $\delta_2$ gives $I_{12}\delta_1+I_{22}\delta_2=0$, hence $\delta_2=-I_{12}\delta_1/I_{22}$. Substituting this value back into the quadratic form leaves the profile curvature

I_{11}-\frac{I_{12}^2}{I_{22}}.

This quantity is the reciprocal of the $(1,1)$ element of $\mathbf{I}^{-1}$. Thus the local profile-based variance for $\widehat{\alpha}_1$ agrees with the corresponding inverse-information variance. When the predictors are highly collinear, the nuisance coefficient can offset changes in the coefficient of interest, reducing profile curvature and increasing uncertainty.

Worked Example: Checking a Point in the Joint Likelihood Region

Use the same hypothetical quadratic likelihood, with maximum at $(0.80,-0.30)$. Consider the candidate pair $(0.30,0.20)$. Its displacement from the maximum is $(-0.50,0.50)$. For the stated information matrix, the likelihood-ratio distance is

\begin{pmatrix}-0.50&0.50\end{pmatrix} \begin{pmatrix}100&98\\98&100\end{pmatrix} \begin{pmatrix}-0.50\\0.50\end{pmatrix} =1.00.

This value is below the two-parameter 95% cutoff of approximately $5.99$, so the candidate pair is inside the approximate joint 95% likelihood-ratio region. The displacement is along a difference-like direction, where the likelihood is relatively flat. This joint result does not mean that the one-parameter cutoff is 5.99: the one-parameter profile interval uses 3.84, while the two-parameter joint region uses 5.99.

Diagnostics, Alternatives, and Limitations

Useful diagnostics combine numerical checks with the shape of the likelihood. Inspect the estimated correlation between coefficient estimates, the eigenvalues and condition of the information matrix, and profile plots for each association. A long, tilted contour indicates a ridge: substantial changes in the separate coefficients can yield similar likelihood. Also check whether optimization results change with starting values or convergence tolerances, and whether estimates are unusually sensitive to small changes in the fitted longitudinal trajectories or model specification.

A common mistake is to interpret a nonsignificant individual coefficient as evidence that its marker has no hazard association, or to interpret a significant combined effect as proof that both markers have distinct effects. With collinearity, the data may support a combined predictor while providing little evidence about how its association should be divided between the markers. Report coefficient estimates with their uncertainty and describe the instability; do not turn a lack of precision into evidence of no effect.

The main alternatives answer different questions. A combined marker score or a prespecified weighted burden can target an overall association when separate effects are not the scientific target. Dropping one marker may yield a more stable coefficient for the remaining marker, but changes the model and its interpretation; it does not establish that the omitted marker has no effect. Penalized estimation may stabilize coefficients, but its results depend on the penalty and should not be presented as if they were ordinary unpenalized likelihood estimates. Re-expressing coefficients as sums and contrasts can clarify which combinations are estimated precisely, without creating information that the data do not contain.

Profile likelihood is also not a cure for weak identification. Its usual chi-square cutoffs rely on regular large-sample conditions. A very flat or irregular profile, a boundary estimate, or a likelihood with multiple modes can make those approximations unreliable. In such cases, state the limitation, check the model fit and optimization carefully, and avoid claiming precise separate marker effects. A precise estimate of a combined association should be reported only when that combination was defined and evaluated in the model.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

If marker trajectories are strongly correlated, what coefficient direction is often weakly identified: a sum-like direction or a difference-like direction?
For a one-coefficient likelihood-ratio interval at 95%, which approximate chi-square cutoff is used? What cutoff applies to a joint region for two coefficients?
Using the hypothetical example, why is the estimated sum more precise than either coefficient alone?
Does a wide profile interval containing zero establish that the marker has no hazard association?

Solutions. (1) Often the difference-like direction, because one coefficient can increase as the other decreases while their combined predictor changes little. The exact direction depends on the fitted model and data. (2) The one-degree-of-freedom cutoff is about 3.84; the two-degree-of-freedom joint cutoff is about 5.99. (3) The negative covariance reduces the variance of the sum: $2(1-0.98)/3.96\approx0.01010$, giving $SE\approx0.1005$, smaller than the individual standard errors of about $0.5025$. (4) No. It indicates substantial uncertainty under the model and approximation; it does not establish a zero association.