Worked Example: Screening Correlated Renal Markers

Why Compare Correlated Renal Markers Conditionally?

Creatinine-based and cystatin-C-based measures can both carry information about renal function, and the two measures may be correlated. If each marker is tested in a separate model, each test asks whether that marker improves fit without the other marker. It does not answer the more specific question of whether the marker contributes information after accounting for the other marker. For covariate selection, that distinction matters: a marker can look predictive on its own but add little once a correlated alternative is included.

The worked comparison below uses conditional likelihood-ratio tests (LRTs), then checks whether the model fits and whether the selection decision is stable under resampling. The tests compare statistical evidence in specified models; they do not establish that a marker causes a change in clearance or that one marker is clinically superior.

Definition: Let $R_{\mathrm{cr},i}$ and $R_{\mathrm{cys},i}$ be individual $i$'s creatinine-based and cystatin-C-based renal-function markers, measured in the same units, and let $R_{\mathrm{cr,ref}}$ and $R_{\mathrm{cys,ref}}$ be fixed positive reference values. $TVCL$ is typical clearance at both reference values. Individual clearance $CL_i$ includes the individual random effect. In the two-marker model, $$ CL_i=TVCL\left(\frac{R_{\mathrm{cr},i}}{R_{\mathrm{cr,ref}}}\right)^{\theta_{\mathrm{cr}}} \left(\frac{R_{\mathrm{cys},i}}{R_{\mathrm{cys,ref}}}\right)^{\theta_{\mathrm{cys}}} \exp(\eta_{CL,i}). $$ The covariate-predicted typical clearance for individual $i$ is the part before $\exp(\eta_{CL,i})$. Thus, $CL_i$ is individual clearance, not typical clearance.

Set Up the Nested Models

Use four models with the same structural model, residual-error model, random-effects structure, observations, and estimation method. The reference values are fixed across models. Write $M_0$ for the model with neither marker, $M_{\mathrm{cr}}$ for creatinine alone, $M_{\mathrm{cys}}$ for cystatin C alone, and $M_{\mathrm{both}}$ for both markers. The clearance equations are:

\begin{aligned} M_0:\quad &CL_i=TVCL\exp(\eta_{CL,i}),\\ M_{\mathrm{cr}}:\quad &CL_i=TVCL\left(\frac{R_{\mathrm{cr},i}}{R_{\mathrm{cr,ref}}}\right)^{\theta_{\mathrm{cr}}}\exp(\eta_{CL,i}),\\ M_{\mathrm{cys}}:\quad &CL_i=TVCL\left(\frac{R_{\mathrm{cys},i}}{R_{\mathrm{cys,ref}}}\right)^{\theta_{\mathrm{cys}}}\exp(\eta_{CL,i}),\\ M_{\mathrm{both}}:\quad &CL_i=TVCL\left(\frac{R_{\mathrm{cr},i}}{R_{\mathrm{cr,ref}}}\right)^{\theta_{\mathrm{cr}}} \left(\frac{R_{\mathrm{cys},i}}{R_{\mathrm{cys,ref}}}\right)^{\theta_{\mathrm{cys}}}\exp(\eta_{CL,i}). \end{aligned}

For these comparisons, assume $\eta_{CL,i}$ follows the same mean-zero distribution in every model, conventionally $N(0,\omega^2_{CL})$, with the same other model assumptions. The covariate slopes are estimated, and their null values are zero. These models are nested: for example, $M_{\mathrm{cr}}$ is obtained from $M_{\mathrm{both}}$ by setting $\theta_{\mathrm{cys}}=0$. Consequently, adding one marker adds one parameter, so the conditional comparison has $df=1$.

A conditional test of creatinine compares $M_{\mathrm{cys}}$ with $M_{\mathrm{both}}$; its null hypothesis is $H_0:\theta_{\mathrm{cr}}=0$, conditional on cystatin C being in the model. A conditional test of cystatin C compares $M_{\mathrm{cr}}$ with $M_{\mathrm{both}}$; its null hypothesis is $H_0:\theta_{\mathrm{cys}}=0$, conditional on creatinine being in the model.

Theorem: For nested models fitted to the same data, define the objective function as $OFV=-2\ell$, where $\ell$ is the maximized log likelihood. The LRT statistic for a reduced model and a full model is $$ \Delta OFV=OFV_{\mathrm{reduced}}-OFV_{\mathrm{full}} =2\left(\ell_{\mathrm{full}}-\ell_{\mathrm{reduced}}\right). $$ Under the null hypothesis, regularity conditions, and a correctly specified likelihood, $\Delta OFV$ is approximately distributed as $\chi^2_{df}$, where $df$ is the number of added parameters. The p-value is the probability, under that null reference distribution, of a statistic at least as large as the observed $\Delta OFV$.

Worked Conditional Tests

Worked Example: Testing Each Marker Given the Other

Suppose a hypothetical population-PK analysis gives the following minimized objective functions: $OFV_0=1200$, $OFV_{\mathrm{cr}}=1188$, $OFV_{\mathrm{cys}}=1190$, and $OFV_{\mathrm{both}}=1182$. Assume all four models used the same subjects and estimation specifications, converged successfully, and satisfy the regularity conditions for the usual LRT approximation.

Creatinine conditional on cystatin C. The reduced model is $M_{\mathrm{cys}}$, and the full model is $M_{\mathrm{both}}$: $$ \Delta OFV=1190-1182=8. $$ There is one added slope, so $df=1$. Under $H_0:\theta_{\mathrm{cr}}=0$, $8$ has a $\chi^2_1$ upper-tail probability of approximately $0.00468$. At $\alpha=0.05$, the result is evidence that creatinine improves fit conditional on cystatin C in this specified model.

Cystatin C conditional on creatinine. Here the reduced model is $M_{\mathrm{cr}}$: $$ \Delta OFV=1188-1182=6. $$ Again $df=1$, and the $\chi^2_1$ upper-tail probability is approximately $0.0143$. At $\alpha=0.05$, cystatin C also improves fit conditional on creatinine. These results support retaining both markers under this model and decision rule; they do not show that either association is causal.

The joint comparison of $M_0$ with $M_{\mathrm{both}}$ gives $1200-1182=18$, with $df=2$. Its $\chi^2_2$ upper-tail probability is approximately $0.000123$, testing the joint null that both slopes are zero. The conditional tests answer a different question: whether each individual slope adds information after the other slope is included.

Why the Conditional Comparison Matters

The LRT statistic is the increase in maximized log likelihood, multiplied by two. Under the null, Wilks' result gives the approximate chi-square reference distribution when the models are nested, the null parameter is not on a boundary, and the likelihood and estimation behave regularly. Here the added slope is tested at zero, an interior value, so a one-degree-of-freedom reference is appropriate when the other stated conditions hold.

The four objective functions above also show why separate, marginal tests are not substitutes for conditional tests. Relative to $M_0$, adding creatinine alone improves the objective function by $1200-1188=12$; adding cystatin C alone improves it by $1200-1190=10$. Those comparisons do not isolate each marker's contribution after adjustment for the other. With correlated predictors, their information overlaps. Conditional estimates can therefore be less precise, and their magnitudes can differ from estimates in the one-marker models.

A small conditional p-value is evidence against a zero slope under the specified model and its assumptions. A large p-value is not evidence that the marker has no relationship with clearance: it can reflect limited information, collinearity, measurement error, model misspecification, or a genuinely small conditional association. As emphasized in the earlier material on collinearity and conditional covariate effects, the relevant interpretation is information added given the other included terms, not a general ranking of predictors.

Worked Example: Translating the Two-Marker Model

Suppose a hypothetical fitted two-marker model has $TVCL=12.0$ L/h, $R_{\mathrm{cr,ref}}=90$ mL/min/1.73 m$^2$, $R_{\mathrm{cys,ref}}=75$ mL/min/1.73 m$^2$, $\theta_{\mathrm{cr}}=0.32$, and $\theta_{\mathrm{cys}}=0.21$. For an individual with markers 60 and 50 mL/min/1.73 m$^2$, respectively, the two ratios both equal $2/3$. At $\eta_{CL,i}=0$, the covariate-predicted typical clearance is: $$ 12.0\left(\frac{60}{90}\right)^{0.32} \left(\frac{50}{75}\right)^{0.21} =12.0\left(\frac{2}{3}\right)^{0.53} =12.0\exp\left(0.53\ln(2/3)\right) \approx 12.0(0.8066) =9.68\ \text{L/h}. $$ This is the typical clearance at those covariate values. If this individual's random effect were $\eta_{CL,i}=0.10$, individual clearance would instead be $9.68\exp(0.10)\approx10.7$ L/h. The random-effect multiplier does not change the definition of the covariate-predicted typical clearance.

Diagnostics and Selection Stability

Before interpreting a conditional LRT, verify that the comparison is valid. Both models must use the same observations and likelihood construction; a change in the structural model, residual model, or random-effects structure would make the difference harder to attribute to the added marker. Confirm that the more complex model has a finite, lower objective function and that estimation converged to a credible solution. Review parameter estimates, standard errors, covariance or Hessian diagnostics, and profiles when available. A nominal p-value is not persuasive if the added slope is poorly identified or the fit has failed.

Also inspect the covariate data. Check units, reference values, missingness, ranges, and whether the two markers have enough independent variation to distinguish their slopes. A scatterplot or correlation summary can reveal strong association and sparse regions of covariate support. Examine model diagnostics for systematic residual patterns and influential individuals; a marker effect may be compensating for a misspecified structural or error model. Empirical Bayes estimates of individual random effects can be useful context, but shrinkage means they should not be treated as error-free outcomes in a screening regression.

A bootstrap stability summary asks how often a specified selection procedure retains each marker under resampling. For repeated concentration data, resample individuals with all their observations, rather than resampling individual records independently. In each replicate, refit the candidate models and apply the same conditional-test rule. The resulting frequencies describe the stability of that procedure in the observed data; they are not probabilities that a marker is truly associated with clearance.

Worked Example: Summarizing Bootstrap Selection Stability

Suppose 200 hypothetical subject-level bootstrap replicates are analyzed using the same forward rule: begin with neither marker, add the remaining marker with the smaller conditional LRT p-value if it is below $0.05$, then retest the remaining marker conditional on the selected one and stop if it does not pass $0.05$. Assume every replicate converges. The final selected models are both markers in 105 replicates, creatinine only in 58, cystatin C only in 27, and neither in 10.

Creatinine is included in $105+58=163$ replicates, so its inclusion frequency is $163/200=0.815$, or $81.5\%$. Cystatin C is included in $105+27=132$ replicates, giving $132/200=0.660$, or $66.0\%$. Both are selected together in $105/200=52.5\%$ of replicates. These summaries suggest that creatinine is more stable under this particular rule, while the exact two-marker choice is less stable than the full-data fit alone might suggest. They do not turn the bootstrap frequencies into inferential probabilities about the markers.

Alternatives, Limitations, and Common Mistakes

Conditional LRTs are natural when the scientific question concerns added likelihood under nested models. Information criteria such as AIC compare fit while penalizing model size and can be useful when choosing among several candidates, but their decision target differs from a conditional null-hypothesis test. Cross-validation compares predictive performance on held-out data, while bootstrap stability describes sensitivity of a selection rule to resampling. These approaches complement rather than automatically replace conditional testing; their conclusions can differ because they ask different questions.

The LRT relies on a correct likelihood, nested models, comparable fits, and a suitable large-sample approximation. Small samples, weak identification, boundary parameters, or convergence problems can undermine the chi-square calibration. Strong correlation may also make individual slopes unstable even when the joint model clearly improves fit over $M_0$. Moreover, marker measurement error and unmeasured differences can distort associations. A statistical selection result alone does not establish predictive transportability, biological mechanism, or clinical importance.

Mistake: interpreting separate marginal tests as conditional evidence. A significant one-marker test only shows improvement over the no-marker model. Correct the comparison by testing $M_{\mathrm{cys}}$ against $M_{\mathrm{both}}$ for creatinine, and $M_{\mathrm{cr}}$ against $M_{\mathrm{both}}$ for cystatin C.
Mistake: subtracting the objective functions in the wrong order. Use $OFV_{\mathrm{reduced}}-OFV_{\mathrm{full}}$, which is nonnegative when the full nested model achieves at least as high a likelihood. A negative difference signals a fit, optimization, or model-comparison problem.
Mistake: treating a nonsignificant conditional test as proof of no effect. It means the data did not provide sufficient evidence for a nonzero conditional slope under the chosen test and model; correlated predictors can make that test imprecise.
Mistake: calling a bootstrap inclusion frequency the probability the marker is true. It is the fraction of resamples selecting the marker under a specified procedure, not a posterior probability or a measure of causal truth.

Key takeaway: Compare each renal marker with and without the other marker in an otherwise identical, nested clearance model. Report the conditional LRT, its assumptions and diagnostics, and selection stability as complementary evidence—not as a claim that a selected marker causes a clearance change.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a conditional test of cystatin C, which models are compared, and what is the null hypothesis?
A reduced model has $OFV=925$ and its full nested model has $OFV=921.5$, with one added slope. Calculate the LRT statistic and state its approximate p-value.
Why can a marker have a small p-value in a one-marker model but a large p-value conditional on the other renal marker?
What does a bootstrap inclusion frequency of $70\%$ mean, and what does it not mean?

Solutions. (1) Compare $M_{\mathrm{cr}}$ with $M_{\mathrm{both}}$; the null is $H_0:\theta_{\mathrm{cys}}=0$, conditional on creatinine. (2) $\Delta OFV=925-921.5=3.5$, with $df=1$; the $\chi^2_1$ upper-tail p-value is approximately $0.0614$, so it is not significant at $\alpha=0.05$. (3) The markers can share predictive information because they are correlated; the first test does not adjust for the second, while the conditional test isolates added information and may be less precise. (4) Under the specified bootstrap and selection rule, the marker was selected in 70% of resamples. This is not a 70% probability that the marker is true or causal.