When Separate Endpoint Fits Are Not Joint Models

Separate Fits and the Meaning of “Joint”

Two endpoints measured on the same participant can each be modeled with a separate regression, and each fit can describe its endpoint’s marginal relationship with exposure or covariates. That does not, by itself, specify how the two outcomes occur together. The distinction matters whenever the analysis needs their joint distribution or the uncertainty of a quantity involving both endpoint estimates.

For example, two separately fitted models can estimate the probability of improvement on each of two binary pharmacodynamic endpoints. Multiplying their likelihoods treats the outcomes as conditionally independent given the modeled covariates. If the outcomes remain associated after conditioning on those covariates, that product is not the likelihood of the observed paired outcomes. It may still give useful marginal estimates, but uncertainty calculations that treat the fits as independent can be wrong.

Definition: A separate endpoint fit estimates a model for one endpoint at a time. A joint model specifies a probability distribution for the outcomes together, including their dependence. For outcomes measured on the same subject, multiplying endpoint-specific likelihoods gives the joint likelihood only when the outcomes are conditionally independent given the modeled covariates and parameters.

Let \(Y_{ij}\) denote endpoint \(j\) for subject \(i\), where \(j=1,\ldots,J\), and let \(x_i\) denote that subject’s modeled covariates. Suppose endpoint \(j\) has marginal density or probability mass function \(f_j(y_{ij}\mid x_i;\theta_j)\), with endpoint-specific parameters \(\theta_j\). Assume subjects are independent of one another. If the endpoints are also conditionally independent within subject given \(x_i\), the joint likelihood is

L(\theta_1,\ldots,\theta_J) = \prod_{i=1}^{n}\prod_{j=1}^{J} f_j(y_{ij}\mid x_i;\theta_j).

Its log-likelihood is a sum of endpoint-specific log-likelihoods. If each parameter vector belongs only to its own endpoint model, maximizing this product gives the same endpoint estimates as fitting each endpoint separately. The factorization is not merely a computational convenience: it asserts that the joint conditional probability equals the product of the marginal conditional probabilities.

What the Product Assumption Implies

For two endpoints, conditional independence means

P(Y_{i1}=a,Y_{i2}=b\mid x_i) = P(Y_{i1}=a\mid x_i)P(Y_{i2}=b\mid x_i)

for all possible outcome values \(a\) and \(b\). This condition concerns dependence remaining after conditioning on the covariates in the models. Two endpoints can be associated in the observed data because they both depend on exposure or disease severity, yet be conditionally independent after adequate adjustment. Conversely, fitting separate marginal models does not establish conditional independence.

The implication for uncertainty can be seen from the scores. Write \(u_{ij}(\theta_j)\) for subject \(i\)’s score contribution from endpoint \(j\). The score for the product likelihood is the sum of those contributions. Its variance contains cross-endpoint terms:

\operatorname{Var}\left(\sum_j u_{ij}\right) = \sum_j\operatorname{Var}(u_{ij}) + 2\sum_{j<k}\operatorname{Cov}(u_{ij},u_{ik}).

Under the assumed conditional independence, the cross-covariances are zero in the correctly specified likelihood. If the endpoints are dependent, they need not be zero. The separate fits can still estimate the marginal parameters consistently when their marginal models are correct and subjects are independent, but the usual product-likelihood information calculation sets cross-endpoint covariance to zero. Thus, standard errors for individual endpoint parameters may remain appropriate under their own correct marginal models, while joint uncertainty and endpoint contrasts can be misestimated.

Key result: Multiplying marginal likelihoods makes the joint uncertainty calculation as if endpoint score contributions were independent within subject. For a quantity combining endpoint estimates, the omitted covariance can either increase or decrease its variance; the direction depends on the sign of the dependence and on how the estimates enter the quantity.

Worked Examples

Worked Example: Writing a Product Likelihood for Paired Binary Outcomes

In a hypothetical pilot dataset, eight participants each have two binary outcomes: response on endpoint A and response on endpoint B. The paired counts are three participants positive on both, one positive only on A, none positive only on B, and four negative on both. Let \(p_A\) and \(p_B\) be the marginal response probabilities.

The separate Bernoulli likelihoods use four A responses and three B responses. Their product is

L_{\mathrm{prod}}(p_A,p_B) = p_A^4(1-p_A)^4p_B^3(1-p_B)^5.

The separate maximum-likelihood estimates are \(\hat p_A=4/8=0.5\) and \(\hat p_B=3/8=0.375\). Substituting them gives

L_{\mathrm{prod}}(0.5,0.375) = (0.5)^8(0.375)^3(0.625)^5 \approx 0.00001965.

This product specifies the joint probability of a participant being positive on both outcomes as \(p_Ap_B=0.5(0.375)=0.1875\). The observed proportion positive on both is \(3/8=0.375\). The discrepancy illustrates why the product is not automatically the likelihood for these paired records: it imposes a joint pattern implied by independence, whereas the observed pairings need not follow that pattern. In a larger dataset, this difference is reflected in cross-endpoint association and uncertainty.

Worked Example: Dependence Changes Inference for a Paired Risk Difference

Consider hypothetical binary responses on two endpoints from the same 400 participants. The paired counts are: 156 positive on both endpoints, 44 positive only on A, 20 positive only on B, and 180 negative on both. The estimand is the difference in marginal response probabilities, \(p_A-p_B\), not a causal effect.

The marginal estimates are \(\hat p_A=(156+44)/400=0.50\) and \(\hat p_B=(156+20)/400=0.44\). Thus, the estimated difference is \(0.50-0.44=0.0600\), or 6.00 percentage points. For paired observations, the estimated covariance of the two binary indicators is

\widehat{\operatorname{Cov}}(Y_A,Y_B) = \frac{156}{400}-(0.50)(0.44) = 0.390-0.220 = 0.170.

Using the large-sample plug-in variance for a difference of paired sample proportions gives

\widehat{\operatorname{Var}}(\hat p_A-\hat p_B) = \frac{ \hat p_A(1-\hat p_A)+\hat p_B(1-\hat p_B) -2\widehat{\operatorname{Cov}}(Y_A,Y_B) }{n} = \frac{0.250+0.2464-0.340}{400} = 0.000391.

The paired standard error is therefore \(\sqrt{0.000391}=0.0198\), or 1.98 percentage points. A normal-approximation 95% confidence interval is \(0.0600\pm1.96(0.0198)\), which is approximately \([0.0212,0.0988]\). The corresponding Wald statistic for \(H_0:p_A-p_B=0\) is \(0.0600/0.0198=3.03\), with a two-sided normal-approximation p-value of about 0.0024.

If the endpoint fits are treated as independent, the covariance term is omitted. The resulting standard error is

SE_{\mathrm{ind}} = \sqrt{\frac{0.250+0.2464}{400}} = 0.0352.

That calculation gives a 95% interval of \(0.0600\pm1.96(0.0352)\), or approximately \([-0.0090,0.1290]\), and a Wald statistic of \(0.0600/0.0352=1.70\), with a two-sided p-value of about 0.089. In this example, positive within-subject association makes the paired standard error smaller than the independence-based one. The inference changes because the product fit discards the covariance needed for the contrast, not because either marginal response estimate changed.

These Wald calculations use a large-sample normal approximation. Their use should be assessed in context; notably, the observed discordant pairs (44 and 20) are numerous here, making the approximation more credible than it would be with very few discordances.

Worked Example: Negative Dependence Can Increase Uncertainty

Suppose 100 hypothetical participants have two measurements on the same scale. The estimated difference in means is 1.50 units, the sample standard deviations are 2.00 and 3.00 units, and the sample correlation between paired measurements is \(-0.50\). Assume independent participants and use a large-sample normal approximation.

The paired variance of the difference in sample means is estimated using the covariance between measurements. Since the sample covariance is \((-0.50)(2.00)(3.00)=-3.00\) square units,

\widehat{\operatorname{Var}}(\bar Y_1-\bar Y_2) = \frac{2.00^2+3.00^2-2(-3.00)}{100} = \frac{19.00}{100} = 0.190

in squared units. The paired standard error is \(\sqrt{0.190}=0.436\) units, giving a 95% interval \(1.50\pm1.96(0.436)=[0.645,2.355]\) units. Treating the endpoint estimates as independent would instead give \(SE=\sqrt{(4+9)/100}=0.361\) units and an interval \([0.793,2.207]\) units. Here negative dependence increases uncertainty in the difference. Therefore, dependence does not always make independent-fit intervals too wide; it can make them too narrow.

Justification and Practical Alternatives

The covariance correction in the paired examples follows directly from the variance identity. For paired subject-level outcomes, the estimated difference is the mean of \(D_i=Y_{i1}-Y_{i2}\). Since

\operatorname{Var}(D_i) = \operatorname{Var}(Y_{i1})+\operatorname{Var}(Y_{i2}) -2\operatorname{Cov}(Y_{i1},Y_{i2}),

independent-endpoint calculations omit exactly the final term. For independent subjects, the variance of the sample mean of \(D_i\) is \(\operatorname{Var}(D_i)/n\). A positive covariance reduces the variance of a difference; a negative covariance increases it. The same principle applies to model-based contrasts, although the covariance must then be calculated for the relevant parameter estimates rather than directly for the observed outcomes.

When endpoint dependence is present but the marginal models are the desired targets, one option is to retain separate marginal fits and calculate uncertainty with a subject-level sandwich estimator. For a product of marginal likelihoods, let \(u_i\) be the sum of the endpoint score contributions for subject \(i\), \(H\) the expected negative derivative of that score, and \(J=\operatorname{Var}(u_i)\). A first-order Taylor expansion of the estimating equation around the true parameter, together with the central limit theorem for independent subjects, gives the asymptotic covariance

\operatorname{Cov}(\hat\theta) \approx \frac{1}{n}H^{-1}JH^{-1}.

The usual product-likelihood calculation effectively uses the model-based information alone. Under the assumed joint likelihood, the information identity makes that calculation appropriate. Under dependence, the subject-level sandwich estimates the score covariance \(J\) from the paired records rather than forcing cross-endpoint terms to zero. It can support inference for contrasts while leaving the marginal endpoint estimates unchanged. It does not, however, supply a full model for joint outcome probabilities.

The main alternatives answer different questions. Separate marginal fits are useful when endpoint-specific exposure-response relationships are the targets. Separate fits with a subject-level robust covariance can preserve those targets while adjusting joint inference for dependence. A model that explicitly specifies a joint outcome distribution is needed when the target concerns co-occurrence, joint probabilities, or other features of the dependence itself. These approaches are not interchangeable: the appropriate choice depends on the estimand and the assumptions the analysis can defend.

Diagnostics, Failure Modes, and Limitations

Good marginal fit does not demonstrate that a product likelihood is a valid joint likelihood. Each endpoint model can reproduce its own response rate or conditional distribution while the pairing of outcomes is poorly represented. Examine paired tables or residual association by covariate and exposure strata, and compare observed joint patterns with those implied by the product of fitted marginal probabilities. For continuous outcomes, inspect paired residual covariance. These checks can reveal dependence, but a small observed association is not proof of independence, particularly with limited data.

Ignoring the pairing. Treating two records from one subject as if they came from unrelated subjects discards covariance. For endpoint contrasts, this can lead to either over- or under-estimated standard errors.
Confusing marginal and joint fit. Correctly fitted marginal endpoint models do not determine the joint probability of both outcomes. Do not use their product for co-occurrence predictions without an independence assumption.
Assuming raw association settles conditional independence. Association can change after conditioning on covariates. Evaluate dependence in the model’s conditional context, not only in pooled data.
Using a sandwich estimate as a dependence model. A robust covariance can improve uncertainty calculations for marginal estimates; it does not describe the full joint outcome distribution or estimate all joint probabilities.

The independence assumption can be plausible when the modeled covariates capture the relevant common predictors and residual association is negligible for the analysis goal. It is still an assumption, not a consequence of fitting endpoints separately. Robust covariance estimates also rely on independent subjects and may be unstable in small samples. When dependence is important, diagnostics are weak, or joint predictions are central, a product of separate endpoint likelihoods has a clear limitation: it does not represent the feature of the data that links the outcomes.

Check Your Understanding

Answer each question before reading the solutions.

Two binary endpoints are recorded for each participant. What assumption makes the product of their marginal likelihoods a joint likelihood?
For a paired endpoint contrast, the two outcomes have positive covariance. Does omitting that covariance make the estimated variance of their difference larger or smaller?
Can a subject-level sandwich covariance estimate provide a model for the probability that both endpoints are positive? Explain.
Separate endpoint models fit their marginal distributions well, but the observed joint response frequency differs from the product of fitted marginal probabilities. What does this suggest?

Solutions. (1) The endpoints must be conditionally independent within subject given the covariates and parameters in the models, in addition to the assumed independence across subjects. (2) Smaller: the paired variance includes \(-2\operatorname{Cov}(Y_1,Y_2)\), so positive covariance reduces the variance of a difference; omitting it overestimates that variance. (3) No. The sandwich estimate adjusts covariance of parameter estimates for dependence in score contributions, but it does not specify the joint outcome distribution. (4) The marginal models may be adequate while conditional independence is not; the product likelihood then fails to represent the observed pairing and should not be treated as the joint likelihood without further justification.