Joint Likelihoods For Mixed Endpoint Types

Why Mixed Endpoints Need One Likelihood

A PK/PD analysis may record outcomes on different scales for the same subject: a continuous biomarker, a binary clinical response, an ordinal severity score, or an event count. Each endpoint can retain its appropriate response model. The joint-modeling task is to combine their contributions in a way that also accounts for dependence between endpoints.

A shared latent effect provides one way to do this. Conditional on that effect, endpoint responses may be modeled as independent, while marginally they can be associated because they share the same unobserved subject-level factor. The essential operation is to multiply the endpoint contributions before integrating over the shared effect.

Definition: Let $Y_{ik}$ denote endpoint $k$ for subject $i$, and let $b_i$ be a subject-level latent effect with density or probability distribution $g(b_i\mid\psi)$. If the endpoint responses are conditionally independent given $b_i$ and the modeled covariates, the subject’s joint likelihood contribution is $$ L_i(\theta,\psi)=\int \left\{\prod_{k\in\mathcal O_i} f_{ik}(y_{ik}\mid b_i,\theta_k)\right\}g(b_i\mid\psi)\,db_i. $$ Here $\mathcal O_i$ is the set of observed endpoints for subject $i$, $f_{ik}$ is the endpoint-specific probability mass or density, $\theta_k$ denotes that endpoint’s parameters, and $\psi$ describes the latent-effect distribution. For a discrete latent effect, the integral is replaced by a sum.

A density contribution for a continuous endpoint and probability contributions for discrete endpoints can appear in the same product. The units of the resulting likelihood contribution include the units contributed by the continuous densities; it is not necessarily a dimensionless probability. For independent subjects, the full likelihood is $\prod_i L_i$. In a repeated-measures model, all observations from a subject that share $b_i$ belong inside that subject’s product, and the integration over $b_i$ is performed once.

Endpoint Contributions Stay Endpoint-Specific

The joint construction does not require replacing the endpoint models with one common response distribution. For example, a continuous response may have a normal density, a binary response a Bernoulli probability, an ordinal response a cumulative-link category probability, and a count response a Poisson or negative-binomial probability. The shared effect enters each endpoint model according to its specified structure.

For a binary response with success probability $p_i(b_i)$, the contribution is $p_i(b_i)^{y_i}\{1-p_i(b_i)\}^{1-y_i}$, where $y_i$ is zero or one. For an ordinal response $Y_i$ with ordered categories $1,\ldots,J$, a cumulative-logit model may specify

F_j(b_i)=\operatorname{logit}^{-1}(\tau_j-\lambda b_i),\quad j=1,\ldots,J-1,\qquad F_0(b_i)=0,\quad F_J(b_i)=1,\qquad P(Y_i=j\mid b_i)=F_j(b_i)-F_{j-1}(b_i),\quad j=1,\ldots,J.

The thresholds must be ordered, and the category probabilities must be nonnegative and sum to one. For a Poisson count $C_i$, a log-link model may specify $C_i\mid b_i\sim\operatorname{Poisson}(\mu_i(b_i))$, with contribution $\exp\{-\mu_i(b_i)\}\mu_i(b_i)^{C_i}/C_i!$. These are examples of endpoint-specific likelihood contributions, not restrictions that every joint model must use these links.

Assumptions to state: The response models and covariate structure are correctly specified; the distribution $g(b_i\mid\psi)$ is specified; and endpoint responses are conditionally independent given the shared latent effects and modeled covariates if their contributions are multiplied. If residual dependence remains after conditioning, the conditional product is not justified and a richer conditional joint distribution is needed. Likelihood-based treatment of missing endpoint values also requires an appropriate missingness assumption; informative missingness may require modeling the missingness process.

Why Multiplication Comes Before Integration

For two endpoints, conditional independence gives $f_1(y_1\mid b)f_2(y_2\mid b)$ at a particular value of the latent effect. Averaging that product over the effect’s distribution yields the joint marginal likelihood. In general, it is not the product of the two endpoint likelihoods after each has been separately averaged.

\begin{aligned} L(y_1,y_2) &=\int f_1(y_1\mid b)f_2(y_2\mid b)g(b)\,db,\\ L_1(y_1)L_2(y_2) &=\left\{\int f_1(y_1\mid b)g(b)\,db\right\} \left\{\int f_2(y_2\mid b)g(b)\,db\right\}. \end{aligned}

The first expression uses the same latent value for both endpoint contributions within a subject. The second effectively averages each endpoint separately and then combines those averages; it discards the dependence induced by the shared effect. This distinction is the main reason that separately fitted endpoint likelihoods do not, by themselves, define a joint likelihood.

Worked Example: Continuous and Binary Endpoints

Hypothetical data and model. One subject has continuous response $Y=1.2$ units and binary response $R=1$. Suppose a shared latent effect takes values $b=+1$ and $b=-1$, each with probability $0.5$. Conditional on $b$, $Y\sim N(b,0.5^2)$, and $R$ is Bernoulli with probability $\operatorname{logit}^{-1}(0.2+0.8b)$. Assume conditional independence.

At $b=+1$, the normal density is

f_Y(1.2\mid +1)=\frac{1}{0.5\sqrt{2\pi}} \exp\left[-\frac{(1.2-1)^2}{2(0.5)^2}\right] =0.7365\ \text{per unit}.

The binary success probability is $\operatorname{logit}^{-1}(1)=0.7311$ if the linear predictor is $0.2+0.8b$. Thus the conditional joint contribution at $b=+1$ is $0.7365(0.7311)=0.5385$ per unit.

At $b=-1$, the binary predictor is $0.2-0.8=-0.6$, so its success probability is $\operatorname{logit}^{-1}(-0.6)=0.3543$. The normal density is

f_Y(1.2\mid -1)=\frac{1}{0.5\sqrt{2\pi}} \exp\left[-\frac{(1.2+1)^2}{2(0.5)^2}\right] \approx 0.0000499\ \text{per unit}.

The conditional joint contribution at $b=-1$ is approximately $0.0000499(0.3543)=0.0000177$ per unit. Integrating over the two-point distribution gives

L(1.2,1)=0.5(0.5385)+0.5(0.0000177)\approx 0.2693\ \text{per unit}.

The calculation combines the density and the binary probability at each shared $b$, then averages. Averaging the two endpoint contributions separately and multiplying would not produce this joint likelihood.

Worked Ordinal-and-Count Likelihood

The next example applies the same construction to two discrete endpoints. It also shows how an ordinal category probability is obtained by subtracting cumulative probabilities, rather than treating an ordinal score as a continuous measurement.

Worked Example: Ordinal Category and Poisson Count

Hypothetical data and model. A subject has ordinal outcome $Y=2$ on a three-category scale and count $C=2$. Let $b$ take values $+1$ and $-1$ with probabilities $0.5$ each. For the ordinal endpoint, use $P(Y\leq j\mid b)=\operatorname{logit}^{-1}(\tau_j-\lambda b)$, with $\tau_1=-1.6$, $\tau_2=-0.4$, and $\lambda=-0.6$. For the count, use $C\mid b\sim\operatorname{Poisson}\{\exp(0.5b)\}$. Assume conditional independence.

At $b=+1$, the two cumulative-logit arguments are $-1.6-(-0.6)(1)=-1$ and $-0.4-(-0.6)(1)=0.2$. Therefore,

P(Y=2\mid +1)=\operatorname{logit}^{-1}(0.2)-\operatorname{logit}^{-1}(-1) =0.549834-0.268941=0.280893.

The count mean is $\exp(0.5)=1.6487$, so

P(C=2\mid +1)=\exp\{-\exp(0.5)\}\frac{\exp(1)}{2} \approx 0.2613569.

The conditional joint contribution is $0.280893(0.2613569)\approx0.0734133$, or $0.0734$ to four significant figures.

At $b=-1$, the ordinal cumulative-logit arguments are $-2.2$ and $-1$, giving

P(Y=2\mid -1)=\operatorname{logit}^{-1}(-1)-\operatorname{logit}^{-1}(-2.2) =0.268941-0.0997505=0.169191.

The count mean is $\exp(-0.5)=0.6065$. Consequently,

P(C=2\mid -1)=\exp\{-\exp(-0.5)\}\frac{\exp(-1)}{2} \approx 0.1003.

The conditional joint contribution is approximately $0.169191(0.100291)=0.0169684$. Averaging the two conditional contributions gives

L(Y=2,C=2)=0.5(0.0734133)+0.5(0.0169684) =0.0451909\approx 0.0452.

The result is a joint probability for these discrete outcomes under the specified model. Both endpoint contributions at each latent-effect value are included before averaging over that effect.

Repeated Observations Share the Same Integration

If a subject contributes several observations, the product includes each observed response conditional on the subject’s latent effect. The effect is then integrated once for that subject, not once per observation. This is what allows the shared effect to induce marginal association across repeated responses.

Worked Example: Two Continuous Measurements and a Binary Response

Hypothetical data and model. A subject has two continuous measurements $Y_1=1$, $Y_2=1$, and a binary response $R=1$. Let $b=\pm1$ with equal probabilities. Conditional on $b$, the continuous measurements are independent with $Y_j\sim N(b,1)$, and $R$ is independent of them with success probability $\operatorname{logit}^{-1}(0.5b)$. Thus, conditional independence holds for all three responses.

At $b=+1$, each continuous density is $\phi(1;1,1)=1/\sqrt{2\pi}=0.398942$. Their product is $0.159155$. The binary probability is $\operatorname{logit}^{-1}(0.5)=0.622459$, giving a conditional contribution of $0.159155(0.622459)=0.09907$ per squared measurement unit.

At $b=-1$, each continuous density is $\phi(1;-1,1)=\exp(-2)/\sqrt{2\pi}=0.053991$. Their product is approximately $0.002915$. The binary probability is $\operatorname{logit}^{-1}(-0.5)=0.377541$, so the conditional contribution is $0.002915(0.377541)=0.001100$ per squared measurement unit.

The subject’s contribution is therefore $0.5(0.09907)+0.5(0.001100)=0.05009$ per squared measurement unit. All three terms were multiplied at each value of $b$; there was one average over the shared two-point distribution.

Diagnostics, Alternatives, and Limitations

A mathematically valid product can still describe the data poorly. Check that ordinal category probabilities are nonnegative and sum to one, that count probabilities use the intended count model, and that continuous densities use the appropriate scale and variance structure. Check whether the shared-effect distribution and the way $b_i$ enters each endpoint are scientifically defensible. A shared effect may induce association, but does not guarantee that the model captures every source or pattern of dependence.

Warning signs include systematic discrepancies in endpoint-specific predictions, poor calibration of joint outcome patterns, residual association after conditioning on the fitted latent effects, and sensitivity of results to plausible changes in the latent-effect distribution. Sparse combinations of endpoint outcomes can also make joint patterns difficult to assess. These checks should be made in context; agreement on each endpoint’s marginal distribution alone does not establish that the joint distribution is adequate.

Separate endpoint fits are simpler and may be useful when the goal is strictly endpoint-specific description, but they do not estimate the shared-latent-effect joint likelihood above. Copula models, covered in the earlier material on copula models, provide a different route to flexible dependence while retaining specified margins. Neither approach is universally preferable: a shared-effect model gives dependence a latent-variable interpretation, while a copula specifies dependence through a joint distribution construction. Both require assumptions that should be evaluated against the scientific question and the available data.

A common mistake is to multiply separately integrated likelihoods and call the result a joint likelihood. The reasoning is that each endpoint has already been averaged over the latent effect, so their product must represent the subject. The correction is to preserve the shared value within the subject: form the conditional product first, then integrate that product over the latent-effect distribution. Otherwise, the shared-effect dependence is lost.

Specify each response model.
Choose the appropriate density or probability mass function for each observed endpoint and state how its parameters depend on the latent effect and covariates.

Form the conditional contribution.
Under conditional independence, multiply the observed endpoint contributions for a subject at the same value of that subject’s latent effect.

Integrate once per subject.
Average the complete conditional product over the latent-effect distribution. For independent subjects, multiply the resulting subject-level contributions.

Key takeaway: A mixed-endpoint joint likelihood preserves the response model for each endpoint. Conditional on shared latent effects, multiply the relevant endpoint likelihood contributions; then integrate over the latent effects once for each subject.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A subject has a continuous and a binary endpoint that are conditionally independent given $b$. In what order are the two contributions combined and averaged over $b$?
For an ordinal endpoint, the fitted cumulative probabilities at a particular $b$ are $P(Y\leq1\mid b)=0.20$ and $P(Y\leq2\mid b)=0.65$. What is $P(Y=2\mid b)$?
Two endpoint contributions at $b=+1$ have product $0.12$, and at $b=-1$ have product $0.04$. If the two latent-effect values are equally likely, what is the joint contribution?
Why is the product of the two separately integrated endpoint likelihoods generally not the joint likelihood under a shared-effect model?

Solutions. (1) Multiply the continuous density by the binary probability conditional on the same $b$, then integrate that product over the distribution of $b$. (2) $P(Y=2\mid b)=P(Y\leq2\mid b)-P(Y\leq1\mid b)=0.65-0.20=0.45$. (3) The joint contribution is $0.5(0.12)+0.5(0.04)=0.08$. (4) Separately integrating first averages each endpoint without preserving their common latent-effect value; multiplying those averages generally does not retain the dependence induced by the shared effect.