Worked Binary Dose-Response Model

Why a Binary Dose-Response Model Needs Random Effects

A binary response records one of two outcomes, such as response or no response. Across subjects, the probability of response may rise with dose, but subjects can differ in both their baseline response tendency and their sensitivity to dose. A logistic nonlinear mixed-effects (NLME) model represents these features together: dose enters the response model, while subject-specific random effects allow the dose-response curves to vary.

The distinction between a subject-specific probability and a population-averaged probability is essential. The former conditions on that subject's random effects; the latter averages over the random-effects distribution. They are generally not the same because the logistic function is nonlinear.

Model: Let $Y_{ij}$ be the binary response for subject $i$ at observation $j$, and let $x_{ij}$ be dose in units of 100 mg. Use the random-intercept, random-slope logistic model $$ Y_{ij}\mid b_i \sim \operatorname{Bernoulli}(p_{ij}), \qquad \operatorname{logit}(p_{ij})=\log\left(\frac{p_{ij}}{1-p_{ij}}\right) =\beta_0+\beta_1x_{ij}+b_{0i}+b_{1i}x_{ij}. $$ Here $\beta_0$ and $\beta_1$ are fixed effects, and $b_{0i}$ and $b_{1i}$ are subject $i$'s random intercept and random dose slope. A common assumption is $b_i=(b_{0i},b_{1i})^\mathsf{T}\sim N(0,\Omega)$, independently across subjects, with the conditional responses independent given the random effects and covariates.

The model is conditional: $p_{ij}$ is a response probability given the subject's random effects and dose. With this coding, $\beta_1$ is the typical subject's log-odds change per 100 mg when $b_{1i}=0$; $\exp(\beta_1)$ is the corresponding odds ratio. The slope for subject $i$ is $\beta_1+b_{1i}$, not $\beta_1$ alone. A positive slope means that the subject's odds rise with dose; it does not mean that every subject must have a positive slope.

Fitting and Interpreting the Model

For one subject, conditional on $b_i$, the likelihood contribution from binary observations is the product of their Bernoulli probabilities. To obtain the likelihood contribution used for population-level estimation, the random effects must be integrated out:

L_i(\theta)=\int \left\{\prod_j p_{ij}(b_i)^{y_{ij}} [1-p_{ij}(b_i)]^{1-y_{ij}}\right\} \phi(b_i;0,\Omega)\,db_i.

Here $\theta$ denotes the fixed effects and random-effects parameters, and $\phi(b_i;0,\Omega)$ is the density of the assumed random-effects distribution. The integral is usually evaluated numerically during likelihood-based fitting. The fitted random effects for an observed subject are estimates conditional on that subject's data; they are not additional fixed effects shared by the population.

Worked Example: Interpreting a Hypothetical Model Fit

Suppose a logistic NLME model is fitted to a hypothetical study with repeated binary responses from 80 subjects across doses from 0 to 200 mg. Dose is represented as $x=\text{dose}/100\text{ mg}$. Suppose the fitted fixed effects are $\hat{\beta}_0=-2.00$ and $\hat{\beta}_1=1.20$, and the estimated random-effect standard deviations are 0.60 for the intercept and 0.40 for the dose slope, with estimated correlation zero.

For a subject whose random effects are both zero, the log odds at 0 mg are $-2.00$. The response probability is $1/(1+\exp(2.00))=0.1192$, or about 11.9%. At 100 mg, $x=1$, so the log odds are $-2.00+1.20=-0.800$, and the probability is $1/(1+\exp(0.800))=0.3100$, or 31.0%.

The fixed dose effect corresponds to an odds ratio of $\exp(1.20)=3.320$ per 100 mg for a subject with random slope zero. This is an odds ratio, not a 3.32-fold increase in probability. The random slope standard deviation of 0.40 indicates variation in log-odds change per 100 mg across subjects under the model. It does not mean that every subject's slope is positive.

The assumptions matter to interpretation and fitting. Dose must be represented on the specified scale, and the binary outcome is modeled as Bernoulli conditional on the random effects. The normal random-effects assumption describes the distribution of subject-level deviations; conditional independence states that after conditioning on these deviations and the covariates, observations have no further dependence. These are modeling assumptions, not consequences of observing repeated outcomes.

Subject-Specific Predictions and Likelihood Contributions

Once estimates of a subject's random effects are available, substituting them into the linear predictor gives that subject's conditional response probabilities. This calculation is distinct from averaging over all possible random effects. It is useful for illustrating how a fitted intercept or slope changes predictions for an individual subject.

Worked Example: A Subject With a Random Intercept and Slope

Use the fitted fixed effects $\hat{\beta}_0=-2.00$ and $\hat{\beta}_1=1.20$ above. Suppose a hypothetical subject has estimated random effects $\hat b_{0i}=0.30$ and $\hat b_{1i}=-0.20$. Calculate the subject's probabilities at 0 and 100 mg.

At 0 mg, $x=0$, so the random slope contributes nothing. The linear predictor is $-2.00+0.30=-1.70$. Thus $\hat p_{i}(0)=1/(1+\exp(1.70))=0.1545$, or 15.45%.

At 100 mg, $x=1$, and the linear predictor is $-2.00+1.20+0.30-0.20=-0.700$. Therefore $\hat p_i(100)=1/(1+\exp(0.700))=0.3318$, or 33.2%. The subject's net deviation from the typical log odds at 100 mg is $0.30-0.20=0.10$, so their fitted log odds are 0.10 higher than the fixed-effects-only log odds at that dose. Their dose slope is $1.20-0.20=1.00$ log-odds units per 100 mg, smaller than the typical slope of 1.20, but still positive.

If this subject had outcomes $y=0$ at 0 mg and $y=1$ at 100 mg, their likelihood conditional on these estimated random effects would be $(1-0.1545)(0.3318)=0.8455\times0.3318=0.2805$, using the rounded probabilities. This is a conditional contribution evaluated at estimated random effects, not the integrated subject likelihood used to fit the population model.

Population-Averaged Probabilities

A population-averaged response probability at dose $x$ averages the conditional probability over the random-effects distribution:

P(Y=1\mid x)=\int \operatorname{logit}^{-1}\!\left(\beta_0+\beta_1x+b_0+b_1x\right) \phi(b;0,\Omega)\,db.

This integral usually has no simple closed form for a logistic model with normal random effects. Numerical integration, such as Gaussian quadrature, can approximate it. The population average is not obtained by setting the random effects to zero: that substitution gives the probability for a subject at the center of the random-effects distribution. Nor is the average log odds generally equal to the logit of the average probability.

Worked Example: Averaging Over Random Effects

Continue with the hypothetical fitted model, assuming independent random effects with standard deviations $\sigma_0=0.60$ and $\sigma_1=0.40$. At $x=1$, the random contribution $b_0+b_1$ has mean zero and variance $0.60^2+0.40^2=0.52$. Therefore the population probability is the expectation of $\operatorname{logit}^{-1}(-0.80+U)$, where $U\sim N(0,0.52)$.

For illustration, approximate this one-dimensional expectation by five-point Gauss-Hermite quadrature. The standard-normal nodes are $0,\pm1.3556,\pm2.8570$, with weights $0.53333, 0.22208, 0.22208, 0.01126, 0.01126$, respectively. Since $\sqrt{0.52}=0.7211$, the corresponding linear predictors are approximately $-0.800,-1.778,0.178,-2.8602,1.2602$ when paired with nodes $0,-1.3556,1.3556,-2.8570,2.8570$. Their logistic probabilities are approximately $0.3100,0.1446,0.5443,0.0542,0.7791$.

The weighted average is $0.53333(0.3100)+0.22208(0.1446)+0.22208(0.5443)+0.01126(0.0542)+0.01126(0.7791)\approx0.3277$. Thus the population-averaged response probability at 100 mg is approximately 32.8%. The fixed-effects-only probability was 31.0%; the difference arises because the logistic transformation is nonlinear and the population contains variation in both random effects. The exact value depends on the model parameters and the numerical integration accuracy.

How a Random Dose Slope Changes Predictions

A random intercept shifts a subject's log odds by the same amount at every dose. A random slope changes the dose-response gradient, so the difference between subjects can grow, shrink, or change direction as dose changes. Specifically, the subject-specific log odds are $\beta_0+b_{0i}+(\beta_1+b_{1i})x$. The slope deviation is multiplied by $x$, so it has no effect at $x=0$, and its contribution increases in magnitude as the dose distance from zero grows.

This structure can produce crossing subject-specific curves. A subject with a high baseline response tendency but a low dose slope may have higher predicted probability at low dose and lower predicted probability than another subject at a sufficiently high dose. That possibility is scientifically meaningful only if it is supported by data and is plausible for the dose range being modeled.

Worked Example: Comparing Two Dose Slopes

Consider two hypothetical subjects with the same random intercept, $b_0=0.20$, but different random slopes: subject A has $b_1=0.30$, and subject B has $b_1=-0.50$. Use $\beta_0=-2.00$ and $\beta_1=1.20$.

At $x=0$, both have log odds $-2.00+0.20=-1.80$, giving probability $1/(1+\exp(1.80))=0.1419$. At $x=1$, subject A's log odds are $-2.00+0.20+1.20+0.30=-0.300$, so the probability is $0.4256$. Subject B's log odds are $-2.00+0.20+1.20-0.50=-1.10$, giving probability $0.2497$.

The difference between the subjects is zero at 0 mg but appears at 100 mg because their slopes differ by $0.80$ log-odds units per 100 mg. Subject B still has an increasing response probability over this dose range: their slope is $1.20-0.50=0.70$, which is positive. A negative random slope is not by itself evidence of a decreasing dose response; the total slope must be considered.

Assumptions, Diagnostics, and Alternatives

Before interpreting predictions, check whether the model structure is credible for the data and intended dose range. Relevant checks include whether the dose-response pattern is broadly compatible with a linear effect on the log-odds scale, whether fitted random-effect variation is identifiable from the number and placement of repeated doses, and whether the estimated random-effect distribution is a reasonable approximation. Sparse observations per subject can make individual slopes especially uncertain.

Warning signs include unstable estimates of random-slope variance, a correlation estimate near its boundary, convergence problems, or fitted probabilities that fail to represent observed response frequencies across dose. These signs do not prove a particular assumption is false, but they indicate that the model may be over-parameterized or poorly supported. Predictions far outside the observed dose range are particularly fragile: a linear log-odds slope can imply increasingly extreme probabilities without evidence that the same trend continues.

A simpler alternative is a random-intercept logistic model, which permits baseline heterogeneity but imposes a common dose slope. It can be preferable when repeated dose information is too limited to estimate slope variation. A probit mixed model changes the link function but retains conditional random effects and the distinction between subject-specific and population-averaged probabilities. These are alternatives to compare based on scientific interpretation and data support; neither removes the need to state the conditioning and assumptions.

Mistake: treating the fixed slope as every subject's slope. The individual slope is $\beta_1+b_{1i}$; the fixed slope describes the center of the random-effects distribution.
Mistake: calling the probability at zero random effects a population average. It is a conditional probability at the center of the distribution. The population average requires integration over random effects.
Mistake: interpreting an odds ratio as a probability ratio. $\exp(\beta_1)$ multiplies odds for the fixed-effects-only comparison; the probability change depends on the starting probability.
Mistake: assuming a negative random slope means response falls with dose. Check the total slope $\beta_1+b_{1i}$, not $b_{1i}$ alone.

Key takeaway: A binary dose-response NLME model combines a link-scale dose effect with subject-level variation. Substitution gives conditional subject predictions; integrating those probabilities over the random-effects distribution gives population-averaged predictions. A random dose slope changes how individual predictions evolve with dose.

Check Your Understanding

Answer the questions first, then compare your reasoning with the solutions.

In the model above, what is the subject-specific dose slope?
If $\beta_1=1.20$ and $b_{1i}=-0.50$, does that subject's fitted response probability decrease with dose?
Why is the probability at $b_0=b_1=0$ not generally the population-averaged probability?
What does $\exp(\beta_1)$ describe in this model?
When might a random-intercept model be a more defensible choice than a random-intercept-and-slope model?

Solutions. 1. The subject-specific slope on the log-odds scale is $\beta_1+b_{1i}$, per 100 mg. 2. No: the total slope is $1.20-0.50=0.70$, which is positive, so the fitted probability increases with dose. 3. The logistic function is nonlinear, so applying it at mean random effects does not generally equal averaging the transformed probabilities over the random-effects distribution. 4. It is the odds ratio per 100 mg for a subject with random slope zero, conditional on the model's random-effects formulation. 5. When the data have too few repeated dose observations per subject, or otherwise do not support reliable estimation of slope variability, the simpler model avoids an unsupported random-slope parameter.