Marginal Mean Models for Clustered Responses

What a Marginal Mean Model Specifies

When outcomes are collected repeatedly from the same person, or from several units within a group, the observations form clusters. A central modeling question is what response level to describe. A marginal mean model describes the expected value of each outcome at specified observed covariates. It does not, merely by doing so, specify every aspect of how outcomes in the same cluster occur together.

Use $i$ to denote a cluster and $j$ to denote an outcome within that cluster. Let $Y_{ij}$ be the response and let $X_i$ denote the observed covariates used to describe the cluster’s outcomes. The covariates may include values that vary across responses within a cluster, such as measurement time. Write $E[Y_{ij}\mid X_i]$ for the conditional expectation given those observed covariates.

Definition: A marginal mean model specifies the coordinate-wise conditional expectations $E[Y_{ij}\mid X_i]=\mu_{ij}(\beta)$, for each outcome $j$ in cluster $i$, where $\mu_{ij}(\beta)$ is a stated function of the observed covariates and model parameter $\beta$. The specification is correct for a parameter value $\beta_0$ if these equalities hold with $\beta=\beta_0$ almost surely for the covariate values under study.

Here, “marginal” refers to modeling each response’s mean rather than specifying the whole joint law of the cluster’s responses. The expectation is still conditional on the observed covariates. These are two distinct features: the model can condition on measured information while specifying only one mean for each response, not the joint distribution of the response vector.

The collection of responses in a cluster can be written $Y_i=(Y_{i1},\ldots,Y_{im_i})$, where $m_i$ is the number of outcomes in that cluster. Its conditional joint distribution is the distribution of this entire vector given $X_i$. A marginal mean model constrains that distribution through the expectations of its coordinates, but does not usually determine the distribution itself. In particular, a mean specification alone does not assert that outcomes within a cluster are independent.

Worked Examples

Worked Example: Means for Two Binary Measurements

Suppose each cluster has a binary treatment indicator $A_i$, equal to 0 or 1, and two binary responses measured at times $t=0$ and $t=1$. Consider the mean specification $\mu_{ij}=0.20+0.10A_i+0.05t_j$. Since each response is binary, its expected value is its probability of being 1.

For a cluster with $A_i=1$, the mean at time 0 is $0.20+0.10(1)+0.05(0)=0.30$. The mean at time 1 is $0.20+0.10(1)+0.05(1)=0.35$. Thus the two conditional response probabilities are 0.30 and 0.35.

The expected sum of the two responses is $0.30+0.35=0.65$. This mean specification says nothing yet about the probability that both responses equal 1. Several different joint distributions can have these same two response probabilities.

Worked Example: Same Binary Means, Different Joint Distributions

Let $Y_1$ and $Y_2$ be binary responses at fixed covariates, with $P(Y_1=1)=0.30$ and $P(Y_2=1)=0.60$. First take the responses to be independent. Then the four joint probabilities are $P(1,1)=0.30(0.60)=0.18$, $P(1,0)=0.30(0.40)=0.12$, $P(0,1)=0.70(0.60)=0.42$, and $P(0,0)=0.70(0.40)=0.28$. They are nonnegative and sum to $0.18+0.12+0.42+0.28=1$.

Now consider a different joint distribution: assign probabilities 0.30, 0, 0.30, and 0.40 to outcomes $(1,1),(1,0),(0,1),(0,0)$, respectively. These probabilities are also nonnegative and sum to 1. The first response has probability $0.30+0=0.30$ of being 1; the second has probability $0.30+0.30=0.60$ of being 1. Thus both joint distributions have the same marginal means, but they are not the same joint distribution. In the second one, the event $Y_1=1$ always occurs together with $Y_2=1$; in the first, the responses are independent.

Worked Example: Conditional Means and an Overall Mean

Let $X$ be a binary observed covariate with $P(X=0)=P(X=1)=0.5$, and let $Y$ be a binary response. Suppose the conditional means are $E[Y\mid X=0]=0.20$ and $E[Y\mid X=1]=0.80$. Because $Y$ is binary, these are also the two conditional probabilities of a response equal to 1.

Averaging over the covariate distribution gives $E[Y]=0.5(0.20)+0.5(0.80)=0.50$. A model with $E[Y\mid X]=0.50$ at both covariate values would have the same overall mean, since $0.5(0.50)+0.5(0.50)=0.50$. But it would not reproduce either of the original conditional means. The overall average alone therefore cannot tell us whether the mean varies with the observed covariate.

What the Means Determine

For a binary outcome, a conditional mean determines its one-response conditional distribution. This is a special property of binary responses: their only possible values are 0 and 1.

Theorem: If $Y$ takes values in $\{0,1\}$, then for any observed covariates $X$, $E[Y\mid X]=\mu(X)$ if and only if $P(Y=1\mid X)=\mu(X)$ and $P(Y=0\mid X)=1-\mu(X)$, almost surely. Consequently, a valid binary-response mean must satisfy $0\leq\mu(X)\leq1$ almost surely.

Proof. Conditional on $X$, the response equals 1 with probability $P(Y=1\mid X)$ and 0 with probability $P(Y=0\mid X)$. Therefore $E[Y\mid X]=1\cdot P(Y=1\mid X)+0\cdot P(Y=0\mid X)=P(Y=1\mid X)$. Since the two probabilities sum to 1, $P(Y=0\mid X)=1-E[Y\mid X]$. A probability lies between 0 and 1, so the conditional mean does as well. Conversely, if the two stated probabilities hold, the same expectation calculation gives $E[Y\mid X]=\mu(X)$. $\square$

This result concerns one response at a time. It does not say that coordinate-wise means determine the distribution of a cluster vector. The following characterization makes that distinction precise for two binary responses.

Theorem: Suppose $Y_1,Y_2\in\{0,1\}$ have conditional means $p_1=E[Y_1\mid X]$ and $p_2=E[Y_2\mid X]$, where $X$ is fixed or conditioned upon. Their joint distribution is determined by the additional quantity $q=P(Y_1=1,Y_2=1\mid X)$. The four conditional probabilities are $$ \begin{aligned} P(Y_1=1,Y_2=1\mid X)&=q,\\ P(Y_1=1,Y_2=0\mid X)&=p_1-q,\\ P(Y_1=0,Y_2=1\mid X)&=p_2-q,\\ P(Y_1=0,Y_2=0\mid X)&=1-p_1-p_2+q. \end{aligned} $$ They are valid probabilities exactly when $\max(0,p_1+p_2-1)\leq q\leq\min(p_1,p_2)$.

Proof. The probability of $(1,1)$ is $q$ by definition. The event $Y_1=1$ consists of $(1,1)$ and $(1,0)$, so the probability of $(1,0)$ must be $p_1-q$. Similarly, the probability of $(0,1)$ must be $p_2-q$. Subtracting those three cell probabilities from 1 gives the probability $1-p_1-p_2+q$ of $(0,0)$.

For all four probabilities to be nonnegative, the first cell requires $q\geq0$, the second requires $q\leq p_1$, the third requires $q\leq p_2$, and the fourth requires $q\geq p_1+p_2-1$. Combining these inequalities gives the stated interval. Conversely, any $q$ in this interval makes all four cells nonnegative, and their sum is $q+(p_1-q)+(p_2-q)+(1-p_1-p_2+q)=1$. The resulting distribution has the required two means, because its probabilities of $Y_1=1$ and $Y_2=1$ are $q+(p_1-q)=p_1$ and $q+(p_2-q)=p_2$. Thus, when the interval allows more than one $q$, the same means are compatible with different joint distributions. $\square$

In the numerical example with $p_1=0.30$ and $p_2=0.60$, the allowed interval is $\max(0,0.30+0.60-1)=0$ to $\min(0.30,0.60)=0.30$. The independent distribution uses $q=0.18$, while the second distribution uses $q=0.30$. Both values are allowed, confirming directly that the specified means do not select one joint distribution.

Interpreting the Model Carefully

A marginal mean model is a target for the response expectations given observed covariates. A conditional joint distribution is a richer object: it describes which combinations of outcomes can occur and with what probabilities. Even when the mean functions are fully specified, additional information is needed to determine such features as the association between two responses in a cluster.

This distinction matters when describing what a model asserts. A claim about $E[Y_{ij}\mid X_i]$ is not a claim that outcomes are independent, nor is it a complete probability model for the response vector. Conversely, an assumption about the joint distribution includes implications for the coordinate means, but those implications must be checked against the intended mean specification.

The conditioning information also matters. If means vary with observed covariates, averaging over different covariate distributions can produce different overall means. In the third example, the overall mean of 0.50 depends on the specified equal probabilities for $X=0$ and $X=1$. It is not a replacement for the two conditional means, and it need not apply to a population with a different distribution of $X$.

Common Mistake

Wrong reasoning: “I have written down a mean model for every response, so I have specified the model for the cluster; therefore the outcomes in a cluster are independent.” The conclusion does not follow. Coordinate-wise expectations do not determine the joint distribution, as the two valid tables with the same binary means show.

Correction: State separately what the mean model specifies and what, if anything, is assumed about the joint distribution or within-cluster dependence. Do not add independence as an unstated consequence of a mean equation. Also keep conditional means given observed covariates distinct from averages obtained after averaging over a covariate distribution.

Check Your Understanding

Use the distinction between coordinate-wise means and a cluster’s joint distribution to answer the following.

For a binary response $Y$, why does $E[Y\mid X]=0.35$ determine the conditional probability that $Y=1$?
For two binary responses with means 0.20 and 0.50, find the allowed interval for $P(Y_1=1,Y_2=1\mid X)$.
Does specifying the conditional means of all responses in a cluster imply that the responses are independent? Explain.
If $X$ has two equally likely values and the conditional means are 0.10 and 0.90, calculate the overall mean. Why does that overall mean not describe the two conditional means?
What additional probability is needed, beyond the two means, to specify the joint distribution of two binary responses?