Why a Pair of Logistic Models Needs a Compatibility Check
In fully conditional specification, separate models describe each variable conditional on the other variables. For a pair of binary variables, each conditional model can look reasonable on its own, yet the two models may not describe any single joint distribution. Compatibility asks whether there is one joint probability table whose conditional distributions are exactly the two specified models.
This tutorial considers two binary variables \(X\) and \(Y\), each coded 0 or 1. The focus is the precise constraint on the two cross-predictor effects. The result relies on full support: all four joint cells have positive probability, so the relevant conditional odds and log odds are finite.
The intercept \(a\) is the log conditional odds of \(X=1\) when \(Y=0\); \(c\) is the log conditional odds of \(Y=1\) when \(X=0\). The coefficients \(b\) and \(d\) describe how the other variable changes those conditional log odds. Compatibility requires those cross-effects to agree, not the intercepts.
Deriving the Equality Constraint
Let \(p_{xy}=P(X=x,Y=y)\). Under full support, each \(p_{xy}>0\). The odds ratio for the \(2\times2\) table is $$ \operatorname{OR}=\frac{p_{11}p_{00}}{p_{10}p_{01}}. $$ It can be calculated from either conditional model. In the model for \(X\mid Y\), the odds of \(X=1\) at \(Y=0\) are \(p_{10}/p_{00}\), and at \(Y=1\) are \(p_{11}/p_{01}\). Therefore, $$ \frac{p_{11}/p_{01}}{p_{10}/p_{00}} =\frac{p_{11}p_{00}}{p_{10}p_{01}} =\operatorname{OR}. $$ The logistic specification says these same odds are \(e^a\) and \(e^{a+b}\), respectively. Their ratio is \(e^b\), so \(\operatorname{OR}=e^b\).
The conditional model for \(Y\mid X\) gives the same table odds ratio from the other direction. At \(X=0\), the odds of \(Y=1\) are \(p_{01}/p_{00}\); at \(X=1\), they are \(p_{11}/p_{10}\). Thus, $$ \frac{p_{11}/p_{10}}{p_{01}/p_{00}} =\frac{p_{11}p_{00}}{p_{10}p_{01}} =\operatorname{OR} =e^d. $$ A single joint table has one odds ratio, so \(e^b=e^d\). Since the exponential function is one-to-one, \(b=d\). This proves necessity.
It is also sufficient. Suppose \(b=d\). Define four positive, unnormalized cell weights by $$ w_{00}=1,\qquad w_{10}=e^a,\qquad w_{01}=e^c,\qquad w_{11}=e^{a+c+b}. $$ Let \(Z\) be their sum and set \(p_{xy}=w_{xy}/Z\). For \(Y=0\), the conditional odds for \(X=1\) are \(w_{10}/w_{00}=e^a\); for \(Y=1\), they are \(w_{11}/w_{01}=e^{a+b}\). These give the first logistic model. For \(X=0\), the conditional odds for \(Y=1\) are \(w_{01}/w_{00}=e^c\); for \(X=1\), they are \(w_{11}/w_{10}=e^{c+b}=e^{c+d}\), using \(b=d\). These give the second model. Normalizing the weights does not change conditional odds. Hence a joint distribution exists, proving sufficiency.
Worked Example: Constructing a Compatible Joint Table
Suppose the two specified models are \(\operatorname{logit}P(X=1\mid Y=y)=-0.4+0.7y\) and \(\operatorname{logit}P(Y=1\mid X=x)=0.2+0.7x\). The parameters are finite, so each conditional probability is strictly between 0 and 1. The slopes agree, satisfying the compatibility condition.
Using the construction, the weights are \(w_{00}=1\), \(w_{10}=e^{-0.4}=0.6703\), \(w_{01}=e^{0.2}=1.2214\), and \(w_{11}=e^{-0.4+0.2+0.7}=e^{0.5}=1.6487\). Their sum is \(Z=4.5404\). Dividing each weight by this sum gives the joint table:
| \(Y=0\) | \(Y=1\) | |
|---|---|---|
| \(X=0\) | \(p_{00}=0.2202\) | \(p_{01}=0.2690\) |
| \(X=1\) | \(p_{10}=0.1476\) | \(p_{11}=0.3631\) |
The rounded cells sum to approximately 1.000, as expected. Check one conditional: at \(Y=0\), \(P(X=1\mid Y=0)=0.1476/(0.2202+0.1476)\approx0.4013\). The specified model gives \(\operatorname{expit}(-0.4)=1/(1+e^{0.4})\approx0.4013\). At \(Y=1\), the model gives \(\operatorname{expit}(0.3)\approx0.5744\). The joint table therefore corresponds to the given \(X\mid Y\) model; the weight construction establishes that it also corresponds to the \(Y\mid X\) model.
The table’s odds ratio is \(e^{0.7}\approx2.014\). Thus, in this joint distribution, the conditional odds of \(X=1\) are about 2.014 times as large at \(Y=1\) as at \(Y=0\), and the conditional odds of \(Y=1\) are also about 2.014 times as large at \(X=1\) as at \(X=0\).
Diagnosing a Mismatch
If the fitted slopes differ, the conditional specifications demand different odds ratios for the same joint table. No choice of intercepts can remove that disagreement: the intercepts determine the baseline conditional odds in each model, while the mismatch is in the shared association parameter. This is an algebraic incompatibility, not merely a concern about how well an algorithm mixes.
Worked Example: Showing That Two Conditionals Are Incompatible
Consider \(\operatorname{logit}P(X=1\mid Y=y)=-0.4+0.7y\) and \(\operatorname{logit}P(Y=1\mid X=x)=0.2+0.2x\). Both are valid full-support logistic models because all parameters are finite. The first requires the table odds ratio to be \(e^{0.7}\approx2.014\).
The second requires the same odds ratio to be \(e^{0.2}\approx1.221\). These cannot both be the odds ratio of one joint table, since \(2.014\ne1.221\). Equivalently, compatibility would require \(b=d\), but \(0.7\ne0.2\). The pair is therefore incompatible.
Changing \(a=-0.4\) or \(c=0.2\) can change the conditional baseline odds, but it cannot change the ratio of odds across the two levels of the predictor. The slope mismatch remains.
Worked Example: Recovering Compatible Slopes From a Joint Table
Suppose a hypothetical joint distribution has \(p_{00}=0.30\), \(p_{10}=0.20\), \(p_{01}=0.10\), and \(p_{11}=0.40\). The entries are positive and sum to 1, so the full-support condition holds.
For the \(X\mid Y\) model, the intercept is \(a=\log(p_{10}/p_{00})=\log(0.20/0.30)=\log(2/3)\approx-0.4055\). Its slope is the change in conditional log odds: \(b=\log\{(p_{11}/p_{01})/(p_{10}/p_{00})\}=\log\{(0.40/0.10)/(0.20/0.30)\}=\log(6)\approx1.792\).
For the \(Y\mid X\) model, the intercept is \(c=\log(p_{01}/p_{00})=\log(0.10/0.30)=\log(1/3)\approx-1.099\). Its slope is \(d=\log\{(p_{11}/p_{10})/(p_{01}/p_{00})\}=\log\{(0.40/0.20)/(0.10/0.30)\}=\log(6)\approx1.792\). Thus \(b=d\), as the compatibility result requires. The shared odds ratio is \(6\).
Practical Checks, Alternatives, and Limitations
For a pair of fitted binary logistic conditionals with a 0/1 predictor, a direct compatibility check is to compare the two cross-predictor coefficients. The equivalent check is to exponentiate each coefficient and compare the implied conditional odds ratios. When the models are estimated from data, coefficient estimates will generally be uncertain; a close numerical match is not proof that the population conditionals are exactly compatible. The algebraic criterion concerns the specified distributions, not sampling variation in estimated coefficients.
A useful comparison is with specifying a joint distribution first. A saturated joint model for two binary variables has four cell probabilities that sum to 1, or three free parameters. It automatically yields a compatible pair of conditionals. In contrast, fitting two logistic regressions separately allows the two slopes to differ, so compatibility is not guaranteed. Imposing a shared slope is one way to constrain the pair so that it can come from a joint table.
This result assumes both variables have full support and that the conditional models use the stated 0/1 coding. If a cell has probability zero, an odds ratio may be zero, infinite, or undefined, and finite logistic coefficients need not describe that boundary case; the full-support derivation does not apply without modification. The equality is also about the two specified binary conditionals only. It is not, by itself, a check of compatibility for a larger collection of variables.
A common mistake is to compare intercepts and conclude that unequal intercepts imply incompatibility, or that equal intercepts guarantee compatibility. Neither conclusion is correct: intercepts represent baseline conditional odds, while the equality constraint concerns the slopes. Another mistake is to treat a stable sequence of updates as evidence that the conditionals share a joint distribution. Stability of an algorithm and algebraic compatibility are distinct questions.
Check Your Understanding
For each question, use the full-support assumption unless the question says otherwise. Solutions are included below the questions.
- For \(\operatorname{logit}P(X=1\mid Y=y)=0.3-0.5y\) and \(\operatorname{logit}P(Y=1\mid X=x)=-0.1-0.5x\), state whether the pair is compatible and give its implied odds ratio.
- A pair of models has slopes \(b=0.4\) and \(d=0.6\). What odds ratios do the models imply, and why is the pair incompatible?
- Explain what the intercept \(a\) means in the model for \(X\mid Y\), and why it does not determine the marginal odds for \(X\).
- If a joint table has odds ratio \(3\), what slopes must its two binary logistic conditionals have?
Solutions. (1) The slopes agree, so the pair is compatible; its odds ratio is \(e^{-0.5}\approx0.6065\). (2) The two implied odds ratios are \(e^{0.4}\approx1.492\) and \(e^{0.6}\approx1.822\). They differ, but one joint table can have only one odds ratio, so the models are incompatible. (3) \(a=\log\{P(X=1\mid Y=0)/P(X=0\mid Y=0)\}\), the log odds conditional on \(Y=0\). Marginal odds for \(X\) require averaging over the distribution of \(Y\), not just using this conditional value. (4) Each conditional slope must equal the log odds ratio, \(\log(3)\approx1.099\).