Mixed-Type Graphical Models for Omics Features

Why Mixed-Type Networks Need Conditional Models

A multi-omics network may include continuous measurements, binary indicators, and categorical states in the same sample. A Gaussian graphical model gives a useful description when all modeled features are continuous and jointly Gaussian, but its conditional distributions do not directly describe binary or categorical nodes. A mixed graphical model instead specifies a suitable conditional distribution for each node and uses the way that distribution changes with its neighbors to define edges.

The key is that nodewise models cannot be treated as unrelated prediction problems if the goal is a single undirected network. Their conditional distributions must be compatible with one joint distribution. The earlier material on graphical models and pseudolikelihood establishes why conditionals can be used for network estimation; here the focus is how to specify them for different feature types and interpret their parameters.

Definition: A mixed graphical model is an undirected graphical model whose nodes may have different variable types and whose nodewise conditional distributions are chosen to suit those types. An edge joins two nodes when their variables are conditionally dependent given all the other modeled variables. A nodewise coefficient represents an edge only when the conditional models are compatible with a common joint distribution.

A Pairwise Joint Model and Its Conditionals

Let \(Y_1,\ldots,Y_J\) be continuous features, \(B_1,\ldots,B_K\) binary features coded as 0 or 1, and \(C_1,\ldots,C_L\) categorical features. For each categorical feature, choose one level as a reference level. A useful pairwise model has a joint density or mass function proportional to the exponential of node and pairwise potentials:

\begin{aligned} \log p(y,b,c) ={}& \text{constant} -\frac{1}{2}y^\mathsf{T}Qy+a^\mathsf{T}y +\sum_k h_k b_k+\sum_l \eta_l(c_l)\\ &+\sum_{k<m}\kappa_{km}b_kb_m +\sum_{l<r}\psi_{lr}(c_l,c_r) +\sum_{j,k}\lambda_{jk}y_jb_k\\ &+\sum_{j,l}\gamma_{jl}(c_l)y_j +\sum_{k,l}\xi_{kl}(c_l)b_k . \end{aligned}

Here \(Q\) is a symmetric positive-definite matrix; that condition ensures the continuous part is integrable, including across the finite set of discrete configurations. The functions \(\eta_l\), \(\psi_{lr}\), \(\gamma_{jl}\), and \(\xi_{kl}\) encode categorical main effects or interactions. Reference-level coding makes their contrasts interpretable; an interaction between a continuous feature and a categorical feature can have a separate coefficient for each nonreference level. In conditional sums, extend the pairwise parameters to reversed endpoint indices by setting \(\kappa_{km}=\kappa_{mk}\) when \(k>m\), and \(\psi_{lr}(c_l,c_r)=\psi_{rl}(c_r,c_l)\) when \(l>r\).

Conditioning on all other nodes leaves only terms involving the node being conditioned upon. This gives three familiar conditional forms. For a continuous node,

Y_j\mid Y_{-j},B,C \sim N\left( \frac{a_j-\sum_{r\ne j}Q_{jr}Y_r+ \sum_k\lambda_{jk}B_k+\sum_l\gamma_{jl}(C_l)} {Q_{jj}}, \frac{1}{Q_{jj}} \right).

Thus its conditional mean is linear in the other continuous and binary features, with level-specific shifts from categorical features. For a binary node, the conditional distribution is Bernoulli, with log odds

\log\frac{P(B_k=1\mid\text{others})}{P(B_k=0\mid\text{others})} =h_k+\sum_{m\ne k}\kappa_{km}B_m +\sum_j\lambda_{jk}Y_j+\sum_l\xi_{kl}(C_l).

For a categorical node, the conditional probability of level \(c\) is proportional to the exponential of all terms involving that level:

P(C_l=c\mid\text{others})= \frac{\exp\left\{\eta_l(c)+ \sum_{r\ne l}\psi_{lr}(c,C_r)+ \sum_k\xi_{kl}(c)B_k+\sum_j\gamma_{jl}(c)Y_j\right\}} {\sum_d\exp\left\{\eta_l(d)+ \sum_{r\ne l}\psi_{lr}(d,C_r)+ \sum_k\xi_{kl}(d)B_k+\sum_j\gamma_{jl}(d)Y_j\right\}}.

This is a multinomial logistic conditional distribution, with one category serving as the reference for the log-odds contrasts. In these equations, “others” means all modeled nodes except the node on the left-hand side.

Edge rule: In this pairwise model, a continuous-continuous edge is present when \(Q_{jr}\ne0\); a continuous-binary edge when \(\lambda_{jk}\ne0\); and a continuous-categorical edge when at least one level contrast in \(\gamma_{jl}\) is nonzero. A discrete pair has an edge only when its interaction potential cannot be written as a sum of functions of the separate endpoints: for binary-binary nodes this requires \(\kappa_{km}\ne0\); for binary-categorical nodes, \(\xi_{kl}(c)\) must vary across categorical levels; and for categorical-categorical nodes, \(\psi_{lr}\) must have a nonzero cross-difference \(\psi_{lr}(c,d)-\psi_{lr}(c',d)-\psi_{lr}(c,d')+\psi_{lr}(c',d')\) for some levels \(c,c',d,d'\). These are conditional-dependence edges, not causal effects.

Why the Conditional Forms Follow

The conditionals can be checked directly from the joint model. When isolating \(Y_j\), the terms involving it have the form \(-Q_{jj}Y_j^2/2+t_jY_j\), where \(t_j=a_j-\sum_{r\ne j}Q_{jr}Y_r+\sum_k\lambda_{jk}B_k+\sum_l\gamma_{jl}(C_l)\). Completing the square gives a normal distribution with mean \(t_j/Q_{jj}\) and variance \(1/Q_{jj}\). For \(B_k\), the terms involving it are \(B_k\) times its displayed log-odds; since \(B_k\) can only be 0 or 1, normalizing the two possible weights gives the Bernoulli logistic conditional. For \(C_l\), retaining each possible level’s terms and normalizing their exponentials gives the categorical formula above.

This derivation also explains why parameters for an undirected edge are shared across endpoint conditionals. For example, the continuous-binary interaction \(\lambda_{jk}Y_jB_k\) appears both in the conditional mean of \(Y_j\) and in the conditional log odds of \(B_k\). Independently fitting two regressions need not enforce this shared interaction structure.

Worked Example: A Continuous Feature and a Binary Feature

Suppose two hypothetical omics features are a standardized continuous measurement \(Y\) and a binary indicator \(B\). Consider the joint model

p(y,b)\ \propto\ \exp\left(-\frac{1}{2}y^2+0.2y+0.6yb-0.4b\right).

The quadratic coefficient of \(y\) is negative, so the continuous conditional is integrable for either value of \(b\). Holding \(B=b\) fixed, the terms involving \(y\) are \(-y^2/2+(0.2+0.6b)y\). Therefore,

Y\mid B=b\sim N(0.2+0.6b,1).

In particular, the conditional mean is \(0.2\) standardized measurement units when \(B=0\), and \(0.8\) units when \(B=1\). In the other direction, the conditional log odds are \(-0.4+0.6y\). At \(y=1\), the log odds equal \(-0.4+0.6(1)=0.2\), so the conditional probability is

P(B=1\mid Y=1)=\frac{e^{0.2}}{1+e^{0.2}} =\frac{1.2214}{2.2214}\approx0.5498.

The interaction coefficient \(0.6\) is nonzero, so the two nodes have an edge. This indicates conditional association in the specified model; it does not establish that the continuous measurement causes the binary state or vice versa.

Worked Example: A Three-Level Categorical Conditional

Let \(C\) be a hypothetical three-level feature with reference level 0 and levels 1 and 2. Suppose its conditional log-odds relative to level 0, given binary feature \(B\), are \(0.2+0.5B\) for level 1 and \(-0.4+0.2B\) for level 2. At \(B=1\), the two log-odds are \(0.7\) and \(-0.2\). The unnormalized conditional weights for levels 0, 1, and 2 are therefore \(1,e^{0.7},e^{-0.2}\).

Using \(e^{0.7}\approx2.01375\) and \(e^{-0.2}\approx0.81873\), the normalizing sum is \(1+2.01375+0.81873=3.83248\). Hence

\begin{aligned} P(C=0\mid B=1)&=\frac{1}{3.83248}\approx0.26093,\\ P(C=1\mid B=1)&=\frac{2.01375}{3.83248}\approx0.52544,\\ P(C=2\mid B=1)&=\frac{0.81873}{3.83248}\approx0.21363. \end{aligned}

The probabilities sum to approximately 1.00000, allowing for rounding. Because the level-specific log odds change with \(B\), the binary-categorical interaction is nonzero and the two nodes are conditionally dependent in this model. The interpretation is relative to the chosen reference level: for example, the level-1 versus level-0 log-odds contrast increases by 0.5 when \(B\) changes from 0 to 1.

Worked Example: Checking Compatibility of Gaussian Conditionals

Suppose two hypothetical continuous features are separately modeled as \(X_1\mid X_2\sim N(0.5X_2,1)\) and \(X_2\mid X_1\sim N(0.2X_1,1)\). Can these be the conditionals of one nonsingular bivariate Gaussian distribution?

For a Gaussian joint distribution with precision matrix \(Q\), the conditional variance of \(X_1\) is \(1/Q_{11}\), and its regression slope on \(X_2\) is \(-Q_{12}/Q_{11}\). Thus the slope divided by the conditional variance equals \(-Q_{12}\). Applying the same identity to \(X_2\) gives the compatibility requirement

\frac{\text{slope of }X_1\mid X_2}{\operatorname{Var}(X_1\mid X_2)} = \frac{\text{slope of }X_2\mid X_1}{\operatorname{Var}(X_2\mid X_1)}.

The proposed values give \(0.5/1=0.5\) on the left and \(0.2/1=0.2\) on the right. Since these are unequal, the conditionals cannot come from one common Gaussian joint distribution. By contrast, equal slopes of 0.5 with both conditional variances 1 would correspond to \(Q_{11}=Q_{22}=1\), \(Q_{12}=-0.5\); this matrix is positive definite because its eigenvalues are \(0.5\) and \(1.5\). Compatibility is a modeling condition, not a cosmetic constraint.

Assumptions, Diagnostics, and Alternatives

The pairwise model above assumes that observations (samples) are independent draws from the same joint distribution, that the variables are coded and measured consistently, and that the chosen conditional forms are adequate. It assumes Gaussian conditionals with constant conditional variance for continuous nodes, logistic conditionals for binary nodes, and multinomial logistic conditionals for categorical nodes. The positive-definite \(Q\) condition is essential for a proper joint distribution. The pairwise specification also omits higher-order interactions, such as an effect that appears only for a particular combination of three nodes.

Before interpreting estimated edges, check the data and fitted conditionals. Useful warning signs include separation or extremely large logistic coefficients, categorical levels with very few observations, implausible or strongly varying residual spread for continuous nodes, and systematic prediction errors across levels of a neighboring categorical feature. Strongly dependent features or sparse categories can make edge estimates unstable. In a fitted joint model, also verify the required symmetry and positive-definiteness constraints rather than relying on apparently plausible separate regressions.

A Gaussian graphical model is simpler when all variables are continuous and its distributional assumptions are reasonable, but it does not directly provide Bernoulli or categorical node conditionals. Separate nodewise regressions allow each node to be modeled flexibly, but may produce incompatible edge estimates unless symmetry and other compatibility constraints are imposed. A mixed graphical model provides type-appropriate conditionals within a common pairwise structure, at the cost of stronger modeling assumptions. No single choice removes the need to examine fit, coding, sparsity, and uncertainty.

Common Mistakes and Limitations

Reading a conditional coefficient causally. An edge records conditional dependence given the modeled nodes; it does not by itself identify a direction or causal mechanism.
Treating every nodewise fit as a joint network. Independently estimated conditionals may not be compatible. Enforce or assess the constraints required by the joint model.
Interpreting a categorical coefficient without its reference level. Multinomial coefficients are contrasts; changing the reference level changes their numerical representation, not the underlying conditional distribution.
Using an unsuitable conditional family. A Gaussian conditional can be poor for a skewed or heavy-tailed continuous measurement, while rare categories can make categorical contrasts unreliable.
Assuming pairwise edges capture every dependency. A pairwise model may miss higher-order structure, and unmeasured or omitted variables can affect the conditional associations represented among included nodes.

Key takeaway: Choose a node’s conditional family to match its data type, but define network edges through a compatible joint model. The edge is the nonconstant interaction linking two nodes, interpreted as conditional dependence rather than causation.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a mixed pairwise model, what conditional distribution is used for a binary node, and what does its logit describe?
A continuous-categorical interaction is zero for every nonreference category contrast. Is there an edge from that continuous node to the categorical node under the stated model?
For a Gaussian pair, the conditional slopes are 0.3 in both directions, and both conditional variances are 0.6. Do these values meet the slope-to-variance compatibility requirement?
Why does a nonzero edge coefficient not establish a causal effect?

Solutions. (1) A binary node has a Bernoulli conditional distribution; its logit is the log odds of value 1 versus 0 given all other modeled nodes. (2) No. If all level contrasts are zero, the conditional distribution does not depend on that continuous node through this pairwise interaction. (3) Yes: both slope-to-variance ratios equal \(0.3/0.6=0.5\). This satisfies the stated requirement, though other conditions, including a valid positive-definite precision matrix, are still needed for a joint Gaussian model. (4) The edge encodes conditional association in an undirected model and supplies no direction of influence or control of unmeasured causes.