MiST Models for Mixtures of Fixed and Random Effects

Why Separate Common and Heterogeneous Effects?

A rare-variant set can contain effects that point in a broadly shared direction, effects that vary in size, or both. A burden test is designed to detect a shared direction, but opposing effects can cancel in its aggregate. A variance-component test can detect dispersion in effects without requiring them to point in the same direction, but it does not use a shared direction as directly. MiST (Mixture of Subset Tests) models these two possibilities together: one component describes a common effect, and another describes residual heterogeneity across variants.

The purpose is not to declare every variant causal or to estimate every effect precisely. Rather, the joint test asks whether the set contributes through either the common component, the heterogeneous component, or both. The earlier material on burden and variance-component tests supplies the interpretation of those two components; here the focus is their model-based combination and calibration.

Definition: Let $G_i=(G_{i1},\ldots,G_{im})^T$ be the allele-count vector for individual $i$ at $m$ variants in a prespecified set. A MiST effect model writes the vector of variant effects as $$ \beta=\mu a+b,\qquad E(b)=0,\qquad \operatorname{Var}(b)=\tau D. $$ Here $a$ is a prespecified direction, often the vector of ones or a specified weighting direction; $\mu$ is its shared effect coefficient; $b$ represents variant-specific departures; $D$ is a prespecified positive semidefinite covariance or weight matrix; and $\tau\geq 0$ controls the heterogeneity. The joint null is $H_0:\mu=0,\ \tau=0$.

When $a$ is the vector of ones, each variant has the same modeled effect $\mu$ before the departures $b_j$ are added. Nonuniform entries in $a$ can encode a planned weighted direction. The covariance matrix $D$ determines how the random departures are weighted and correlated; a common simple choice is diagonal, with larger diagonal entries allowing more variability for variants given greater weight. These choices are part of the model and should be specified before examining the association result.

Model and Score Components

For a quantitative outcome, a basic linear version is

Y_i=X_i^T\alpha+G_i^T(\mu a+b)+\varepsilon_i,\qquad \varepsilon_i\sim N(0,\sigma^2).

The covariates $X_i$ include an intercept and any prespecified adjustment variables. The random effects $b$ are modeled as mean-zero and independent of the residual errors, with covariance $\tau D$. Conditional on $b$, the residuals are independent with common variance $\sigma^2$. Under the joint null, the set has neither the modeled shared effect nor random effect variation, while the covariate model remains in place.

For a binary outcome, an analogous model can use a logistic mean model, with the same decomposition of $\beta$. In either setting, score testing fits the null model first and evaluates the score for $\mu$ and for $\tau$ at $\mu=\tau=0$. The exact score and calibration depend on the outcome likelihood and the implementation. The following Gaussian derivation makes the roles of the two scores explicit.

In the linear model, let $X$ be the $n\times q$ covariate matrix, $G$ the $n\times m$ genotype matrix, and $P_X=I-X(X^TX)^{-1}X^T$, assuming $X$ has full column rank. Under the null, $r=P_XY$ is the residual after removing the fitted covariate space. Define $g=P_XGa$ and $A=P_XGDG^TP_X$. For known $\sigma^2$, the efficient score for the common component and the score for the variance component are

U_\mu=\frac{g^Tr}{\sigma^2},\qquad U_\tau=\frac{r^TAr-\sigma^2\operatorname{tr}(A)}{2\sigma^4}.

The common score is linear in the residuals: it measures whether the outcome residuals align with the genotype combination $Ga$. The variance score is a centered quadratic form: it measures whether the residual pattern has more alignment with the genotype-set directions than expected under the null. The centering term makes $E(U_\tau\mid H_0)=0$ in this Gaussian model. Since $\tau$ cannot be negative, evidence for heterogeneity is in the positive direction of $U_\tau$, not the negative direction.

Why the Joint Calibration Needs Care

Under the Gaussian null, let $\lambda_1,\ldots,\lambda_m$ be the nonzero eigenvalues of $A$. The residual can be represented as a projected Gaussian vector, which gives

U_\tau\ \overset{d}{=}\ \frac{1}{2\sigma^2}\sum_k\lambda_k(Z_k^2-1), \qquad Z_k\sim N(0,1)\ \text{independently}.

Thus the random-effect score is generally a weighted sum of centered chi-square variables, not a single standard normal score and not automatically a chi-square with one degree of freedom. The fixed score is normal under the stated Gaussian assumptions. Although their covariance is zero in this model because centered Gaussian variables have zero third moments, zero covariance alone does not establish independence. A joint test must therefore calibrate its statistic under the joint null rather than combine component p-values as if they were independent.

One transparent score-based construction is to standardize each component, use a two-sided contribution for $\mu$, and a one-sided positive contribution for $\tau$. Define $Z_\mu=U_\mu/\sqrt{\operatorname{Var}(U_\mu)}$, $Z_\tau=U_\tau/\sqrt{\operatorname{Var}(U_\tau)}$, and

T=Z_\mu^2+\{\max(0,Z_\tau)\}^2.

This is a joint statistic, not a claim that $T$ has a $\chi^2_2$ null distribution. Its null distribution can be obtained by simulating residuals from the fitted null model, recalculating both scores and $T$ for each replicate, and comparing the observed statistic with that simulated distribution. For a discrete simulation with $B$ replicates and $C$ simulated statistics at least as large as the observed one, a common estimate is $(C+1)/(B+1)$. Other MiST implementations may use a different joint score construction or calibration; the essential requirement is to account for the joint null behavior of the components.

Worked Example: Reading the Two Effect Components

Suppose a hypothetical three-variant set is modeled with $a=(1,1,1)^T$, and its effect vector is represented as $\beta=\mu a+b$. Let the fitted shared effect be $\mu=0.30$ log-odds units per allele, and let the departures be $b=(0.20,-0.25,0.05)^T$.

The modeled effects are obtained by adding the shared value to each departure:

\beta=(0.30+0.20,\ 0.30-0.25,\ 0.30+0.05)^T =(0.50,\ 0.05,\ 0.35)^T.

All three effects are positive in this illustration, but their magnitudes differ substantially. The fixed component describes a shared positive direction; the random component describes departures from that common level. In this illustrative parameterization, the effects are log-odds changes per additional allele, not odds ratios. The model does not establish that the variants are causal, and fitted values alone do not test the joint null.

Worked Example: Computing and Calibrating a Joint Score

Consider hypothetical null-fit score output for a prespecified set: $U_\mu=3.00$, $\operatorname{Var}(U_\mu)=4.00$, $U_\tau=2.00$, and $\operatorname{Var}(U_\tau)=4.00$. These are assumed to be the efficient scores and their null variances from a correctly fitted model.

Standardize each component:

Z_\mu=\frac{3.00}{\sqrt{4.00}}=1.50,\qquad Z_\tau=\frac{2.00}{\sqrt{4.00}}=1.00.

Both scores are positive, so the one-sided random-effect contribution is retained. The joint statistic is

T=1.50^2+\{\max(0,1.00)\}^2=2.25+1.00=3.25.

Suppose, for this hypothetical example, that 5,199 of 99,999 null simulations produce a statistic at least 3.25. Using the plus-one estimate gives $p=(5,199+1)/(99,999+1)=0.0520$. At the 0.05 significance level, this simulated result is not below the threshold. It is evidence calibrated for the joint null under the assumed null model; it is not the probability that the null hypothesis is true.

Assumptions, Diagnostics, and Alternatives

A MiST analysis depends on more than the algebra of its scores. The variant set, allele coding, direction $a$, and covariance or weight structure $D$ must be defined coherently. The null outcome model and covariate adjustment must be adequate; residual dependence, population structure, relatedness, or an incorrect binary-outcome model can distort score calibration if not handled. Rare alleles also make asymptotic approximations less reliable in some settings, so a suitable null simulation or other justified calibration is important. Simulation must reproduce the null fitting and any structure in the actual analysis rather than draw independent scores with assumed marginal distributions.

Useful checks include confirming convergence and fit of the null model, verifying that the simulated null statistics reproduce their intended behavior, and checking that the reported p-value uses the observed statistic and the correct tail. For the random component, inspect the eigenvalues or numerical rank underlying its quadratic form; a few dominant directions can make a generic chi-square approximation poor. A very small or near-zero eigenvalue spectrum can also indicate that the set contributes little independent information after covariate adjustment. These checks do not prove the assumptions, but can reveal implementation or calibration problems.

MiST’s main advantage is its explicit joint sensitivity to a shared direction and heterogeneous effects. A burden test is often more focused when effects are expected to align, and may be easier to interpret as an aggregate direction. A variance-component test is a natural alternative when effect directions or sizes vary; it can lose power when effects are nearly uniform and aligned. MiST accommodates both components but adds modeling and calibration choices. The joint result does not by itself identify which variants contribute, determine their individual effects, or resolve whether the set definition is biologically appropriate.

Worked Example: Checking a Random-Score Calculation

For a small hypothetical Gaussian score calculation, suppose the nonzero eigenvalues of $A$ are $4.00$ and $0.75$, and $\sigma^2=1$. The random-effect score has null representation

U_\tau=\frac{1}{2}\{4.00(Z_1^2-1)+0.75(Z_2^2-1)\},

where $Z_1,Z_2$ are independent standard normal variables. Its null mean is zero, and its variance is

\operatorname{Var}(U_\tau) =\frac{1}{4}\{2(4.00^2)+2(0.75^2)\} =\frac{1}{2}(16+0.5625)=8.28125.

The standard deviation is therefore $\sqrt{8.28125}=2.8777$. This calculation shows why treating the raw quadratic score as a one-degree-of-freedom chi-square statistic would ignore the eigenvalue weights. It also gives the score variance needed for standardization; it does not, by itself, provide the joint p-value, which still requires calibration of the chosen joint statistic.

Common Mistakes and Practical Interpretation

Calling $\mu$ the effect of every variant without qualification. Each modeled effect is $\mu a_j+b_j$; the random departure can make individual effects differ in size or direction.
Using only a burden interpretation for a significant joint result. The joint test can be driven by the common component, the heterogeneous component, or both. Examine component scores as descriptive evidence, but do not treat their separate unadjusted p-values as the calibrated joint test.
Assigning a standard chi-square reference distribution to the combined statistic. The variance score has a quadratic-form null distribution, and the joint dependence matters. Use a valid joint calibration.
Assuming a nonsignificant result proves no variant-set effect. It indicates that this test did not provide sufficient evidence under its model and calibration; limited sample size, allele frequencies, weights, and effect architecture can all affect power.

Key takeaway: MiST represents variant effects as a shared direction plus mean-zero heterogeneity and tests $H_0:\mu=0,\tau=0$. Its strength is joint sensitivity to both architectures; valid inference requires a joint null calibration appropriate to the outcome model and score construction.

Check Your Understanding

Answer each question, then compare with the solutions below.

In $\beta=\mu a+b$, what do $\mu$, $a$, and $b$ represent?
Why is the variance-component score one-sided in the direction of positive evidence?
If standardized scores are $Z_\mu=-1.20$ and $Z_\tau=0.80$, calculate $T=Z_\mu^2+\{\max(0,Z_\tau)\}^2$.
Why is it not generally justified to compare this joint $T$ directly with a $\chi^2_2$ distribution?
Name one situation in which a burden test may be preferable as a focused test, and one reason MiST requires additional care.

Solutions. (1) $\mu$ is the shared coefficient, $a$ specifies its variant direction, and $b$ contains mean-zero variant departures with covariance controlled by $\tau D$. (2) The variance parameter satisfies $\tau\geq0$, so evidence for additional random-effect variation is in the positive score direction. (3) $T=(-1.20)^2+(0.80)^2=1.44+0.64=2.08$. (4) The random score is generally a weighted quadratic form, and the components’ joint null behavior is not necessarily that of two independent standard normal scores. (5) A burden test can be a focused choice when effects are expected to align; MiST needs care because its joint statistic requires calibration that respects the component distributions and dependence.