From Latent Factors to Count and Binary Data
A factor model represents sample-level patterns with a small set of latent scores and describes how each measured feature responds to those scores. For continuous measurements, a Gaussian observation model may be appropriate. Counts and binary outcomes need different likelihoods: a count cannot be negative or fractional, and a binary measurement takes only values 0 and 1. The likelihood and link function connect these data types to the latent linear predictor without treating the observations as Gaussian.
Let \(i\) index samples, \(v\) index an omics view, \(j\) index a feature in that view, and \(k=1,\ldots,K\) index latent factors. Write \(z_{ik}\) for sample \(i\)'s score on factor \(k\), and \(\lambda_{jvk}\) for the loading of feature \(j\) in view \(v\) on that factor. The feature-specific linear predictor can be written as
Here \(\alpha_{jv}\) is a feature-specific intercept, \(x_i\) is a vector of measured covariates, \(\beta_{jv}\) is its feature-specific coefficient vector, and \(o_{ijv}\) is an offset when one is needed. If a model uses shared and view-specific scores, the sum can include the corresponding scores and loadings; the likelihood construction is unchanged. The scores vary across samples, while the loadings describe how a particular feature's predictor changes with those scores.
Count Likelihoods and the Log Link
A standard starting point for a nonnegative integer count \(Y_{ijv}\) is the Poisson distribution. Conditional on the latent scores, covariates, and model parameters, it has mean \(\mu_{ijv}>0\) and variance \(\mu_{ijv}\). The log link maps an unrestricted linear predictor to a positive mean:
These equations describe a conditional observation model: for given scores and covariates, the count follows the stated distribution. A common working assumption is that observations are conditionally independent across samples and features, given the model terms. This does not assert that the raw measurements are independent without conditioning; their dependence may be represented in part through shared latent scores. Whether that representation is adequate must be checked.
For counts measured over differing exposures, such as differing observation times, an exposure \(e_{ijv}>0\) can be included as an offset \(o_{ijv}=\log(e_{ijv})\). The model then implies \(\mu_{ijv}=e_{ijv}\exp(\alpha_{jv}+\sum_k\lambda_{jvk}z_{ik}+x_i^\mathsf{T}\beta_{jv})\). This assumes the expected count is proportional to exposure. A sequencing library-size offset can be useful as a modeling choice, but it does not by itself resolve the compositional nature of sequencing counts.
Worked Example: A Poisson Count With an Exposure Offset
Suppose a hypothetical feature has an observed count of \(y=36\) in a sample with exposure \(e=100\). Use one sample score \(z_{i1}=0.5\), a feature loading \(\lambda_{jv1}=0.4\), intercept \(\alpha_{jv}=-1\), and no additional covariates. Assume the conditional Poisson model and a log-exposure offset.
The predictor is \(\eta_{ijv}=\log(100)-1+(0.4)(0.5)=4.6052-1+0.2=3.8052\). Equivalently, the factor-dependent rate predictor excluding exposure is \(-0.8\). Therefore, \(\mu_{ijv}=100\exp(-0.8)=44.93\) counts. The observed count has conditional likelihood contribution \(P(Y_{ijv}=36)=\exp(-44.93)(44.93)^{36}/36!\).
The score-loading term is \(0.2\), so a one-unit increase in this score, with other terms held fixed, multiplies the expected count by \(\exp(0.4)\approx1.492\). This is a model-based conditional association, not evidence that changing the score would cause the count to change.
The Poisson variance restriction is often too strong for biological count data. An overdispersed alternative is the negative-binomial model. In one common parameterization, its mean is \(\mu\), its dispersion parameter is \(\phi>0\), and its variance is \(\mu+\mu^2/\phi\). As \(\phi\) becomes large, the variance approaches the Poisson variance. The same log-linear predictor can model \(\mu\); the likelihood differs by allowing extra conditional variation.
A brief derivation clarifies how the predictor relates to the data. For one Poisson observation, the log likelihood, omitting no terms that depend on \(\mu\), is \(\ell=y\log(\mu)-\mu-\log(y!)\). Since \(\log(\mu)=\eta\), differentiating with respect to \(\eta\) gives
The feature loading \(\lambda_{jvk}\) enters through \(\eta\), so the derivative with respect to that loading, holding the score fixed, is \(z_{ik}(y-\mu)\). This shows why the score and loading jointly determine how a sample's observed count contributes to estimation: the score scales the feature's count residual on the predictor scale.
Binary Likelihoods and Probability Links
For a binary feature, let \(Y_{ijv}=1\) indicate that the event or feature is present and \(Y_{ijv}=0\) indicate absence. A Bernoulli model uses a success probability \(p_{ijv}\). The logit link is common because it allows any real-valued predictor while producing a probability between zero and one:
Given the probability, the likelihood contribution is \(p_{ijv}^{y}(1-p_{ijv})^{1-y}\). A probit alternative uses \(\Phi^{-1}(p_{ijv})=\eta_{ijv}\), where \(\Phi\) is the standard normal cumulative distribution function. Both links produce valid probabilities; their coefficient scales differ, so coefficients from the two links should not be directly equated.
Worked Example: A Binary Feature Under a Logit Link
For a hypothetical binary feature, suppose \(\alpha_{jv}=0.2\), the sample scores are \(z_{i1}=1.2\) and \(z_{i2}=-0.4\), and the corresponding loadings are \(\lambda_{jv1}=0.5\) and \(\lambda_{jv2}=-0.25\). There are no other terms, and the observed value is \(y=1\).
The linear predictor is \(\eta_{ijv}=0.2+(0.5)(1.2)+(-0.25)(-0.4)=0.2+0.6+0.1=0.9\). The fitted probability is \(p_{ijv}=1/(1+\exp(-0.9))\approx0.7109\). Because \(y=1\), the likelihood contribution is \(p_{ijv}=0.7109\). If the observed value had been 0 instead, the contribution would be \(1-p_{ijv}=0.2891\).
Holding the other terms fixed, a one-unit increase in \(z_{i1}\) raises the log odds by \(0.5\), or multiplies the odds by \(\exp(0.5)\approx1.649\). It does not add 0.5 to the probability: the probability change depends on the starting predictor.
The Bernoulli log likelihood is \(\ell=y\log(p)+(1-y)\log(1-p)\). Under the logit link, differentiating with respect to \(\eta\) gives \(y-p\). Thus the loading derivative is \(z_{ik}(y-p)\), paralleling the Poisson expression \(z_{ik}(y-\mu)\), but the residual compares an observed binary value with a probability.
If each observation represents \(m\) independent trials with a common success probability and the recorded value is the number of successes \(y\), a binomial likelihood is appropriate instead: \(Y\sim\operatorname{Binomial}(m,p)\), with likelihood \({m\choose y}p^y(1-p)^{m-y}\). It is not appropriate to substitute a binomial model merely because data are recorded as integers; the trials and their number must have the stated interpretation.
Worked Example: A Binomial Count of Successes
Suppose a hypothetical assay records 6 positive outcomes among 8 independent trials, each with success probability \(p=0.7\). The probability of exactly 6 positives is \({8\choose6}(0.7)^6(0.3)^2=28(0.117649)(0.09)=0.2965\), to four significant figures.
This is a binomial likelihood contribution, not a Bernoulli contribution, because the recorded observation is the number of successes in eight trials. If the assay instead recorded only whether any positive outcome occurred, that recorded indicator would require a different probability derived from the trial model.
Assumptions, Diagnostics, and Model Choice
The observation model should match how the measurement was generated. In particular, assess whether the count is a nonnegative integer, whether a Poisson variance is plausible conditional on the modeled effects, whether binary values represent the event of interest, and whether the assumed conditional independence is reasonable. For an offset, check that exposure is positive and that proportionality to exposure is scientifically defensible. As with other likelihood models, unmodeled batch effects or dependence can remain even after including latent factors.
| Observation and model | Mean structure | Useful when | Important check |
|---|---|---|---|
| Poisson with log link | \(\log(\mu)=\eta\) | Counts with conditional variance near the mean | Excess variability or systematic residual patterns |
| Negative binomial with log link | \(\log(\mu)=\eta\) | Counts with extra conditional variation | Whether remaining dispersion and zero frequency are adequately described |
| Bernoulli with logit link | \(\operatorname{logit}(p)=\eta\) | One binary outcome per sample-feature pair | Probability calibration and residual structure |
| Bernoulli with probit link | \(\Phi^{-1}(p)=\eta\) | Binary outcomes when a probit link is suitable | Calibration and sensitivity to the link choice |
| Binomial with logit link | \(\operatorname{logit}(p)=\eta\) | A known number of trials summarized by successes | Trial definition and common-probability assumption |
For counts, compare observed and fitted distributions across samples and features, and examine whether variability substantially exceeds the Poisson mean. For binary data, compare observed event frequencies with fitted probabilities across ranges of predicted probability; persistent disagreement suggests poor calibration. In either case, residual patterns by sample, feature, view, or known technical covariates can reveal structure not captured by the predictor. A very large number of zeros may prompt investigation, but zero inflation should not be added automatically: it needs a defensible data-generating interpretation.
The main alternatives differ in the assumptions they make, not just in their formulas. Poisson is a parsimonious count model with a mean-variance equality. Negative binomial allows overdispersion, at the cost of an additional dispersion parameter. Bernoulli logit and probit models encode the same binary outcome but use different probability scales. A binomial model is useful for grouped trials, whereas applying Bernoulli likelihoods separately is appropriate when the individual binary outcomes are recorded and modeled separately.
None of these likelihood choices guarantees that the factor structure is correct. Conditional independence may fail, effects may be nonlinear, and sparse counts can make a feature's parameters weakly informed by the data. A chosen link can also fit poorly. For count data, a library-size offset does not automatically address compositional effects; for binary data, fitted probabilities can be poorly calibrated even when the estimated coefficients are numerically stable. Conclusions about factor scores and loadings therefore depend on both the latent structure and the observation model.
Common Mistakes
- Using a Gaussian likelihood just because counts are numeric. Counts are nonnegative integers, and a Gaussian model can assign probability to impossible negative or fractional values. Use a count likelihood when that matches the observation process.
- Applying a log link to a binary probability. The log link does not constrain its inverse to the interval from 0 to 1. A logit or probit link is designed to produce a valid probability.
- Reading a loading as a direct change in the outcome. In a log-linked count model, a loading changes the log mean; in a logit model, it changes log odds. Neither is generally a constant additive change in the observed outcome.
- Calling every excess-zero data set zero-inflated. Many zeros can arise under ordinary Poisson or negative-binomial models. Check overall fit and the measurement mechanism before adding a distinct zero-generating process.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In a log-linked count model, what does a loading \(\lambda=0.3\) imply for the expected count when the corresponding score increases by one unit, all else fixed?
- A binary feature has predictor \(\eta=0\) under a logit link. What is its fitted probability?
- When is a binomial likelihood more appropriate than a Bernoulli likelihood?
- What variance assumption distinguishes a Poisson count model from the negative-binomial parameterization used here?
- Why does adding a library-size offset not, by itself, settle all concerns about sequencing counts?
Solutions. (1) The log mean rises by \(0.3\), so the expected count is multiplied by \(\exp(0.3)\approx1.350\), not increased by a fixed 0.3 counts. (2) \(p=1/(1+\exp(0))=0.5\). (3) Use a binomial likelihood when the recorded value is the number of successes among a known number of trials with a common success probability; a single binary outcome is Bernoulli. (4) Poisson assumes variance \(\mu\), while this negative-binomial model has variance \(\mu+\mu^2/\phi\). (5) The offset adjusts expected counts for exposure under proportionality, but it does not by itself address compositional dependence or other features of the sequencing measurement process.