NMF Objectives for Count Matrices

Why the Reconstruction Objective Matters for Counts

A count matrix is nonnegative, but that alone does not determine how it should be reconstructed. An error of 2 counts has a different relative meaning when the observed count is 3 than when it is 300. Squared error treats both discrepancies in absolute units; a Poisson-style objective instead arises from a model for how counts vary around their expected values.

Let $V$ be a $p \times n$ matrix of observed counts, with feature $i$ measured in sample $j$. A rank-$r$ nonnegative matrix factorization represents its fitted mean matrix as $M=WH$, where $W$ and $H$ have nonnegative entries and $M_{ij}=(WH)_{ij}$. For count data, the central question is whether $M_{ij}$ is a reasonable model for the expected count in that cell, and whether the assumptions behind that model are plausible.

Definition: The generalized KL divergence between a nonnegative data matrix $V$ and a nonnegative fitted mean matrix $M$ is $$ D_{\mathrm{KL}}(V\|M)=\sum_{i=1}^{p}\sum_{j=1}^{n}\left[V_{ij}\log\left(\frac{V_{ij}}{M_{ij}}\right)-V_{ij}+M_{ij}\right]. $$ For a cell with $V_{ij}=0$, its contribution is $M_{ij}$, using the convention $0\log(0/M_{ij})=0$. The divergence is defined when $M_{ij}>0$ wherever $V_{ij}>0$; a zero fitted mean for a positive observed count gives infinite loss.

The generalized KL divergence is also called the I-divergence. It is not a symmetric distance: in general, $D_{\mathrm{KL}}(V\|M)\ne D_{\mathrm{KL}}(M\|V)$. In this setting the observed counts occupy the first argument and the fitted means the second. The direction matters for the likelihood interpretation and for the numerical value.

From a Poisson Model to the KL Objective

A common probabilistic interpretation assumes that, conditional on the factors, each count is Poisson with mean $M_{ij}$, and that the entries are conditionally independent:

V_{ij}\mid W,H\sim\operatorname{Poisson}(M_{ij}),\qquad M_{ij}=(WH)_{ij}.

This model assumes that $M_{ij}$ is the expected count and that the conditional variance is also $M_{ij}$. Conditional independence means that the joint probability of the matrix is the product of the cell probabilities. These assumptions concern the data-generating model given the factors; merely obtaining nonnegative fitted values does not establish them.

For one cell, the Poisson log likelihood is $V_{ij}\log(M_{ij})-M_{ij}-\log(V_{ij}!)$. Summing over cells gives the log likelihood. The factorial term depends on the observed data but not on $W$ or $H$, so it can be omitted when optimizing the factors. Reversing the sign and adding another data-only term yields the generalized KL objective.

Justification: Under the independent Poisson model, the negative log likelihood, after dropping terms that do not depend on $M$, is $$ \sum_{i,j}\left[M_{ij}-V_{ij}\log(M_{ij})\right]. $$ For $V_{ij}>0$, subtracting the data-only quantity $V_{ij}-V_{ij}\log(V_{ij})$ from this expression gives $V_{ij}\log(V_{ij}/M_{ij})-V_{ij}+M_{ij}$. For $V_{ij}=0$, the same expression reduces to $M_{ij}$, because the log-likelihood contribution is $-M_{ij}$. Thus minimizing $D_{\mathrm{KL}}(V\|M)$ is equivalent to maximizing the independent Poisson likelihood.

The equivalence explains the objective; it does not prove the model is true. In particular, biological counts may vary more than a Poisson model allows, and entries can be associated through sample-level effects or other unmodeled structure. The loss can still be used as a useful criterion, but its literal likelihood interpretation then needs qualification.

Worked Example: Calculating a Poisson-Style Loss With a Zero

Suppose a hypothetical sample has observed counts $V=(3,0)$ for two features. Consider fitted means $M=(2,1)$. Calculate the generalized KL loss and interpret the zero-count contribution.

For the first feature, the contribution is $3\log(3/2)-3+2$. Since $\log(1.5)\approx0.4055$, this is $3(0.4055)-1\approx0.2164$.

For the second feature, the observed count is zero, so its contribution is $1$, not an undefined expression: $0\log(0/1)-0+1=1$.

The total is $D_{\mathrm{KL}}(V\|M)\approx0.2164+1=1.216$. Under the Poisson interpretation, the first fitted mean is 2 counts and the second is 1 count. A zero observation does not require a zero fitted mean; it contributes a penalty equal to that positive fitted mean.

Exposures, Library Depth, and What the Factors Mean

Samples can differ in sequencing depth or another known opportunity for observing counts. If exposure $E_{ij}>0$ is known, a natural model is $V_{ij}\sim\operatorname{Poisson}(E_{ij}\lambda_{ij})$, where $\lambda_{ij}$ is a rate and the expected count is $E_{ij}\lambda_{ij}$. If NMF represents the rates as $\lambda_{ij}=(WH)_{ij}$, the Poisson loss must compare the observed counts with $M_{ij}=E_{ij}(WH)_{ij}$, not with the rates alone.

Equivalently, one may include a known sample exposure $E_j$ for every feature in sample $j$, setting $M_{ij}=E_j(WH)_{ij}$. This distinguishes a high expected count caused by greater exposure from one caused by a higher modeled rate. Whether this is appropriate depends on the measurement design and the scientific target. Dividing counts by library size and then treating the resulting non-integer values as Poisson observations is not the same likelihood model.

Worked Example: Including a Known Exposure

In a hypothetical one-feature, two-sample example, let the exposures be $E=(3,1)$, the fitted rates be $\lambda=(2,5)$ counts per exposure unit, and the observed counts be $V=(9,4)$. Calculate the fitted expected counts and the loss.

The expected counts are $M=(E_1\lambda_1,E_2\lambda_2)=(3\cdot2,1\cdot5)=(6,5)$. The first sample contributes $9\log(9/6)-9+6\approx3.649-3=0.6492$.

The second contributes $4\log(4/5)-4+5\approx4(-0.2231)+1=0.1074$. Thus $D_{\mathrm{KL}}(V\|M)\approx0.6492+0.1074=0.7566$. Comparing the observed counts to the fitted rates $(2,5)$ instead would ignore the threefold exposure in the first sample and would answer a different modeling question.

Comparing Objectives on the Same Counts

Generalized KL loss and squared error can favor different reconstructions. Squared error sums $(V_{ij}-M_{ij})^2$, emphasizing absolute discrepancies. Generalized KL loss measures discrepancies relative to the fitted mean and includes a contribution for every cell, including zeros. The choice is not a claim that one objective is universally superior; it should reflect the measurement scale and the modeling purpose.

Worked Example: Two Candidate Reconstructions, Two Objectives

For a hypothetical one-row count matrix $V=(100,1)$, compare candidate fitted means $M_A=(100,3)$ and $M_B=(102.1,1)$. These are candidate reconstructions; the calculation compares their objectives rather than asserting that either is the true mean.

For squared error, candidate A has loss $(100-100)^2+(1-3)^2=4$. Candidate B has loss $(100-102.1)^2+(1-1)^2=2.1^2=4.41$. Squared error prefers A.

For generalized KL loss, candidate A has loss $0+[1\log(1/3)-1+3]\approx-1.0986+2=0.9014$. Candidate B has loss $[100\log(100/102.1)-100+102.1]+0\approx0.0217$. KL loss prefers B.

The reversal occurs because A fits the large count exactly but predicts 3 for the count of 1, while B makes a modest absolute change to the large count and fits the small count exactly. KL loss gives substantial relative importance to that small count discrepancy. The example illustrates an objective-dependent tradeoff, not a general rule that KL must favor a particular type of feature.

Objective	Typical interpretation	Potential concern for counts
Generalized KL loss	Poisson negative log likelihood up to data-only constants	Conditional Poisson variance and independence may be unrealistic
Squared error	Least squares; commonly associated with constant-variance Gaussian errors	Absolute errors can dominate when counts vary greatly in scale
Other beta-divergence losses	Alternative discrepancy weightings for nonnegative data	A chosen loss is not automatically a correctly specified count likelihood

Assumptions, Diagnostics, and Failure Modes

A Poisson-style objective is most directly justified when entries are counts from a process with conditional mean $M_{ij}$, conditional variance near that mean, and sufficiently independent entries given the modeled structure. Known exposures should be included in the expected counts. The model also requires nonnegative means, with positive means wherever positive counts occur. The factorization $M=WH$ imposes an additional structural restriction: the expected-count matrix must be representable by the chosen nonnegative factors.

Useful warning signs include variability much larger than the fitted means across comparable cells, clusters of zeros that the fitted means do not plausibly explain, and systematic patterns associated with sample depth or feature groups. These signs do not by themselves identify the correct alternative model, but they weaken a literal independent-Poisson interpretation. In such cases, KL can remain an optimization loss; uncertainty and fit claims should not be presented as though the Poisson sampling model had been validated.

Overdispersion: If conditional variance substantially exceeds the mean, a Poisson likelihood can understate variability. Consider whether a count model allowing overdispersion is more appropriate.
Dependence: Shared sample effects or correlated features violate conditional independence unless the modeled structure accounts for them.
Unmodeled depth: Differences in exposure can be mistaken for differences in rates if expected counts omit known depth factors.
Preprocessed values: Log-transformed or otherwise normalized measurements are not raw Poisson counts; applying a Poisson interpretation to them needs a separate justification.
Boundary problems: A fitted mean of zero where the observed count is positive has infinite KL loss. Numerical safeguards should not be mistaken for a statistical explanation of that mismatch.

A common mistake is to say, “The data are counts, so KL is automatically the correct objective.” The correction is that counts motivate a Poisson model only when its conditional mean, variance, independence, exposure, and factorization assumptions are defensible. Another mistake is to interpret $WH$ as a rate even when it was fitted directly to raw counts; without an exposure term, it represents expected counts.

Key takeaway: Choose a count-matrix objective by connecting it to the measurement process. Generalized KL loss is the independent-Poisson negative log likelihood up to a constant, but the interpretation depends on an appropriate mean model, exposures, and sampling assumptions.

Check Your Understanding

Answer each question, then compare with the solutions below.

For one cell with observed count 0 and fitted mean 2, what is its generalized KL contribution?
State the variance assumption in the conditional Poisson model used to motivate KL loss.
Counts are observed with known exposure $E=4$, and the modeled rate is $\lambda=1.5$. What expected count should enter the loss?
For $V=(20,2)$, compare squared-error losses for $M_A=(20,4)$ and $M_B=(22,2)$. Which does squared error prefer?
Give one reason that minimizing KL loss need not justify literal Poisson likelihood conclusions.

Solutions. (1) The contribution is $2$, by the zero-count convention. (2) Conditional on the factors, the Poisson variance equals its mean: $\operatorname{Var}(V_{ij}\mid W,H)=M_{ij}$. (3) The expected count is $E\lambda=4(1.5)=6$. (4) Candidate A has squared loss $(20-20)^2+(2-4)^2=4$; candidate B has $(20-22)^2+(2-2)^2=4$. They tie. (5) For example, substantial overdispersion violates the Poisson variance assumption, even if KL remains a usable reconstruction criterion.