Why Different Coefficients Can Fit the Same Data
A batch-and-condition model may describe the observed expression means perfectly even when its batch and condition coefficients cannot be determined separately. The key is to distinguish the coefficient vector from the fitted means it generates. A direction in coefficient space that the design matrix cannot detect changes the coefficients without changing any fitted value.
This tutorial makes that distinction precise using the null space of the design matrix. The result is useful for understanding which numerical coefficient estimates are arbitrary, which combinations are determined by the data, and why predictions at some design configurations can be unique even when individual coefficients are not. The introductory material on batch-condition confounding establishes why the design may be rank deficient; here the focus is the algebra of equivalent coefficients and identifiable predictions.
For a gene, write the observed expression vector as \(y=X\beta+\varepsilon\), where \(X\beta\) represents the model mean and \(\varepsilon\) represents deviations from that mean. The linear-algebra results below concern the design and the mean \(X\beta\); they do not require normally distributed errors. They do require that the proposed mean is represented by this linear model. In a single-cell analysis, the rows might represent cells or sample-level observations, depending on the analysis, and the columns encode the included batch, condition, and other terms.
Null-Space Shifts and Equivalent Coefficients
If \(v \in \mathcal{N}(X)\), then $$ X(\beta+v)=X\beta+Xv=X\beta. $$ Thus \(\beta\) and \(\beta+v\) produce exactly the same fitted values. More generally, if one coefficient vector fits a given mean vector, every vector obtained by adding a null-space vector fits that same mean vector.
In a rank-deficient model, at least one nonzero null-space vector exists. The coefficients are then not uniquely determined by the fitted means. This does not imply that every function of the coefficients is ambiguous: a particular contrast or prediction may be unchanged by every null-space shift.
This is the linear-algebra form of the criterion used in the earlier material on which contrasts remain estimable. A prediction at an observed design row is identifiable because that row is, by definition, in the row space. A proposed prediction at a new configuration need not be. In particular, a configuration combining a batch and condition that never occur together may require a prediction that the observed design cannot identify.
Worked Example: Perfect Confounding Gives Equivalent Coefficients
Suppose six hypothetical sample-level observations have log-expression values of 9, 10, and 11 units in batch A under control, and 15, 16, and 17 units in batch B under treatment. There are no observations in batch A under treatment or in batch B under control. Use the model \(y_i=\beta_0+\beta_B B_i+\beta_C C_i+\varepsilon_i\), where \(B_i=1\) for batch B and \(C_i=1\) for treatment.
The design rows are \((1,0,0)\) for batch A control and \((1,1,1)\) for batch B treatment. The vector \(v=(0,1,-1)^T\) is in the null space because both types of row give a zero product with \(v\): \(1(0)+0(1)+0(-1)=0\) and \(1(0)+1(1)+1(-1)=0\). Consequently, increasing the batch coefficient by any amount \(t\) and decreasing the condition coefficient by the same amount leaves fitted values unchanged.
The two group means are \((9+10+11)/3=10\) units and \((15+16+17)/3=16\) units. The least-squares fit therefore requires \(\beta_0=10\) and \(\beta_B+\beta_C=6\). For example, both \((10,6,0)^T\) and \((10,0,6)^T\) fit the same means. The first predicts \(10\) for batch A control and \(10+6+0=16\) for batch B treatment. The second predicts \(10\) and \(10+0+6=16\), respectively.
The observed group difference of \(16-10=6\) log-expression units is identified as the sum \(\beta_B+\beta_C\), not as either component separately. The residuals within the two groups are \(-1,0,1\) in each, so the residual sum of squares is \(1+0+1+1+0+1=4\) units squared for either coefficient vector. The identical fitted values necessarily give identical residuals and the same least-squares fit.
Why Fitted Values Remain Unique
Rank deficiency means least-squares coefficients may not be unique; it does not mean the least-squares fitted vector is ambiguous. The least-squares problem chooses a vector in the column space of \(X\) that minimizes its squared distance from \(y\). The closest vector in a fixed subspace is unique: it is the orthogonal projection of \(y\) onto that subspace. Different coefficient vectors may represent that same projected vector.
To see how all coefficient solutions are related, suppose \(\beta_1\) and \(\beta_2\) produce the same fitted vector. Then $$ X\beta_1=X\beta_2 \quad \Longrightarrow \quad X(\beta_1-\beta_2)=0 \quad \Longrightarrow \quad \beta_1-\beta_2 \in \mathcal{N}(X). $$ Conversely, adding any \(v \in \mathcal{N}(X)\) to a coefficient solution leaves its fitted vector unchanged. Thus the full set of coefficient vectors giving the least-squares fit is one solution plus the null space.
This justification is algebraic. A probability model for \(\varepsilon\), such as a normal-error model, may be introduced for inference, but it does not remove the design's null space. A software package may return one particular coefficient vector, for example by imposing a constraint or choosing a generalized-inverse solution. That numerical choice does not make the separately ambiguous coefficients identifiable from the data.
Worked Example: Redundant Coding Creates a Different Kind of Ambiguity
Consider four hypothetical group means, with the order batch A control, batch A treatment, batch B control, and batch B treatment. Their log-expression means are 8, 10, 11, and 13 units. Suppose the design has an intercept, an indicator for batch A, an indicator for batch B, and an indicator for treatment. Its rows are \((1,1,0,0)\), \((1,1,0,1)\), \((1,0,1,0)\), and \((1,0,1,1)\).
The null vector \(v=(-1,1,1,0)^T\) satisfies \(Xv=0\): every row has an intercept contribution of \(-1\) and exactly one batch-indicator contribution of \(+1\). A coefficient vector \(\beta=(8,0,3,2)^T\) gives fitted values \(8,10,11,13\). Adding \(t v\) gives \((8-t,t,3+t,2)^T\). The four fitted values remain, in order, \(8-t+t=8\), \(8-t+t+2=10\), \(8-t+3+t=11\), and \(8-t+3+t+2=13\).
Here the ambiguity comes from including both batch indicators and an intercept: the two batch indicators sum to the intercept column. It is a coding redundancy, not a lack of observed batch-condition combinations. For example, omitting the batch A indicator and using an intercept, a batch B indicator, and a treatment indicator removes this particular null direction. By contrast, the ambiguity in the previous example arose because batch and condition changed together in every observed group. Identifying the source of the null space matters when deciding whether a different parameterization resolves the ambiguity.
Checking Whether a Prediction Is Identifiable
For a proposed design row \(x_0^T\), the prediction is \(x_0^T\beta\). Under a null-space shift \(\beta+v\), its change is \(x_0^Tv\). The prediction is therefore invariant to all equivalent coefficient choices exactly when \(x_0^Tv=0\) for every \(v\) in the null space. This is equivalent to \(x_0\) belonging to the row space of \(X\).
Worked Example: Observed and Unobserved Design Rows
Use the perfectly confounded design from the first example. Its two distinct rows are \(r_1=(1,0,0)\) for batch A control and \(r_2=(1,1,1)\) for batch B treatment. Every fitted coefficient vector has the form \(\beta=(10,6-t,t)^T\), since the observed means require \(\beta_0=10\) and \(\beta_B+\beta_C=6\).
For the observed batch B treatment row, \(x_0=(1,1,1)\), the predicted mean is \(x_0^T\beta=10+(6-t)+t=16\) units for every \(t\). The same is true for the batch A control row, whose prediction is \(10\) units. These rows are in the design's row space.
Now consider the unobserved batch A treatment row, \(x_0=(1,0,1)\). Its predicted mean is \(x_0^T\beta=10+t\), which changes with \(t\). For \(t=0\), this is 10 units; for \(t=6\), it is 16 units. The model and observed data do not select between these values. Algebraically, this row is not orthogonal to the null vector: \((1,0,1)v=-1\), which is not zero. The condition-only contrast has the same problem: \(c=(0,0,1)^T\) gives \(c^Tv=-1\), so \(\beta_C\) is not identifiable. In contrast, the observed group difference uses \(c=(0,1,1)^T\), and \(c^Tv=0\), so \(\beta_B+\beta_C\) is identifiable.
Diagnostics, Alternatives, and Common Pitfalls
A practical diagnostic is to examine the design's rank or compute a basis for its null space. For each null-space direction, inspect which coefficients change and which combinations remain fixed. For a coefficient contrast or proposed prediction, test whether its vector has zero inner product with every null-space basis vector. Equivalently, check whether the vector lies in the row space. A software-reported coefficient table alone may obscure this structure, especially if the software silently drops a column or uses a constraint.
Three approaches to ambiguity answer different questions:
| Approach | What it clarifies | What it does not establish |
|---|---|---|
| Null-space analysis | Which coefficient changes leave all fitted values unchanged | Which ambiguous coefficient value is scientifically correct |
| Reparameterization or reference coding | Whether redundant columns can be removed while preserving the model's fitted-value space | Whether a comparison absent from the design becomes identified |
| Prediction or contrast check | Whether a particular linear function is invariant across equivalent coefficients | Whether the model's assumptions or biological interpretation are adequate |
A common mistake is to interpret the one coefficient vector returned by software as proof that batch and condition effects have been separated. In a rank-deficient design, a program can return numbers, but the null-space shifts show that multiple coefficient vectors give the same fit. The correction is to state which quantities are invariant, such as a particular fitted mean or coefficient combination, and to avoid interpreting a non-identifiable component on its own.
Another mistake is to conclude that rank deficiency makes all predictions unusable. Fitted values at observed design rows remain unique, and some other contrasts can also be identifiable. The relevant question is not simply whether \(X\) is rank deficient, but whether the particular contrast or prediction is orthogonal to every null-space direction. These checks concern identifiability under the specified model; they do not guarantee that the model adequately represents expression or that an identified quantity has a causal interpretation.
Check Your Understanding
Answer each question, then compare with the solutions below.
- For design rows \((1,0,0)\) and \((1,1,1)\), verify that \(v=(0,1,-1)^T\) is in the null space. Which coefficient combination involving \(\beta_B\) and \(\beta_C\) is identified by the second row relative to the first?
- In the same design, suppose \(\beta=(4,3,1)^T\). Find the fitted values at both observed rows after adding \(2v\). What happens to the prediction at row \((1,0,1)\)?
- For a design with an intercept and indicators for both batch A and batch B, explain why \((-1,1,1)^T\) is a null-space vector when each observation belongs to exactly one batch.
- In the first design, is the contrast \(c^T\beta\) identifiable for \(c=(0,1,1)^T\)? Is it identifiable for \(c=(0,0,1)^T\)? Justify each answer using the null-space criterion.
Solutions. (1) Multiplication gives \(Xv=(0,0)^T\), since both rows have equal second and third entries. The first row gives \(\beta_0\), while the second gives \(\beta_0+\beta_B+\beta_C\); their difference identifies \(\beta_B+\beta_C\). (2) The shifted coefficients are \((4,5,-1)^T\). The observed-row fitted values are \(4\) and \(4+5-1=8\), unchanged from the original values \(4\) and \(4+3+1=8\). At \((1,0,1)\), the prediction is \(4-1=3\), whereas before the shift it was \(4+1=5\), so that prediction is not identifiable. (3) Every row has intercept value 1 and exactly one batch indicator equal to 1, so its product with \((-1,1,1)^T\) is \(-1+1=0\). Thus adding a multiple of this vector leaves fitted values unchanged. (4) For \(c=(0,1,1)^T\), \(c^Tv=1-1=0\), so the contrast is identifiable. For \(c=(0,0,1)^T\), \(c^Tv=-1\), so the condition coefficient alone is not identifiable.