Common Interpretation Errors With Empty Cells

Why Empty Cells Invite Overinterpretation

In a factorial design, an empty cell means that no observations were collected for one or more combinations of factor levels. The immediate consequence is missing information about those combinations. It is not, by itself, evidence that a treatment combination has no effect, that its mean equals another cell mean, or that a model term should be treated as zero.

The difficulty is that different quantities can behave differently in the same incomplete design. Some cell means and comparisons may still be identified by the observed data, while other comparisons cannot be determined without additional assumptions. A software output may omit a redundant or non-estimable term, and a numerical fitting routine may still print coefficients. Neither fact alone establishes that the omitted effect is absent or that every printed coefficient is uniquely identified.

A Model for Diagnosing Interpretation Errors

Use a cell-means model for a two-factor design. Let \(\mu_{ij}\) be the mean response for the combination of level \(i\) of factor A and level \(j\) of factor B. For an observation \(k\) in an observed cell, write

The model assumes independent errors with mean zero and a common variance; the normality assumption supports the usual small-sample ANOVA inference. It also assumes the cell means describe the expected responses for the combinations being modeled. An empty cell contributes no observations or design rows. These assumptions do not fill in the missing cell mean.

For the examples below, suppose the hypothetical observations come from independent experimental units, the design and recorded responses have been checked, and residual checks show no severe departures from the common-variance and normal-error assumptions. The values are illustrative, not a real study. There are three observed combinations, each with four responses:

The observed cell means are computed directly: for example, the A0/B0 mean is \((9+10+10+11)/4=10\) points. Let the parameter vector be \(\boldsymbol{\mu}=(\mu_{00},\mu_{01},\mu_{10},\mu_{11})^\mathsf{T}\). In this cell-means representation, design rows for the observed combinations select the first, second, or third parameter. There is no row selecting \(\mu_{11}\).

Why a Dropped Term Does Not Mean “No Effect”

A term can be omitted from a software table because its parameters are aliased with other parameters, because the relevant columns are redundant, or because the software has chosen a particular reduced parameterization. When a combination has no observations, some columns may contain no information about their associated parameters. A term that cannot be separately estimated is not the same thing as a term whose effect has been estimated to be zero.

The distinction matters in reporting. “The interaction was not estimated because the A1/B1 cell was empty” describes a limitation of the design. “There was no interaction” is a substantive claim that would require an estimable contrast and appropriate evidence—or a clearly stated assumption. Likewise, a missing test or p-value is not evidence that the null hypothesis is true.

Why a Printed Generalized-Inverse Estimate May Be Arbitrary

When the design matrix is rank deficient, there can be multiple parameter vectors that produce identical fitted values for the observed data. A generalized inverse is one computational method that can select a solution. That solution can be useful for calculating fitted values or estimable functions, but its individual coordinates are not automatically identified parameters. Their values may depend on the particular coding or generalized-inverse convention.

A Short Justification of the Estimability Check

The row-space criterion follows from asking whether observationally indistinguishable parameter values can disagree about the quantity of interest. In the linear model, the expected response vector is \(X\boldsymbol{\beta}\). If a vector \(\mathbf{h}\) satisfies \(X\mathbf{h}=\mathbf{0}\), then \(\boldsymbol{\beta}\) and \(\boldsymbol{\beta}+\mathbf{h}\) have the same expected responses on every observed design row:

The proposed function changes between these parameter vectors by \(\mathbf{c}^\mathsf{T}\mathbf{h}\). It is therefore identifiable from the design only if this change is zero for every \(\mathbf{h}\) in the null space of \(X\). In symbols, the condition is

A vector is orthogonal to the null space of \(X\) exactly when it belongs to the row space of \(X\). This proves the criterion. In the cell-means example, the design rows select only \(\mu_{00}\), \(\mu_{01}\), and \(\mu_{10}\). Any estimable linear function can therefore involve those observed means, but it cannot depend on \(\mu_{11}\) unless the coefficient of \(\mu_{11}\) is zero.

Adjusted Means Must Match What the Design Identifies

An adjusted mean is a target defined by a particular combination of model means. To interpret one, specify the combination and check whether it is estimable. An appealing calculation or a software label does not establish that the required cell means are supported by the data. In particular, averaging over factor levels can require a missing-cell mean.

Practical Checks Before Reporting

When interpreting software output for a sparse factorial design, separate three questions: what was observed, what quantity is being reported, and whether that quantity is identified. A coefficient table may use constraints that choose one representative from many equivalent parameter vectors. Fitted values for observed combinations can still be uniquely determined even when some individual parameters are not.

If a scientific question requires a non-estimable comparison, the limitation cannot be repaired by choosing a more convenient generalized inverse. It requires additional information or an explicit model assumption that connects the missing combination to observed ones. Such an assumption may be scientifically defensible in some settings, but it must be stated and distinguished from what the observed design alone establishes.

Common Interpretation Errors

• Wrong: “The interaction term was dropped, so the interaction is zero.” Correction: Determine whether the contrast is estimable. A term omitted because of aliasing has not necessarily been tested or shown to be zero.
• Wrong: “The software printed a coefficient for the empty cell, so that is its estimated mean.” Correction: Check whether alternative parameter vectors give the same observed fitted values but different coefficients. A generalized-inverse convention can select a value that the data do not uniquely identify.
• Wrong: “The adjusted means are available because the software produced a table.” Correction: Write out each reported weighted combination and check whether it depends on non-estimable cell means.
• Wrong: “No p-value means there is no evidence of an effect, so the null is true.” Correction: A test that cannot be formed does not supply evidence for either a zero effect or a nonzero effect.