Empty and Unseen Categories in Coded Data

Why an Empty Category Is Different

A categorical predictor divides observations into groups. When a group has observed outcomes, its data can inform the group’s fitted mean. But if a category appears in the coding scheme and has no observations in the training data, there is no outcome information from that group. The category label alone cannot tell us what its mean should be.

This distinction matters both when interpreting regression output and when using a fitted model to predict for new cases. A coefficient may be absent, not estimable, or set to a convenient value by a software convention. A prediction may also be produced by a software rule. None of these automatically supplies evidence about the unobserved group’s outcome mean.

The Group-Means Model

Consider a response \(Y\) and a categorical predictor with groups labelled \(1,\ldots,G\). For observation \(i\) in group \(g\), write the one-way group-means model as

Here \(\mu_g\) is the population mean response for group \(g\), and \(\varepsilon_{ig}\) is the error for that observation. For the standard least-squares model, assume the errors are independent, have mean zero, and have a common variance \(\sigma^2\). Normal errors are additionally assumed when using the usual exact small-sample regression inference. The key estimation argument below is about least squares and does not require normality.

Let \(n_g\) be the number of training observations in group \(g\). If \(n_g>0\), least squares estimates \(\mu_g\) by the sample mean of the outcomes in that group:

If \(n_g=0\), that expression cannot be calculated: its denominator is zero and there are no outcomes to average. More fundamentally, the data contain no information that distinguishes one possible value of \(\mu_g\) from another. This is not merely a small-sample estimate with high uncertainty. The group mean is not identified by these data.

Why the Empty Group Mean Is Not Identified

Least squares chooses model values to minimize the sum of squared residuals, where a residual is an observed outcome minus its fitted value. In the group-means model, the sum of squared residuals is

If a group is empty, its inner sum has no terms and contributes zero to \(SSE\), whatever value is assigned to \(\mu_g\). Thus changing that group’s proposed mean cannot change the criterion being minimized. The data therefore cannot select one value of its mean over another.

For a nonempty group, the minimizing value is its sample mean. To see why, let \(\bar{Y}_g\) be the mean of the group’s observations. Expanding each outcome’s deviation from \(\mu_g\) around \(\bar{Y}_g\) gives

The cross-product term is zero because \(\sum_i(Y_{ig}-\bar{Y}_g)=0\). The first term on the right does not depend on \(\mu_g\), and the second is nonnegative. When \(n_g>0\), it is minimized at \(\mu_g=\bar{Y}_g\). When \(n_g=0\), the group contributes no terms, so there is no such minimizing value singled out by its data.

What an Empty Indicator Column Does

The same issue can be seen using indicator variables. Suppose the categories are A, B, and C, and A is used as the reference category. The reference-coded model is

where \(I(B_i=B)\) equals 1 for an observation in B and 0 otherwise, and \(I(B_i=C)\) is defined similarly. If C has no training observations, \(I(B_i=C)=0\) in every training row. Its entire design-matrix column is zero, so changing \(\beta_C\) changes none of the training fitted values or residuals. The data cannot estimate \(\beta_C\).

This does not mean every coefficient must be unestimable. If A and B are both observed, \(\beta_0\) and \(\beta_B\) can still describe their fitted means. But the C mean, which this coding would express as \(\beta_0+\beta_C\), remains undetermined because \(\beta_C\) is undetermined.

Reference Categories and Coding Choices

An empty group can affect how coefficients appear under a particular coding scheme, but changing coding cannot create missing outcome information. In reference coding, an empty non-reference category produces an all-zero indicator column. If the reference category itself is empty, the intercept is not supported as that reference group’s mean. For example, with A as the empty reference and B and C observed, the fitted means involve \(\beta_0+\beta_B\) and \(\beta_0+\beta_C\), but there are no A observations to estimate \(\beta_0\) separately from those contrasts.

Cell-means coding makes the issue especially visible: each coefficient represents a category mean, and the column for an empty category contains only zeros. Sum coding and other contrast codings change which coefficient combinations correspond to group means, but they do not add observations to the empty group. The important question is not only whether software prints a coefficient; it is whether the data identify the quantity that coefficient is supposed to represent.

A fitting program may drop a zero column, flag a coefficient as not estimable, or impose a convention that produces a numerical output. Such behavior is about representing or handling the model computationally. Unless information has been added through an explicitly stated constraint or other model assumption, it should not be interpreted as evidence about the empty group’s mean.

Prediction for an Unseen Category

A prediction for a new observation in a category represented in training can use the fitted mean for that category (in this simple group-means model). If the new category was absent from training, its group mean was not estimated. A model therefore cannot make a data-based prediction for that group from the group-means model alone.

Prediction software may require every new case to use the same set of predictors and coding rules as the training data. When it encounters a category absent from training, it might return an error, assign the case to a specified default category, or use a separately supplied rule. These choices differ, so the analyst must check what the software does. In particular, mapping an unseen value to the reference category is a rule about the prediction procedure, not evidence that the unseen group has the reference group’s mean.

A Practical Check Before Fitting or Predicting

Before interpreting a categorical-predictor model, check the training data for empty categories and confirm how prediction cases are coded. A useful sequence is:

Common Mistake: Treating a Printed Number as an Estimate

A common mistake is to reason, “The output gives zero for the empty category’s coefficient, so its estimated group mean is zero.” This confuses a computational convention with information in the data. If the indicator column is all zero, every coefficient value produces the same training fitted values. A displayed zero may simply be a placeholder or constraint; it does not show that the empty group’s outcomes average zero.

The correction is to ask what quantity is estimable from the observed rows. Calculate or interpret fitted means for groups with observations, and state that the empty group mean is not identified by the training data. If a prediction for that group is needed, explain the additional data, constraint, or substantive assumption on which it relies.