Order Does Not Automatically Mean Distance
Many predictors have a natural order: a treatment dose may be low, medium, or high; a rating may range from poor to excellent; and an educational level may progress through successive stages. Because the categories are ordered, it can be tempting to replace their labels with numbers and fit a straight line. That choice is not just a convenient way to enter data. It makes a specific assumption about how the response changes from one category to the next.
Treating categories as a factor uses indicator variables to estimate a separate mean response for each category. Treating them as numeric scores instead gives the model one intercept and one slope, requiring the fitted means to change linearly with the assigned scores. The order of categories alone does not establish that this equal-step pattern is appropriate.
The Two Models
Suppose an outcome \(Y\) is measured for observations in \(K\) ordered categories. Label category \(j\) by \(j=1,\ldots,K\), and let \(\mu_j\) denote its mean response. In a factor model, each category has its own mean:
Here \(i\) indexes observations within category \(j\). This is the cell-means form of a categorical-predictor model. Equivalently, it can be represented using an intercept and \(K-1\) indicators. The important feature is that the \(K\) means are not required to lie on a straight line. Their differences can be unequal, and the means need not increase at every step.
In a numeric-score model, assign category \(j\) a score \(s_j\) and use a linear regression:
This model has only two mean parameters, \(\beta_0\) and \(\beta_1\). It predicts the category-\(j\) mean to be \(\beta_0+\beta_1s_j\). Therefore, once the scores are chosen, the fitted means must follow a straight line when plotted against those scores. If consecutive categories are given equally spaced scores, the fitted change from one category to the next is the same at every step.
For both models, the usual linear-model framework assumes that errors have mean zero and a common variance, and that observations are independent. Normal errors are an additional assumption used for exact small-sample inference. These assumptions concern the response around the model’s mean pattern; they do not make a numeric score appropriate. In the examples below, the calculations focus on fitted means, so normality is not needed. Hypothetical independent observations are specified, while distributional assumptions are not treated as established by the examples.
Worked Example: Separate Means for Three Ordered Levels
Suppose a hypothetical training program records a quiz score (points) for three independent participants at each of three ordered practice levels. The data are:
| Practice level | Quiz scores (points) |
|---|---|
| Low | 9, 10, 11 |
| Medium | 13, 14, 15 |
| High | 23, 25, 27 |
The factor model estimates each category mean by its sample mean:
Thus its fitted means are 10, 14, and 25 points. The successive observed mean differences are \(14-10=4\) points and \(25-14=11\) points. The factor model can represent both differences without requiring them to be equal.
If the categories are instead scored 1, 2, and 3, the numeric-score regression gives each category equal weight here because each contains three observations. The score mean is \(\bar{s}=2\), and the mean quiz score is \(\bar{y}=49/3=16.333\) points. The slope is
The intercept is \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{s}=16.333-(7.5)(2)=1.333\). The numeric-score fitted means are therefore \(1.333+7.5(1)=8.833\), \(1.333+7.5(2)=16.333\), and \(1.333+7.5(3)=23.833\) points. They differ from the factor means because the straight-line model cannot reproduce the unequal observed steps of 4 and 11 points.
The setup stipulates distinct independent participants, which supports the independence assumption. The residuals within each group are \(-1,0,1\), so the example has the same within-group spread at each level. This constructed pattern illustrates the mean calculations; it does not establish normal errors or justify an inferential conclusion.
Why the Factor Fit Is the Group Mean
The factor model’s separate fitted means are not arbitrary. For category \(j\), least squares chooses \(\mu_j\) to minimize the sum of squared residuals in that category:
Differentiate with respect to \(\mu_j\), set the derivative equal to zero, and solve:
The second derivative is \(2n_j>0\), so this value minimizes the squared-error sum. Because each category has its own parameter, minimizing the sum over all categories amounts to minimizing each category’s contribution separately. That is why the factor model’s fitted mean in each observed category is its sample mean.
The numeric-score model instead uses one intercept and one slope to fit all categories together. Its fitted means must satisfy \(\hat{\mu}_j=\hat{\beta}_0+\hat{\beta}_1s_j\). For two adjacent categories, their fitted difference is
If the scores are equally spaced, \(s_{j+1}-s_j\) is constant, so every adjacent fitted-mean difference is the same. The common step can be positive, negative, or zero depending on the slope. The model still imposes a straight-line pattern even if the observed category means do not follow it.
Worked Example: What an Equal-Step Score Assumes
Imagine a hypothetical course evaluation with four ordered rating categories, coded 0, 1, 2, and 3. Each category contains two independent respondents, and their satisfaction scores (out of 100) are:
| Rating category | Scores | Category mean |
|---|---|---|
| 0 | 70, 74 | 72 |
| 1 | 74, 78 | 76 |
| 2 | 75, 79 | 77 |
| 3 | 87, 91 | 89 |
The factor fit is the four means: 72, 76, 77, and 89 points. Their adjacent changes are 4, 1, and 12 points, so the means are not equally spaced.
For the numeric-score fit, the score mean is \(\bar{s}=1.5\) and the mean satisfaction score is \(\bar{y}=78.5\). Since each category has the same number of respondents, the regression calculations can use the four category means:
The intercept is \(\hat{\beta}_0=78.5-(5.2)(1.5)=70.7\). The numeric-score model therefore predicts 70.7, 75.9, 81.1, and 86.3 points at scores 0, 1, 2, and 3. It imposes an increase of exactly 5.2 predicted points for each one-point increase in the rating score. That is a modeling assumption, not a conclusion guaranteed by the categories’ order.
The stipulated independent respondents address the independence condition. Each category’s two scores deviate by \(-2\) and \(2\) points from its mean, giving equal within-category spread in this example. As before, the small constructed data do not establish normality; no significance test or interval is being calculated.
Scores Depend on the Assigned Distances
An ordinal scale tells us which category comes before another, but may not tell us how far apart the categories are. Numeric coding supplies that missing distance information. If the same three ordered categories are assigned scores 1, 2, and 3, a score model assumes the gap from the first to the second is the same as the gap from the second to the third. If the categories are instead assigned scores 1, 2, and 4, the second gap is twice as large as the first in the model.
The factor model does not use numerical distances between category labels. Its fitted means depend on which observations belong to each category, not on whether the labels are written as “low, medium, high” or “1, 2, 4.” A numeric-score fit can change when the assigned scores change, because those scores determine the line’s horizontal spacing.
Worked Example: Changing Scores Changes the Numeric Fit
Suppose a hypothetical ordered predictor has three categories with response means of 5, 8, and 12 units. There are two observations in each category, with values one unit below and one unit above its mean. Thus, the factor model fits means of 5, 8, and 12 units, regardless of how the category labels are written.
First assign scores 1, 2, and 3. The score mean is 2 and the response mean is \(25/3\). The slope and intercept are
The fitted means at scores 1, 2, and 3 are \(4.833\), \(8.333\), and \(11.833\) units. Now assign the categories scores 1, 2, and 4, without changing any response. The score mean is \(7/3\), and the cross-product and squared-deviation totals are \(32/3\) and \(14/3\), respectively. Therefore,
The fitted means at scores 1, 2, and 4 are \(3+16/7=5.286\), \(3+32/7=7.571\), and \(3+64/7=12.143\) units. The factor means remain 5, 8, and 12. The score-model predictions changed because the assigned distances changed, while the factor model made no use of those distances.
The hypothetical observations are stipulated to be independent, and their equal within-category deviations give equal group spreads. The fitted-value comparison does not require a normality assumption. In an applied analysis, these model assumptions and the substantive reason for the chosen score spacing would need consideration before drawing conclusions.
Choosing Between the Representations
A factor is a suitable starting point when the categories are genuinely groups and there is no strong reason to treat their gaps as equal. It lets the data show separate category means without forcing a particular trend shape. A numeric score can be useful when the score units have a defensible quantitative meaning and a linear change per score unit is an appropriate model. For example, a dose measured in fixed physical units may support a quantitative interpretation more readily than a rating whose labels merely have an order.
Using a factor uses more mean parameters when there are several categories. A numeric-score model is more restrictive: it summarizes the category pattern with one slope. That restriction can be useful when justified, but a simpler model is not automatically the more appropriate one. In particular, a numeric slope is measured in outcome units per score unit, so its meaning depends on the scores chosen.
Neither representation changes the raw observations or makes an association causal. Both describe a model for the response across categories. The choice is about which pattern of conditional means the model permits and which assumptions are scientifically reasonable.
- Mistake: “The categories have an order, so I can treat them as 1, 2, and 3 without losing anything.” Correction: Order specifies ranking, not equal distances. Scores 1, 2, and 3 impose equal gaps and a straight-line mean pattern.
- Mistake: “The factor model assumes the response is constant across categories.” Correction: It estimates a separate mean for each category; those means may differ freely.
- Mistake: “A numeric-score coefficient is just the average difference between any two neighboring categories.” Correction: It is the fitted change per one score unit. Under equally spaced scores, it imposes the same fitted change at each adjacent step, even if observed mean differences vary.
Check Your Understanding
Answer each question by distinguishing category order from the assumptions added by numeric scoring.
- A factor model has four ordered categories. What does it estimate for the response mean in each category?
- Categories are scored 1, 2, and 3 in a linear regression. What restriction does the model place on adjacent fitted-mean differences?
- Why can changing scores from 1, 2, 3 to 1, 2, 5 change a numeric-score fit but not a factor fit?
- Suppose the observed category means are 10, 13, and 21. What are the two adjacent observed differences, and do they have to equal the score model’s adjacent fitted differences?
- Give one reason to prefer a factor representation and one reason a numeric-score representation might be defensible.