Repeated Predictor Values and Fitted Responses

Introduction

A data set can contain several observations with the same predictor value. For example, multiple hypothetical clinics might have the same number of treatment rooms but different average waiting times. A fitted straight line assigns the same predicted response to all observations with that shared predictor value. The group’s observed mean response is useful for understanding that prediction, but it is not necessarily equal to it.

The key is to distinguish the mean response among observations at a predictor value from the fitted response on the line. Least squares accounts for each individual observation; equivalently, it fits the group means while giving each mean a weight equal to its group size.

Groups, Means, and Fitted Responses

Definition: Suppose the observed data are divided into groups according to their predictor value. For group \(j\), let \(x_j\) be its shared predictor value, \(n_j\) its number of observations, and \(\bar{y}_j\) the mean of the responses in that group. The fitted response at \(x_j\) is \(\hat{y}_j=b_0+b_1x_j\), where \(b_0\) and \(b_1\) are the least-squares intercept and slope. Every observation in group \(j\) has this same fitted response.

For a simple linear regression fit, least squares chooses \(b_0\) and \(b_1\) to minimize the sum of squared differences between observed responses and fitted responses:

\sum_{j=1}^{g}\sum_{i=1}^{n_j}\left(y_{ij}-(b_0+b_1x_j)\right)^2

Here \(g\) is the number of distinct predictor values, and \(y_{ij}\) is the response for observation \(i\) in group \(j\). The predictor must vary across the data to estimate a slope. No distributional assumption is required simply to calculate this descriptive least-squares line.

Why Group Means Matter to Least Squares

Within a group, the predictor is constant, so the line assigns every observation the same fitted value. The group mean summarizes where the group’s responses are centered. The following decomposition shows exactly how the group mean enters the least-squares calculation.

Result: For any proposed line, the sum of squared differences for group \(j\) separates into the group’s squared deviations from its mean and a term involving the distance from that mean to the fitted response:

\sum_{i=1}^{n_j}\left(y_{ij}-(b_0+b_1x_j)\right)^2 = \sum_{i=1}^{n_j}(y_{ij}-\bar{y}_j)^2 +n_j\left(\bar{y}_j-(b_0+b_1x_j)\right)^2

To verify the result, write each difference as \(y_{ij}-(b_0+b_1x_j)=(y_{ij}-\bar{y}_j)+(\bar{y}_j-(b_0+b_1x_j))\), square it, and sum over the group. The cross-product term is zero because the deviations from a group mean sum to zero:

\sum_{i=1}^{n_j}(y_{ij}-\bar{y}_j)=0

The first term in the decomposition does not depend on the line. Therefore, minimizing the total squared error over all observations gives the same line as minimizing

\sum_{j=1}^{g}n_j\left(\bar{y}_j-(b_0+b_1x_j)\right)^2

This is a weighted least-squares criterion on the group means, with group size \(n_j\) as the weight. A group with more observations contributes more to the original sum of squared errors. Fitting the group means without weights would generally give a different line.

Worked Example: Fitting a Line With Three Repeated Predictor Values

Consider a hypothetical data set in which the predictor \(x\) is the number of practice sessions and the response \(y\) is a quiz score in points. The observed pairs are \((1,2),(1,4),(2,3),(2,5),(2,7),(4,6),(4,8)\). Find the mean response in each group and the least-squares line.

Group means. At \(x=1\), the mean score is \((2+4)/2=3\). At \(x=2\), it is \((3+5+7)/3=5\). At \(x=4\), it is \((6+8)/2=7\). The group sizes are 2, 3, and 2, respectively.

For the seven observations, the needed totals are \(\sum x=16\), \(\sum y=35\), \(\sum x^2=46\), and \(\sum xy=92\). Thus \(\bar{x}=16/7\) practice sessions and \(\bar{y}=35/7=5\) points. The least-squares slope and intercept are calculated as follows:

b_1=\frac{\sum xy-(\sum x)(\sum y)/n}{\sum x^2-(\sum x)^2/n} =\frac{92-(16)(35)/7}{46-16^2/7} =\frac{12}{66/7} =\frac{14}{11}\approx1.273

b_0=\bar{y}-b_1\bar{x} =5-\left(\frac{14}{11}\right)\left(\frac{16}{7}\right) =\frac{23}{11}\approx2.091

The fitted line is \(\hat{y}=2.091+1.273x\), with scores measured in points and \(x\) measured in practice sessions. Its fitted responses at the three observed predictor values are approximately 3.364, 4.636, and 7.182 points. These are close to the group means of 3, 5, and 7 points, but none is required to equal its group mean.

For instance, the line’s fitted response at \(x=2\) is \(23/11+(14/11)(2)=51/11\approx4.636\) points, whereas the observed mean there is 5 points. The line balances squared errors across all seven observations, rather than being required to pass through each group mean.

When Does the Fitted Response Equal the Group Mean?

A fitted response equals a group mean when the least-squares line happens to pass through that group’s mean point \((x_j,\bar{y}_j)\). This can happen, but repeated predictor values alone do not guarantee it. If all the group mean points lie on one straight line, the least-squares line does pass through them: a line that matches every group mean makes the weighted squared-deviation criterion zero, its smallest possible value.

Worked Example: Group Means That Lie on a Straight Line

Suppose a hypothetical data set has two observations at each of \(x=10,20,30\) practice minutes. The response is quiz score in points. The response pairs within the groups are \((4,6)\), \((7,9)\), and \((12,14)\), respectively. Find the group means and the fitted line.

The group means are 5, 8, and 13 points. They lie on the line \(\bar{y}=2+0.4x\), since \(2+0.4(10)=6\) — which is not 5. This shows that these particular means do not lie on that proposed line. Calculate the least-squares line instead.

Because each group has the same size, fitting all six observations gives the same slope as fitting the three group means. Their averages are \(\bar{x}=20\) and \(\bar{y}=(5+8+13)/3=26/3\). Using the group means, the sum of products about their averages is \(80\), and the sum of squared predictor deviations is \(200\). Thus:

b_1=\frac{80}{200}=0.4,\qquad b_0=\frac{26}{3}-(0.4)(20)=\frac{2}{3}

The fitted line is \(\hat{y}=2/3+0.4x\). Its predictions at \(x=10,20,30\) are \(14/3\approx4.667\), \(26/3\approx8.667\), and \(38/3\approx12.667\) points. The group means are 5, 8, and 13 points, so the fitted values do not equal them. In particular, checking the first point confirms \(2/3+0.4(10)=14/3\), not 5. Equal group sizes make the means equally weighted, but do not force a line through them.

Group Size Changes the Weight

The weighted criterion also explains why group sizes cannot be ignored. A group of eight observations affects the original least-squares objective eight times as strongly as a group of one observation, assuming the same squared distance between its mean and the fitted line. Consequently, the regression line for all observations need not match the line obtained by giving each distinct predictor value equal weight.

Worked Example: Why an Unequal Group Size Matters

In a hypothetical data set, the response is a measurement in units and the predictor takes values 0, 1, and 2. The group at \(x=0\) contains one response, 0; the group at \(x=1\) contains one response, 0; and the group at \(x=2\) contains eight responses, all equal to 10. The group means are therefore 0, 0, and 10, with sizes 1, 1, and 8.

Across the ten individual observations, \(\sum x=17\), \(\sum y=80\), \(\sum x^2=33\), and \(\sum xy=160\). Hence \(\bar{x}=1.7\) and \(\bar{y}=8\), and:

b_1= \frac{160-(17)(80)/10}{33-17^2/10} =\frac{24}{4.1} =\frac{240}{41} \approx5.854

b_0=8-\left(\frac{240}{41}\right)(1.7) =-\frac{80}{41} \approx-1.951

The fitted line is \(\hat{y}=-1.951+5.854x\). For comparison, if the three group means were treated equally, their unweighted least-squares line would have slope 5 and intercept \(-5/3\). That is a different objective: it gives the group with eight observations the same total weight as either one-observation group. The original individual-level regression correctly reflects all ten observations.

Common Mistake and Practical Use

Mistake: “Several observations share \(x\), so I can replace each group by its mean and fit an ordinary, unweighted line to the means.” This reasoning discards information about how many observations contributed to each mean. In general, it changes the least-squares objective and can change both the slope and intercept.

Correction: If replacing observations by group means to reproduce the original line, retain the group sizes as weights. The decomposition above proves that weighting each squared difference from a group mean by \(n_j\) gives the same coefficient estimates as using every observation. Within-group variation still contributes to the total squared error, but it does not affect which line minimizes that error because it is constant across candidate lines.

Grouping is useful for organizing calculations and describing the data. It makes clear which observations share the same fitted response and how the observed mean at each predictor value compares with that response. But a fitted response is a value on the overall line, not a promise to match the average response within every group.

Key takeaway: Repeated predictor values share one fitted response. Least-squares fitting can be expressed using group means, but each group mean must be weighted by its number of observations. A fitted response equals a group mean only when the fitted line passes through that group’s mean point.

Check Your Understanding

Use the ideas in this tutorial to answer the following questions.

At a predictor value of 5, three responses are 8, 10, and 12. What is the group mean response, and how many fitted responses does a straight-line model assign to these observations?
Why does least squares give a group of six observations more influence than a group of two observations when their mean points are the same distance from a candidate line?
Does a fitted response have to equal the mean response of observations sharing that predictor value? Explain.
What information about a group must be retained when fitting a line to group means so that the result matches regression on the original observations?
Two groups have means that lie on a single straight line. What is the minimum possible value of the weighted squared-deviation criterion for those group means, and why?