Why Use a Redescending Residual Weight?
Least-squares regression gives large residuals substantial influence because it minimizes a sum of squared residuals. Huber regression reduces the influence of large residuals, but its residual score remains bounded rather than dropping to zero. Tukey’s bisquare regression takes a different approach: it gradually downweights residuals as they grow, then gives residuals beyond a chosen cutoff zero weight in the current update.
This can be useful when a small number of responses are severely contaminated and the main aim is to fit the pattern followed by the bulk of the observations. The weights depend on residual size relative to a residual scale, so the cutoff is not simply a threshold in the response variable’s original units.
For \(|u_i|<c\), the weight decreases as \(|u_i|\) increases. Small residuals have weights close to one; moderate residuals have weights between zero and one; and residuals at or beyond the cutoff have zero weight. These are weights for a fitting procedure, not probabilities that observations are valid or invalid.
Loss, Score, and the Weighted Least-Squares Update
A loss function makes precise what the fit is trying to minimize. In the standardized residual \(u\), Tukey’s bisquare loss is
The loss increases with residual size up to the cutoff and then stays constant. Thus, once a residual is beyond the cutoff, making it even larger does not increase its contribution to this loss. The estimate seeks coefficients that minimize the sum of these losses, using a chosen scale \(s\).
The score is the derivative of the loss with respect to \(u\). Within the cutoff, differentiating gives \(\psi(u)=u[1-(u/c)^2]^2\); beyond the cutoff, the loss is constant and the score is zero. The weight is the score divided by the standardized residual when that residual is nonzero. At zero, its continuous limiting value is one.
This identity explains the connection to iteratively reweighted least squares (IRLS). At each iteration, calculate residuals from the current coefficients, standardize them using the current scale, calculate the bisquare weights, and solve a weighted least-squares problem. With one predictor, that update minimizes \(\sum_i w_i(y_i-a-bx_i)^2\) over the intercept \(a\) and slope \(b\). The weights and coefficients are then recalculated until the fitting procedure meets its convergence rule.
Worked Example: Calculating Bisquare Weights
Worked Example: Calculating Bisquare Weights
Suppose a current fit has residual scale \(s=2\) response units. Four hypothetical observations have residuals \(0\), \(2\), \(4\), and \(9.37\) response units. Use \(c=4.685\) to calculate their standardized residuals and weights.
First standardize each residual by dividing by the scale: the values of \(u_i\) are \(0\), \(1\), \(2\), and \(4.685\). For \(u=0\), the weight is one. For \(u=1\),
For \(u=2\),
The last residual is exactly at the cutoff, so \(w(4.685)=[1-1]^2=0\). The corresponding weights are approximately \(1.000\), \(0.911\), \(0.669\), and \(0\). The residual of 4 response units still contributes to the weighted update, but less than a residual of 2 response units; the residual at the cutoff contributes no weight to that update.
Worked Example: One Update With a Contaminated Response
Worked Example: One Update With a Contaminated Response
Consider the following hypothetical data. The response at \(x=0\) is deliberately made much larger than the values that follow an approximately linear pattern. For illustration, begin with the candidate line \(\hat{y}=10+2x\). This is a starting fit for the example, not a claim that every bisquare analysis should use this initialization.
| \(x_i\) | \(y_i\) | Initial fitted value | Initial residual \(r_i\) |
|---|---|---|---|
| -2 | 5.5 | 6 | -0.5 |
| -1 | 8 | 8 | 0 |
| 0 | 30.5 | 10 | 20.5 |
| 1 | 12 | 12 | 0 |
| 2 | 14.5 | 14 | 0.5 |
The residuals in order are \(-0.5,0,20.5,0,0.5\), with median \(0\). Their absolute deviations from that median are \(0.5,0,20.5,0,0.5\), whose median is \(0.5\). Use the MAD-based scale \(s=1.4826(0.5)=0.7413\). This scale is positive, and the \(x\)-values span both sides of the response outlier’s predictor value. The example’s purpose is the response-weight update, not a general diagnostic assessment of the data.
The standardized residuals for the two endpoint observations are approximately \(-0.6745\) and \(0.6745\); the two observations at \(x=-1\) and \(x=1\) have standardized residual zero. The contaminated response has standardized residual \(20.5/0.7413\approx27.66\), well beyond \(c=4.685\). With the bisquare formula, the four ordinary observations have weights approximately \(0.9590,1,1,0.9590\), while the contaminated response has weight zero. For example, the endpoint weight is
Now perform one weighted least-squares update. In the order shown in the table, use rounded weights \(0.9590,1,0,1,0.9590\). The weighted sums are
For a weighted regression with an intercept, the slope is the weighted centered cross-product divided by the weighted centered sum of squares. Here the weighted predictor mean is zero, so the calculation simplifies to
The intercept is the weighted response mean minus the slope times the weighted predictor mean. Since that predictor mean is zero,
This update is approximately \(\hat{y}=10.00+2.198x\). It is closer to the pattern in the four non-contaminated responses than a least-squares fit would be expected to be with the extreme central response included at full weight. It is only one update: a completed IRLS fit recalculates residuals and weights and repeats the weighted regression step. The final coefficients need not equal this one-step result.
Worked Example: Why the Scale Changes the Cutoff in Response Units
Worked Example: Why the Scale Changes the Cutoff in Response Units
Suppose an observation has a residual of \(10\) response units. If the scale is \(s=2\), its standardized residual is \(u=10/2=5\), which exceeds \(4.685\), so its bisquare weight is zero. If the scale is \(s=4\), its standardized residual is \(u=10/4=2.5\), which is inside the cutoff. Its weight is then
Thus the same residual can receive zero weight with one scale and a positive weight with another. The scale is part of the fitting rule, not a cosmetic way to express residuals. A scale estimate that is itself distorted by severe contamination can change which residuals cross the cutoff.
What the Update Does—and What It Does Not Establish
Tukey’s bisquare procedure is designed to limit the effect of sufficiently large residuals in the fitting objective. A zero weight means that, at the current iteration, that observation contributes nothing to the weighted least-squares coefficient update. It does not mean that the observation is certainly erroneous, irrelevant to the scientific question, or suitable for deletion from the data.
The method also has an important computational feature: its loss is not globally convex. As a result, an iterative algorithm can depend on its starting values and can converge to a local solution. In practice, the starting fit, scale estimate, cutoff, and convergence behavior should be reported or otherwise checked. The example above demonstrated a single update rather than asserting that the update had converged.
The usual regression interpretation still requires care. The method is aimed at fitting a central linear pattern under a working model in which that pattern is appropriate for most observations and severe contamination is mainly in responses. It does not establish that the linear relationship is suitable, and a small residual does not guarantee that an observation is harmless. Residual-based weights are calculated after fitting and can be affected by the fit itself.
Common Mistakes
- Using the raw residual in place of the standardized residual. The cutoff applies to \(u_i=r_i/s\), not directly to \(r_i\). Divide by the scale before comparing with \(c\).
- Giving a residual beyond the cutoff a small positive weight. Tukey’s bisquare assigns it weight exactly zero under the stated rule.
- Calling zero weight an automatic deletion rule. It means zero contribution to the current weighted update, not a verdict about data quality or scientific relevance.
- Reporting one weighted update as the converged fit. IRLS generally recalculates residuals and weights and repeats the update; convergence must be checked.
Check Your Understanding
Try these questions without looking back.
- With \(c=4.685\), what weight does Tukey’s bisquare assign to a standardized residual exactly at the cutoff? What about one beyond it?
- A residual is \(6\) response units and the scale is \(2\). Calculate its standardized residual and determine whether its bisquare weight is zero.
- Explain why the same raw residual can have different bisquare weights under two different scale estimates.
- In IRLS, what calculation is repeated after a weighted least-squares update?
- Why does a zero bisquare weight not, by itself, justify deleting an observation from a data set?