Standard Errors and Tests After Rescaling

Why Rescaling Changes Uncertainty but Not the Evidence

A regression coefficient is reported in particular units, so its standard error and confidence interval must also be expressed in those units. If a predictor is changed from hours to minutes, for example, the numerical slope becomes smaller because it describes the change in outcome for one minute rather than one hour. Its standard error changes by the same factor. The ratio of the estimated slope to its standard error, however, does not change in magnitude. Thus, a unit conversion changes how the estimate and its uncertainty are displayed, but not the corresponding two-sided test of whether the slope is zero.

To make that statement precise, we need to track the coefficient, its standard error, the null hypothesis, and the test statistic together. A conversion must not be applied to the coefficient while leaving its uncertainty or the null hypothesis in the old units.

The Regression Model and the Rescaling Rule

Consider a multiple linear regression with one outcome and \(q\) predictors:

Y_i=\beta_0+\beta_1X_{i1}+\cdots+\beta_qX_{iq}+\varepsilon_i,\qquad i=1,\ldots,n.

For the usual exact \(t\)-based inference, assume the conditional mean is correctly represented by this linear model, the errors are independent with mean zero and common variance \(\sigma^2\), and the errors are normally distributed conditional on the predictors. The predictor columns must have full rank (there can be no exact linear dependence among them and the intercept), and \(n>q+1\), so residual degrees of freedom are available. Normality gives exact small-sample \(t\) inference under these assumptions. If the assumptions are not credible, changing measurement units does not repair the problem.

Definition: Suppose the outcome is converted by \(Y_i^*=aY_i\), and one predictor is converted by \(X_{ij}^*=cX_{ij}\), where \(a\) and \(c\) are nonzero constants. If other predictors are unchanged, the corresponding slope is \(\beta_j^*=(a/c)\beta_j\). The standard error of its estimate changes by the magnitude of the same factor: \(SE(\hat{\beta}_j^*)=|a/c|SE(\hat{\beta}_j)\).

Here, a positive conversion factor \(c\) describes ordinary changes of units, such as hours to minutes. A positive \(a\) can likewise convert the outcome from one unit to another. An intercept can also change when the outcome is rescaled, but the result above concerns the slope associated with the converted predictor.

Why the Standard Error Gets the Same Scale Factor

The rule follows from how the fitted model and its uncertainty transform. First suppose only predictor \(X_j\) is rescaled, with \(X_j^*=cX_j\), while the outcome is unchanged. To give the same fitted values, its coefficient must be \(\beta_j^*=\beta_j/c\), since

\beta_j^*X_j^*=\frac{\beta_j}{c}(cX_j)=\beta_jX_j.

The other coefficients need not change. The fitted values and residuals are therefore unchanged, so the residual variance estimate is unchanged as well. In the usual least-squares covariance formula, rescaling a design-matrix column by \(c\) scales the variance of its corresponding coefficient by \(1/c^2\). Taking the square root gives \(SE(\hat{\beta}_j^*)=SE(\hat{\beta}_j)/|c|\).

Now suppose instead that the outcome is rescaled by \(a\), with predictors unchanged. Every fitted value and residual is multiplied by \(a\); the residual variance estimate is multiplied by \(a^2\). The coefficient estimates are multiplied by \(a\), and their standard errors by \(|a|\). Doing both conversions gives the combined factor \(|a/c|\) for the standard error. This also shows why rescaling cannot improve or worsen whether the model assumptions hold: the same residual pattern is simply expressed in different units.

Key result: Let \(k=a/c\). Then \(\hat{\beta}_j^*=k\hat{\beta}_j\) and \(SE(\hat{\beta}_j^*)=|k|SE(\hat{\beta}_j)\). A confidence interval transforms by multiplying both endpoints by \(k\) (and reversing their order if \(k<0\)). A two-sided test of \(H_0:\beta_j=0\) has the same \(p\)-value after rescaling.

Confidence Intervals and Tests After Conversion

A \(100(1-\alpha)\%\) confidence interval for a coefficient is its estimate plus or minus a critical \(t\) value times its standard error:

\hat{\beta}_j\ \pm\ t_{1-\alpha/2,\,df}\,SE(\hat{\beta}_j),\qquad df=n-q-1.

Under the rescaling rule, both the center and the margin of error change by the same amount in the converted units. When \(k>0\), the endpoints are simply multiplied by \(k\); when \(k<0\), the endpoints are multiplied by \(k\) and their order is reversed to keep the lower endpoint first. The confidence level and the underlying set of plausible coefficient values have not changed.

For a test of \(H_0:\beta_j=0\), the test statistic is

t=\frac{\hat{\beta}_j}{SE(\hat{\beta}_j)}.

After rescaling, \(t^*=(k/|k|)t\). For the usual positive unit conversions, \(k>0\), so the signed statistic itself is unchanged. If \(k<0\), its sign reverses but its magnitude remains the same. In either case, a two-sided test has the same \(p\)-value, since that value is determined by \(|t|\) and the same degrees of freedom.

A nonzero null value must be converted too. To test \(H_0:\beta_j=b_0\), the corresponding hypothesis in the new units is \(H_0:\beta_j^*=kb_0\). With this matching null value, the test statistic is \((\hat{\beta}_j^*-kb_0)/SE(\hat{\beta}_j^*)\), which equals the original statistic when \(k>0\) and has the opposite sign when \(k<0\). The two-sided \(p\)-value remains unchanged.

Worked Example: Converting a Predictor from Hours to Minutes

Suppose a hypothetical simple regression relates a training measure, \(Y\), in points to training time in hours. There are \(n=20\) observations, so the residual degrees of freedom are \(20-1-1=18\). Assume the observations are independent, the predictor has variation, and the residual checks for this hypothetical data set are consistent with a linear mean, roughly constant variance, and normal errors. The estimated slope is \(2.4\) points per hour, with \(SE=0.8\) points per hour. What happens if training time is expressed in minutes?

Let \(X^*=60X\), so one hour is 60 minutes. The new slope and standard error are

\hat{\beta}_1^*=\frac{2.4}{60}=0.0400\text{ points per minute},\qquad SE(\hat{\beta}_1^*)=\frac{0.8}{60}=0.01333\text{ points per minute}.

For \(H_0:\beta_1=0\), the original statistic is \(t=2.4/0.8=3.00\). In the converted units it is \(t^*=0.0400/0.01333=3.00\). With \(df=18\), the two-sided \(p\)-value is approximately \(0.007685\) in either scale. The data provide evidence against a zero slope at, for example, the \(0.05\) significance level; the result does not depend on whether time is recorded in hours or minutes.

For a 95% interval, the critical value is approximately \(t_{0.975,18}=2.1009\). In points per hour, the interval is

2.4\pm 2.1009(0.8)=2.4\pm1.68072=[0.7193,\ 4.0807]\text{ points per hour}.

Dividing each endpoint by 60 gives \([0.01199,\ 0.06801]\) points per minute. Both intervals describe the same range of slope values in different units.

Worked Example: Converting Both Outcome and Predictor Units

In a hypothetical multiple regression, a coefficient for infusion time is estimated as \(120\) milligrams per hour, with standard error \(30\) milligrams per hour. There are 30 observations and two predictors, including infusion time, so \(df=30-2-1=27\). Assume the predictors have full rank, the observations are independent, and residual checks are consistent with the stated linear, constant-variance, normal-error model. Convert the outcome from milligrams to grams and the predictor from hours to minutes.

The outcome conversion is \(Y^*=Y/1000\), and the predictor conversion is \(X^*=60X\). Thus \(a=1/1000\), \(c=60\), and \(k=a/c=1/60000\). The converted estimate and standard error are

\hat{\beta}_j^*=\frac{120}{60000}=0.002000\text{ grams per minute},\qquad SE(\hat{\beta}_j^*)=\frac{30}{60000}=0.0005000\text{ grams per minute}.

The original test statistic for a zero slope is \(120/30=4.00\); in the new units it is \(0.002000/0.0005000=4.00\). With 27 degrees of freedom, the two-sided \(p\)-value is less than \(0.001\) in both scales.

Using \(t_{0.975,27}\approx2.0518\), the original 95% interval is

120\pm2.0518(30)=120\pm61.554=[58.446,\ 181.554]\text{ milligrams per hour}.

Dividing both endpoints by 60,000 gives approximately \([0.0009741,\ 0.003026]\) grams per minute. As expected, the transformed interval has the same confidence level and is centered on the transformed coefficient.

Worked Example: Transforming a Nonzero Null Value

Consider a hypothetical multiple regression with 20 observations and two predictors, giving \(df=17\). For one predictor, suppose the estimated slope is \(24\) milligrams per hour and its standard error is \(8\) milligrams per hour. Assume the design has full rank, observations are independent, and the residual checks are consistent with the model assumptions above. Test whether the slope is \(12\) milligrams per hour, then express the same test in milligrams per minute.

In the original units, the null hypothesis is \(H_0:\beta_j=12\), and

t=\frac{24-12}{8}=1.50.

The two-sided \(p\)-value with 17 degrees of freedom is approximately \(0.152\), so this result is not statistically significant at the \(0.05\) level. The 95% interval, using \(t_{0.975,17}\approx2.1098\), is

24\pm2.1098(8)=24\pm16.8784=[7.1216,\ 40.8784]\text{ milligrams per hour}.

For minutes, the coefficient estimate is \(24/60=0.4000\), its standard error is \(8/60=0.1333\), and the null value must become \(12/60=0.2000\), all in milligrams per minute. The converted statistic is

t^*=\frac{0.4000-0.2000}{0.1333}=1.50.

Its two-sided \(p\)-value is again approximately \(0.152\). Dividing the original interval endpoints by 60 gives \([0.1187,\ 0.6813]\) milligrams per minute. Testing the converted estimate against the old null value of 12 would be a different, incorrectly specified question.

What Changes and What Does Not

These transformations matter because regression output is often compared across reports that use different measurement units. A coefficient of \(0.04\) points per minute and a coefficient of \(2.4\) points per hour may look numerically very different, but they describe the same fitted relationship. Their standard errors and interval widths differ numerically for the same reason: the units differ.

The following quantities change under a positive unit conversion:

The numerical coefficient and its standard error, in the same proportion.
The numerical endpoints of its confidence interval, in the converted units.
The units attached to the coefficient, standard error, and interval.

For the corresponding hypothesis test, the degrees of freedom, the magnitude of the \(t\)-statistic, and the \(p\)-value stay the same. The model's fitted relationships and residual pattern are also unchanged when the conversions are applied consistently. None of these facts implies that a regression association is causal; interpretation still depends on the study design and the model.

Common Mistakes

Keeping the standard error fixed. If a slope is divided by 60 to change its units from per hour to per minute, its standard error must also be divided by 60. Otherwise the computed test statistic and interval are inconsistent with the converted estimate.
Concluding that the \(p\)-value changes because the coefficient changes. The standard error changes by the same magnitude factor, so the ratio used by the test does not change in magnitude. For a two-sided test, the \(p\)-value is invariant.
Leaving a nonzero null value in the old units. A null value of 12 milligrams per hour is \(0.2\) milligrams per minute, not 12 milligrams per minute. The null parameter must be transformed along with the estimated coefficient.
Assuming rescaling fixes a poor model. A unit conversion changes numerical scales, not curvature, dependence, unequal error variance, or other problems with the model assumptions.
Ignoring direction in a one-sided test. Ordinary unit conversions use positive factors, so they preserve direction. If a variable is instead multiplied by a negative constant, the signed \(t\)-statistic reverses, and the corresponding one-sided alternative must reverse as well. The two-sided \(p\)-value remains unchanged.

A useful check is to calculate the statistic both before and after conversion. If the coefficient, null value, and standard error have all been converted correctly, the two-sided test must give the same result, apart from rounding.

Check Your Understanding

Answer each question using the transformation rules from this tutorial.

A slope is \(3.6\) units of outcome per hour, with standard error \(1.2\). What are the estimate and standard error per minute, and what happens to the zero-slope \(t\)-statistic?
If an outcome is converted by multiplying it by 100, by what factor does a slope standard error change when the predictor units are unchanged?
A 95% interval has endpoints \(2\) and \(8\) in the old units. If the coefficient is converted using a positive factor of \(0.1\), what are the new endpoints?
A test concerns whether a slope equals 5 units per hour. What null value should be used after converting the predictor from hours to minutes?
Why does a negative rescaling preserve a two-sided \(p\)-value but reverse the signed \(t\)-statistic?