Studentization and Scale-Invariant Mean Tests

Why Measurement Units Matter

A squared mean norm gives more weight to coordinates with larger numerical values. That can be appropriate when the coordinates share a meaningful common scale, but it can be misleading when one variable is measured in millimetres and another in metres. A change of units should not, by itself, alter the evidence in a test that is intended to treat coordinates relative to their own variability.

Coordinate studentization addresses this problem by dividing each squared sample mean by an estimate of that coordinate’s variance. This makes the resulting statistic invariant to separate changes of units. But invariance is not the same as correct calibration: standardizing each coordinate does not remove correlation among coordinates.

Definition: Let $\mathbf X_1,\ldots,\mathbf X_n$ be independent and identically distributed $p$-dimensional random vectors with mean $\boldsymbol\mu$ and covariance matrix $\boldsymbol\Sigma$. Write $\bar{\mathbf X}=n^{-1}\sum_{i=1}^n\mathbf X_i$, and let $s_j^2=(n-1)^{-1}\sum_{i=1}^n(X_{ij}-\bar X_j)^2$ be the sample variance of coordinate $j$. For testing $H_0:\boldsymbol\mu=\mathbf 0$, define the raw quadratic statistic and the diagonally studentized statistic by $$ Q_n=n\sum_{j=1}^p\bar X_j^2,\qquad T_n=n\sum_{j=1}^p\frac{\bar X_j^2}{s_j^2}. $$ The studentized statistic is defined when every $s_j^2>0$.

The factor $n$ puts the squared mean on the scale of sampling variation: under the null, each coordinate of $\sqrt n\,\bar{\mathbf X}$ typically has variance of constant order. $Q_n$ uses the same numerical scale for every coordinate. $T_n$ instead measures each sample mean relative to its estimated marginal standard deviation. Neither statistic uses the off-diagonal entries of the sample covariance matrix.

What a Change of Units Does

Suppose the recorded coordinates are converted by $Y_{ij}=d_jX_{ij}$, where each $d_j>0$. In vector notation, $\mathbf Y_i=D\mathbf X_i$, with $D=\operatorname{diag}(d_1,\ldots,d_p)$. Then $\bar Y_j=d_j\bar X_j$, and the sample variance transforms as $s_{Y,j}^2=d_j^2s_j^2$.

Theorem: Under separate positive rescalings of the coordinates, the raw statistic becomes $Q_{n,Y}=n\sum_jd_j^2\bar X_j^2$, whereas the diagonally studentized statistic is unchanged: $T_{n,Y}=T_n$.

Proof. Substitute $\bar Y_j=d_j\bar X_j$ into the raw statistic to obtain

Q_{n,Y}=n\sum_{j=1}^p\bar Y_j^2 =n\sum_{j=1}^pd_j^2\bar X_j^2.

For the studentized statistic, substitution of both transformed quantities gives

T_{n,Y} =n\sum_{j=1}^p\frac{\bar Y_j^2}{s_{Y,j}^2} =n\sum_{j=1}^p\frac{d_j^2\bar X_j^2}{d_j^2s_j^2} =n\sum_{j=1}^p\frac{\bar X_j^2}{s_j^2} =T_n.

The cancellation holds coordinate by coordinate, provided the sample variances are nonzero. It establishes unit invariance, not a reference distribution for $T_n$. $\square$

Worked Example: Changing the Units of Two Coordinates

Suppose four observations have first-coordinate values $1,2,3,4$, and second-coordinate values $2,0,2,0$. Compute both statistics, then convert the first coordinate by a factor of $10$ and the second by a factor of $0.1$.

The coordinate means are $\bar X_1=2.5$ and $\bar X_2=1$. The sample variances are $s_1^2=[(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2]/3=5/3$ and $s_2^2=[1^2+(-1)^2+1^2+(-1)^2]/3=4/3$. Thus

Q_n=4(2.5^2+1^2)=4(6.25+1)=29, \qquad T_n=4\left(\frac{6.25}{5/3}+\frac{1}{4/3}\right) =4(3.75+0.75)=18.

After conversion, the means are $25$ and $0.1$, while the variances are $100(5/3)$ and $0.01(4/3)$. Therefore

Q_{n,Y}=4(25^2+0.1^2)=2500.04,\qquad T_{n,Y}=4\left(\frac{625}{100(5/3)}+ \frac{0.01}{0.01(4/3)}\right)=18.

The raw value changes dramatically because the numerical size of the first coordinate changed. The studentized value is exactly the same, as the invariance result predicts.

The Null Limit and the Role of Correlation

For fixed dimension, the null behavior of $T_n$ can be derived directly. Let $\sigma_j^2=\operatorname{Var}(X_{1j})>0$, and define the population correlation matrix $R$ by $R_{jk}=\operatorname{Cov}(X_{1j},X_{1k})/(\sigma_j\sigma_k)$.

Theorem: Suppose $p$ is fixed, the vectors $\mathbf X_i$ are iid with $\boldsymbol\mu=\mathbf 0$, and $0<\sigma_j^2<\infty$ for every coordinate. Then $$ T_n\xrightarrow{d}\mathbf Z^\mathsf{T}R\mathbf Z, \qquad \mathbf Z\sim N_p(\mathbf 0,I_p). $$ In particular, if $R=I_p$, the limit is $\chi_p^2$. Diagonal studentization alone does not generally yield a $\chi_p^2$ limit when coordinates are correlated.

Proof. The multivariate central limit theorem gives $\sqrt n\,\bar{\mathbf X}\xrightarrow{d}N_p(\mathbf 0,\boldsymbol\Sigma)$. The weak law of large numbers applied to each coordinate and its square implies $s_j^2\xrightarrow{p}\sigma_j^2$. Since $p$ is fixed and each $\sigma_j^2>0$, the vector of sample variances converges in probability to a vector with strictly positive entries. Slutsky’s theorem therefore allows replacement of each $s_j^2$ by $\sigma_j^2$ in the limit:

T_n\xrightarrow{d} \sum_{j=1}^p\frac{W_j^2}{\sigma_j^2}, \qquad \mathbf W\sim N_p(\mathbf 0,\boldsymbol\Sigma).

Writing $W_j=\sigma_j Z_j$ jointly gives a standardized Gaussian vector with covariance $R$. Hence the limit is $\sum_j Z_j^2$, where the $Z_j$ in this expression are jointly distributed with correlation matrix $R$; equivalently, it is $\mathbf U^\mathsf{T}\mathbf U$ for $\mathbf U\sim N_p(\mathbf 0,R)$. In the theorem’s displayed representation, $\mathbf U$ can be written $R^{1/2}\mathbf Z$, so the limit is $\mathbf Z^\mathsf{T}R\mathbf Z$ for independent standard-normal coordinates in $\mathbf Z$. If $R=I_p$, this is the sum of $p$ independent squared standard normals, which has a $\chi_p^2$ distribution. $\square$

Worked Example: Correlation Changes the Limiting Calibration

Take $p=2$, with both marginal variances equal to $1$ and correlation $\rho=0.8$. The limiting studentized statistic under the theorem is $\mathbf Z^\mathsf{T}R\mathbf Z$, where $R=\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix}$. The eigenvalues of $R$ are $1.8$ and $0.2$, so an orthogonal change of coordinates expresses the limit as

1.8Z_1^2+0.2Z_2^2, \qquad Z_1,Z_2\ \text{independent }N(0,1).

This is not $\chi_2^2$, whose corresponding weights are $1$ and $1$. Both distributions have mean $2$, but their variances differ. The variance of the displayed limit is $2(1.8^2+0.2^2)=6.56$, using $\operatorname{Var}(Z_j^2)=2$; the variance of $\chi_2^2$ is $4$. Thus matching the mean, or dividing by marginal standard deviations, does not justify a chi-square calibration.

The theorem is a fixed-$p$ result. It does not by itself justify a chi-square approximation when $p$ grows with $n$, nor does it establish that estimating all marginal variances is accurate uniformly over a growing set of coordinates. High-dimensional calibration requires additional assumptions and an appropriate reference distribution. The key point here is narrower: even in fixed dimension, the limiting law retains the correlation matrix.

When Diagonal Studentization Is Valid

There are two distinct questions. First, is a statistic invariant to the units in which its coordinates are recorded? Second, is its null distribution correctly calibrated for the dependence in the data? $T_n$ answers the first question affirmatively for positive coordinate-wise rescalings. It answers the second only with additional information or assumptions.

In fixed dimension, if the coordinates are uncorrelated, then $R=I_p$ and the theorem gives a $\chi_p^2$ limiting null distribution. For jointly Gaussian coordinates, zero correlations imply independence. More generally, independence is a sufficient condition for $R=I_p$, but uncorrelatedness alone does not imply independence outside special distributional settings. If correlations are present, a chi-square calibration that ignores them is generally not justified; one must account for the correlation structure or use a calibration designed for the setting.

Studentizing each coordinate also does not mean that the full covariance has been standardized. A full covariance adjustment would involve off-diagonal terms as well as marginal variances. That is a different operation, and in high dimensions it may be difficult to estimate reliably. Diagonal studentization is particularly useful when unit invariance is essential and marginal variances can be estimated, but it should not be mistaken for whitening the vector.

Worked Example: Independent Gaussian Coordinates

Suppose $p=2$ and the observations have independent coordinates, each normally distributed with mean zero and variance $\sigma_j^2>0$. For each coordinate, the usual one-sample $t$ statistic is $t_j=\sqrt n\,\bar X_j/s_j$. Under these assumptions, each $t_j$ has a $t_{n-1}$ distribution, and the two statistics are independent because they are computed from independent coordinate data. Consequently,

$$ T_n=t_1^2+t_2^2. $$

For finite $n$, this is a sum of two independent squared $t_{n-1}$ variables, not exactly a $\chi_2^2$ variable. For example, at $n=10$, each squared $t_9$ has expectation $9/(9-2)=9/7$, so $E[T_n]=18/7$, whereas a $\chi_2^2$ variable has expectation $2$. As $n$ increases, each $t_{n-1}$ converges in distribution to a standard normal, and the sum converges to $\chi_2^2$, in agreement with the fixed-dimension theorem.

Worked Example: Why a Correlated Pair Is Not Made Independent by Studentization

Suppose the two coordinates have marginal variances $4$ and $9$, and correlation $0.5$. Their covariance is $0.5\sqrt{4}\sqrt{9}=3$, so $\boldsymbol\Sigma=\begin{pmatrix}4&3\\3&9\end{pmatrix}$. After dividing each coordinate by its own standard deviation, the marginal variances become $1$, but the covariance becomes $3/(2\cdot3)=0.5$. Thus the standardized correlation matrix is still $R=\begin{pmatrix}1&0.5\\0.5&1\end{pmatrix}$, not $I_2$. Its eigenvalues are $1.5$ and $0.5$, and the limiting statistic has distribution $1.5Z_1^2+0.5Z_2^2$, not $\chi_2^2$.

The calculation shows exactly what diagonal studentization does: it removes marginal scales $2$ and $3$, while leaving the dimensionless association $0.5$ intact.

Common Mistake and Practical Interpretation

Wrong reasoning: “I divided each coordinate by its sample standard deviation, so the coordinates now have unit variance and the sum of squares must be chi-square.” This confuses unit variance with independence. Standardization fixes marginal scales; it does not eliminate off-diagonal correlations.

Correction: Check the dependence structure and the calibration theorem being used. In the fixed-dimensional result above, the limit is $\mathbf Z^\mathsf{T}R\mathbf Z$, reducing to $\chi_p^2$ when $R=I_p$. In a high-dimensional application, do not transfer that fixed-dimensional conclusion without verifying the relevant high-dimensional assumptions.

Scale invariance matters when variables are recorded in different units or when investigators need a test whose numerical result does not depend on arbitrary coordinate scaling. It is a valuable design property, but it is not a substitute for modeling dependence. The practical distinction is simple: marginal standardization controls units; calibration controls false-positive probability.

Check Your Understanding

Use the definitions and results above to answer these questions.

Under the transformation $Y_{ij}=d_jX_{ij}$, what happens to the raw statistic and to the diagonally studentized statistic?
State the assumptions of the fixed-dimension null-limit theorem and its limiting distribution.
Why does a correlation matrix different from the identity generally invalidate a $\chi_p^2$ calibration for the diagonally studentized statistic?
In the independent Gaussian example, why is the finite-sample statistic not exactly chi-square?
What is the difference between scale invariance and valid null calibration?