Orthogonalized Gnanadesikan-Kettenring Scatter

Why Orthogonalize a Pairwise Robust Scatter Estimate?

Pairwise robust covariance estimates are appealing because they reduce a multivariate problem to robust univariate scale calculations. But assembling those estimates into a matrix does not guarantee a valid scatter matrix: the resulting symmetric matrix can have a negative eigenvalue. Orthogonalized Gnanadesikan–Kettenring (OGK) scatter addresses this by using the pairwise matrix to choose orthogonal directions, then estimating scale separately along those directions.

The resulting estimate is positive semidefinite when the rotated scales are nonnegative. That is a useful structural guarantee, not a claim of full affine equivariance. The coordinatewise standardization and sum-difference calculations generally make OGK depend on the coordinate system.

Definition: Let $X=(X_1,\ldots,X_p)^\top$ be a random vector, and let $s$ be a univariate scale functional that is translation invariant and scale equivariant: $s(a+bW)=|b|s(W)$. For variables $U,V$, the Gnanadesikan–Kettenring pairwise scale estimate is $$ \operatorname{GK}(U,V)=\frac{s^2(U+V)-s^2(U-V)}{4}. $$ For data, the same formula is applied to the corresponding columns, using a robust scale calculated across observations.

The formula is motivated by the covariance identity $$ \operatorname{Cov}(U,V)=\frac{\operatorname{Var}(U+V)-\operatorname{Var}(U-V)}{4}. $$ For a robust scale, however, the sum-difference formula is a proposed robust pairwise estimate; it is not automatically a covariance or part of a positive semidefinite matrix. In particular, the identity is exact for standard deviation because variance is quadratic in linear combinations. A general robust scale need not have that property.

The OGK Construction

Here is a one-pass version of OGK. Let $x_1,\ldots,x_n\in\mathbb{R}^p$ be observations. Choose a robust univariate location functional $t$ and scale functional $s$, with $s$ translation invariant and scale equivariant. Calculate a location $T_j$ and positive scale $S_j$ for each original coordinate $j$. The scale must be positive for every coordinate included in the calculation.

Standardize coordinatewise.
Set $y_{ij}=(x_{ij}-T_j)/S_j$. The resulting data matrix $Y$ has $n$ rows and $p$ columns.

Construct the pairwise matrix.
Set $U_{jj}=1$. For $j\ne k$, set $U_{jk}=\{s^2(Y_j+Y_k)-s^2(Y_j-Y_k)\}/4$, where each scale is computed over the $n$ observations. Then $U$ is symmetric.

Find orthogonal directions.
Take an orthogonal eigendecomposition $U=Q\Lambda Q^\top$, where $Q^\top Q=I$. Negative eigenvalues of $U$ are allowed at this stage.

Estimate scales after rotation.
For each column of $YQ$, calculate its robust scale $d_k$. Form $D=\operatorname{diag}(d_1^2,\ldots,d_p^2)$.

Transform scatter back.
The OGK scatter estimate is $\widehat{\Sigma}_{\mathrm{OGK}}=\operatorname{diag}(S_1,\ldots,S_p)\,QDQ^\top\,\operatorname{diag}(S_1,\ldots,S_p)$.

The robust scales $d_k$ in Step 4 are calculated from the rotated data, not obtained by taking square roots of the eigenvalues of $U$. This distinction matters if $U$ is indefinite. Orthogonalization supplies directions; the new univariate scale calculations supply the nonnegative variances used in the final estimate.

Worked Calculations

Worked Example: Recovering a Pairwise Covariance Under Normality

Suppose $(X_1,X_2)^\top$ is bivariate normal with mean zero and covariance matrix $$ \begin{pmatrix}4&2\\2&9\end{pmatrix}. $$ Use the normal-consistent median absolute deviation as the robust scale: the median absolute deviation is divided by $\Phi^{-1}(0.75)$, so for a normal variable with standard deviation $\sigma$, $s=\sigma$.

The sum has variance $4+9+2(2)=17$, so $s(X_1+X_2)=\sqrt{17}$. The difference has variance $4+9-2(2)=9$, so $s(X_1-X_2)=3$. Thus $$ \operatorname{GK}(X_1,X_2)=\frac{17-9}{4}=2. $$ This agrees with the covariance in the specified model. The marginal scales are $2$ and $3$; after standardization, the corresponding pairwise entry is $2/(2\cdot3)=1/3$. This agreement follows from normal-consistent scale calibration and the normal model, not from a general identity for every robust scale and distribution.

Worked Example: A Two-Dimensional OGK Estimate

Let $X$ be bivariate normal with location $(10,20)^\top$, marginal standard deviations $2$ and $4$, and correlation $0.6$. Again use the normal-consistent median absolute deviation. Coordinatewise standardization gives $Y_1=(X_1-10)/2$ and $Y_2=(X_2-20)/4$, so both standardized marginal scales are 1.

The standardized variables have variances 1 and covariance $0.6$. Their sum and difference have variances $2+2(0.6)=3.2$ and $2-2(0.6)=0.8$. Therefore their robust scales are $\sqrt{3.2}$ and $\sqrt{0.8}$, and the pairwise matrix is $$ U=\begin{pmatrix}1&0.6\\0.6&1\end{pmatrix}. $$ Its unit eigenvectors are $q_+=(1,1)^\top/\sqrt{2}$ and $q_-=(1,-1)^\top/\sqrt{2}$, with eigenvalues $1.6$ and $0.4$. The rotated standardized coordinates have variances $1.6$ and $0.4$, so the robust scales calculated in those directions are $\sqrt{1.6}$ and $\sqrt{0.4}$. Hence $D=\operatorname{diag}(1.6,0.4)$, and $$ QDQ^\top=\begin{pmatrix}1&0.6\\0.6&1\end{pmatrix}. $$ Transforming back with the original marginal scales gives $$ \widehat{\Sigma}_{\mathrm{OGK}}= \begin{pmatrix}2&0\\0&4\end{pmatrix} \begin{pmatrix}1&0.6\\0.6&1\end{pmatrix} \begin{pmatrix}2&0\\0&4\end{pmatrix} = \begin{pmatrix}4&4.8\\4.8&16\end{pmatrix}. $$ This equals the model covariance. In this Gaussian example, the calibrated robust scale of every linear combination equals its standard deviation, so the calculations reproduce the covariance exactly.

Why the Final Matrix Is Positive Semidefinite

Theorem: If $Q$ is orthogonal and $D$ is diagonal with nonnegative entries, then $QDQ^\top$ is positive semidefinite. Consequently, the one-pass OGK scatter estimate is positive semidefinite whenever its original-coordinate scales are positive and its rotated scales are nonnegative.

Proof. Let $v\in\mathbb{R}^p$ be arbitrary. Since $Q^\top v$ is a vector and $D=\operatorname{diag}(d_1^2,\ldots,d_p^2)$, $$ v^\top QDQ^\top v=(Q^\top v)^\top D(Q^\top v)=\sum_{k=1}^p d_k^2\bigl((Q^\top v)_k\bigr)^2\geq 0. $$ Thus $QDQ^\top$ is positive semidefinite. Write $A=\operatorname{diag}(S_1,\ldots,S_p)$. For any $w$, $w^\top A Q D Q^\top A w=(A w)^\top Q D Q^\top (A w)\geq 0$, so the back-transformed OGK estimate is also positive semidefinite. $\square$

Worked Example: Repairing an Indefinite Pairwise Matrix

Consider the symmetric matrix $$ G=\begin{pmatrix}1&0.9&0.9\\0.9&1&-0.9\\0.9&-0.9&1\end{pmatrix}. $$ It has eigenvectors $q_1=(1,-1,-1)^\top/\sqrt{3}$, $q_2=(2,1,1)^\top/\sqrt{6}$, and $q_3=(0,1,-1)^\top/\sqrt{2}$, with respective eigenvalues $-0.8$, $1.9$, and $1.9$. For example, multiplying $G$ by $(1,-1,-1)^\top$ gives $(-0.8,0.8,0.8)^\top$; multiplying by $(2,1,1)^\top$ gives $(3.8,1.9,1.9)^\top$; and multiplying by $(0,1,-1)^\top$ gives $(0,1.9,-1.9)^\top$. The vectors are mutually orthogonal, and their stated eigenvalues show that $G$ is indefinite.

Let $Q$ have these unit vectors as its columns in the listed order. Suppose the robust scales of the corresponding rotated coordinates are $0.5$, $0.8$, and $1.1$. The orthogonalized scatter is $QDQ^\top$, where $D=\operatorname{diag}(0.25,0.64,1.21)$. Direct multiplication gives $$ QDQ^\top= \begin{pmatrix} 0.51&0.13&0.13\\ 0.13&0.795&-0.415\\ 0.13&-0.415&0.795 \end{pmatrix}. $$ Its eigenvalues are the diagonal entries of $D$, namely $0.25$, $0.64$, and $1.21$, so it is positive definite. The negative eigenvalue of $G$ does not enter as a negative variance: $G$ determines the rotation, while the rotated robust scales determine $D$.

Derivation and Interpretation of the Pairwise Formula

The sum-difference construction comes from expanding two variances. The terms $\operatorname{Var}(U)$ and $\operatorname{Var}(V)$ appear in both expansions, while the covariance term changes sign: $$ \operatorname{Var}(U+V)=\operatorname{Var}(U)+\operatorname{Var}(V)+2\operatorname{Cov}(U,V), $$ $$ \operatorname{Var}(U-V)=\operatorname{Var}(U)+\operatorname{Var}(V)-2\operatorname{Cov}(U,V). $$ Subtracting gives $4\operatorname{Cov}(U,V)$, which proves the identity when $s^2$ is variance. Substituting a robust scale squared instead replaces variance by a robust measure of spread; the calculation motivates a robust pairwise estimate but does not prove it is a covariance matrix.

This distinction also explains why $U$ can be indefinite. Its entries come from different univariate scale calculations on different sums and differences. Nothing in translation invariance and scale equivariance alone forces these entries to satisfy all positive-semidefinite constraints. The eigenvector step is therefore substantive: it uses the pairwise matrix to select orthogonal directions, after which robust scales are estimated anew.

Affine Equivariance: What OGK Does Not Guarantee

A scatter estimator is affine equivariant if, for every nonsingular matrix $B$ and translation vector $a$, transforming observations by $x_i\mapsto a+Bx_i$ transforms the estimate according to $\widehat{\Sigma}\mapsto B\widehat{\Sigma}B^\top$. OGK is not generally affine equivariant. Its initial marginal scales are computed along the chosen coordinate axes, and its pairwise scales use sums and differences of those standardized coordinates. A general linear transformation mixes the axes, changing those marginal calculations and the subsequent pairwise matrix.

The sum-difference formula would produce a quadratic scatter form if the squared scale of every linear combination behaved as a quadratic form. Quadratic forms obey the parallelogram identity; a general robust scale functional need not. Thus there is no general algebraic guarantee that the pairwise entries transform as the entries of $B\widehat{\Sigma}B^\top$. Orthogonalization ensures a nonnegative quadratic form in the final estimate, but it does not undo the coordinate dependence already introduced.

In special settings, including the calibrated Gaussian example above, the scale calculations can coincide with standard deviations of linear combinations, and the result can recover an affine-equivariant target. That special agreement should not be mistaken for a general property of OGK. The estimator remains useful as a computationally direct robust scatter construction, but its positive-semidefinite output and its transformation behavior are separate questions.

Common Pitfall

Wrong reasoning: “The pairwise matrix is symmetric, so it is a covariance matrix; if it has a negative eigenvalue, the eigenvalue must be an error, or it should be replaced by its square root.” Symmetry alone does not imply positive semidefiniteness, and a negative eigenvalue cannot be square-rooted as a real variance.

Correction: Treat the pairwise matrix as an intermediate direction-finding matrix. Use an orthogonal eigendecomposition even if some eigenvalues are negative, calculate robust scales of the rotated coordinates, and form $QDQ^\top$ from their squares. This produces a positive-semidefinite scatter estimate, but does not establish affine equivariance.

Key takeaway: OGK combines coordinatewise robust standardization, robust sum-difference pairwise estimates, an orthogonal rotation, and robust scales in the rotated directions. The final scale matrix is positive semidefinite by construction; general affine equivariance does not follow.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

What assumption on the univariate scale makes the sum-difference formula equal covariance for every pair of variables?
Why can the pairwise matrix in OGK have a negative eigenvalue without preventing construction of the final scatter estimate?
For $U=\begin{pmatrix}1&0.4\\0.4&1\end{pmatrix}$, identify its unit eigenvectors and eigenvalues.
If $D$ has nonnegative diagonal entries and $Q$ is orthogonal, show why $QDQ^\top$ cannot have a negative quadratic form.
Which stages of OGK explain why it is not generally affine equivariant?