Numerical Rank, Pseudoinverses, and Tolerance Choices

Why Numerical Rank Requires a Tolerance

A matrix can be exactly full rank and still behave, for computation, almost as if it were rank deficient. This occurs when some singular values are tiny relative to the largest: inverting them amplifies small changes in the data, while rounding can make the distinction between zero and a very small value unreliable. In statistical work, that distinction matters because it can change fitted values, coefficient sizes, and the dimension of a fitted subspace.

Numerical rank is therefore not simply an exact property read from a computer’s output. It is a decision made relative to a tolerance. The tolerance should reflect both the scale and precision of the computation and the question being asked.

Definition: Let $A$ be an $m \times n$ real matrix with singular values $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_q \geq 0$, where $q=\min(m,n)$. For a specified tolerance $\tau \geq 0$, the numerical rank of $A$ at tolerance $\tau$ is $$ r_\tau(A)=\#\{i:\sigma_i>\tau\}. $$ The inequality is strict: a singular value equal to $\tau$ is not retained. A common choice is $\tau=\max(\tau_{\mathrm{abs}},\tau_{\mathrm{rel}}\sigma_1)$, using stated absolute and relative tolerances. The choice is part of the numerical procedure and should be reported.

If $A=0$, all its singular values are zero, so its numerical rank is zero for any nonnegative tolerance. More generally, a tolerance at least as large as $\sigma_1$ also gives rank zero. In contrast, the exact rank counts singular values that are mathematically nonzero. Numerical rank can differ from exact rank because finite precision cannot reliably distinguish an extremely small singular value from zero in every context.

The Truncated Pseudoinverse

Write a singular value decomposition of $A$ as $A=U\Sigma V^T$, with singular values in descending order. Let $r=r_\tau(A)$, and denote the corresponding left and right singular vectors by $u_i$ and $v_i$. The truncated matrix keeps the singular directions above the cutoff and discards the others.

Definition: The truncated matrix and its truncated pseudoinverse at tolerance $\tau$ are $$ A_\tau=\sum_{i=1}^{r}\sigma_i u_i v_i^T, \qquad A_\tau^+=\sum_{i=1}^{r}\frac{1}{\sigma_i}v_i u_i^T. $$ An empty sum is the zero matrix. When $r=0$, $A_\tau^+=0$. This is the Moore–Penrose pseudoinverse of the truncated matrix $A_\tau$; it need not equal the Moore–Penrose pseudoinverse of the original $A$, which retains every mathematically nonzero singular value.

For a response vector $y\in\mathbb{R}^m$, the truncated coefficient vector is $\hat\beta_\tau=A_\tau^+y$. Its fitted vector using the truncated matrix is $A_\tau\hat\beta_\tau$. The same fitted vector results from multiplying $\hat\beta_\tau$ by the original $A$, because the coefficient has no component in any discarded right-singular direction.

Worked Example: Two Cutoffs, Two Numerical Ranks

Consider $A=\operatorname{diag}(10,0.4,0.02)$. Its singular values are $10$, $0.4$, and $0.02$. Compare an absolute tolerance of $0.05$ with a relative tolerance of $0.05\sigma_1$.

Absolute cutoff. Here $\tau=0.05$. Since $10>0.05$, $0.4>0.05$, and $0.02$ is not greater than $0.05$, the numerical rank is 2. The truncated pseudoinverse is $$ A_{0.05}^+=\operatorname{diag}(0.1,2.5,0). $$

Relative cutoff. Here $\tau=0.05(10)=0.5$. Only $10$ exceeds $0.5$, so the numerical rank is 1, and $$ A_{0.5}^+=\operatorname{diag}(0.1,0,0). $$

The same singular values produce different retained subspaces under these two rules. A statement such as “the numerical rank is 2” is incomplete unless the tolerance convention is also given.

What Truncation Does to Least-Squares Fits

The retained left singular vectors span the directions in response space that the truncated matrix can fit. The result below makes precise how the truncated pseudoinverse chooses coefficients and fitted values.

Theorem: Suppose $r=r_\tau(A)>0$, and let $U_r$ and $V_r$ contain the first $r$ left and right singular vectors, respectively. For any $y\in\mathbb{R}^m$, the vector $$ \hat\beta_\tau=V_r\operatorname{diag}(1/\sigma_1,\ldots,1/\sigma_r)U_r^T y $$ is the minimum-Euclidean-norm coefficient vector among the minimizers of $\|A_\tau\beta-y\|_2$. Its fitted vector is $U_rU_r^Ty$, the orthogonal projection of $y$ onto the retained left-singular subspace.

Proof. Set $z=U_r^Ty$. For any coefficient vector $\beta$, the truncated fit is $$ A_\tau\beta=U_r\operatorname{diag}(\sigma_1,\ldots,\sigma_r)V_r^T\beta. $$ Because the columns of $U_r$ are orthonormal, the residual decomposes into a component in their span and a component perpendicular to that span. Thus $$ \|A_\tau\beta-y\|_2^2 = \|\operatorname{diag}(\sigma_1,\ldots,\sigma_r)V_r^T\beta-z\|_2^2 +\|(I-U_rU_r^T)y\|_2^2. $$ The second term does not depend on $\beta$. The first term is minimized at zero, which is attainable because each retained $\sigma_i$ is positive: take $V_r^T\beta=\operatorname{diag}(1/\sigma_1,\ldots,1/\sigma_r)z$. Hence the fitted vector is $U_rz=U_rU_r^Ty$. Any coefficient with these retained coordinates can also include a component orthogonal to the columns of $V_r$, but that component does not affect the fit and increases the squared norm unless it is zero. The displayed $\hat\beta_\tau$ has zero such component and is therefore the minimum-norm minimizer. $\square$

This result describes the truncated fit, not necessarily the best fit using every direction of the original matrix. If a direction is discarded, the fit cannot use it even when the exact matrix has a nonzero singular value there. Conversely, retaining a very small singular value can make a coefficient large in order to fit a small response component.

Worked Example: A Small Singular Value and a Large Coefficient

Let $A=\operatorname{diag}(4,1,0.01)$ and $y=(8,2,1)^T$. First use $\tau=0.05$. The singular values $4$ and $1$ exceed the cutoff, while $0.01$ does not, so $r=2$ and $$ A_\tau^+=\operatorname{diag}(0.25,1,0). $$ The coefficient and fitted vectors are $$ \hat\beta_\tau=A_\tau^+y=(2,2,0)^T, \qquad A\hat\beta_\tau=(8,2,0)^T. $$ The residual is $y-A\hat\beta_\tau=(0,0,1)^T$, with squared norm $1$.

Now lower the cutoff to $\tau=0.005$. All three singular values are retained, giving $$ A_\tau^+=\operatorname{diag}(0.25,1,100). $$ Consequently, $\hat\beta_\tau=(2,2,100)^T$, and $A\hat\beta_\tau=(8,2,1)^T$. The residual is zero, but fitting the final response component requires a coefficient of $100$, because its design direction has scale $0.01$.

If these are three observations in an unregularized linear model, the rank-2 fit leaves one residual degree of freedom, whereas the rank-3 fit leaves none. The latter fit alone does not provide a residual variance estimate with positive residual degrees of freedom. Thus the cutoff can change not just the fitted vector but the conditions for downstream uncertainty calculations.

Conditioning and Sensitivity of the Coefficients

The smallest retained singular value controls how much the truncated inverse can amplify a change in the response. This statement follows directly from the orthonormal singular-vector representation.

Result: If $r>0$, then $$ \|A_\tau^+\|_2=\frac{1}{\sigma_r}, \qquad \|A_\tau^+y\|_2\leq \frac{\|y\|_2}{\sigma_r}. $$ For two response vectors $y$ and $\tilde y$, with the same matrix and cutoff, $$ \|A_\tau^+y-A_\tau^+\tilde y\|_2 \leq \frac{\|y-\tilde y\|_2}{\sigma_r}. $$

Proof. For any $y$, orthonormality of the columns of $V_r$ gives $$ \|A_\tau^+y\|_2^2 = \sum_{i=1}^{r}\frac{(u_i^Ty)^2}{\sigma_i^2} \leq \frac{1}{\sigma_r^2}\sum_{i=1}^{r}(u_i^Ty)^2 \leq \frac{\|y\|_2^2}{\sigma_r^2}. $$ The last inequality holds because the sum of squared coordinates along orthonormal vectors cannot exceed the squared norm. Equality in the operator norm is attained by taking $y=u_r$, for which $\|A_\tau^+u_r\|_2=1/\sigma_r$. Hence $\|A_\tau^+\|_2=1/\sigma_r$. Applying the same bound to $y-\tilde y$ proves the final inequality. $\square$

A cutoff that retains a very small $\sigma_r$ can therefore produce a large response to even a modest change in $y$. Raising the cutoff may improve this coefficient stability by removing that direction, but it also changes the fitted subspace and can increase residual error. Tolerance selection is a trade-off, not a purely mechanical way to discover an unambiguous rank.

Worked Example: Scaling Changes a Relative Numerical Rank

Take $A=\operatorname{diag}(3,0.03)$ and use a relative tolerance of $0.02$, so that $\tau=0.02\sigma_1$. Initially, $\sigma_1=3$ and $\tau=0.06$. The singular value $0.03$ is below the cutoff, so the numerical rank is 1.

Now change the units of the second variable by multiplying the second column by $100$. This gives $AD=\operatorname{diag}(3,3)$, where $D=\operatorname{diag}(1,100)$. The new relative cutoff is $0.02(3)=0.06$, and both singular values exceed it. The numerical rank is now 2.

Multiplying a column by a nonzero constant preserves exact rank, but it changes the singular values and can change numerical rank. This is why units and scaling matter when a cutoff is applied to a design matrix. Standardizing variables can be appropriate, but it changes the parameterization and should be justified by the analysis rather than treated as a neutral computational step.

Choosing and Reporting a Tolerance

Software often supplies a default tolerance based on machine precision, matrix dimensions, and the largest singular value. Such defaults are useful starting points, not universal definitions of the scientifically relevant rank. Measurement error, rounding in the data, variable scaling, and the intended downstream calculation can all matter. In particular, a tolerance meant to detect directions below numerical precision may be too small for data whose entries are uncertain much more than that.

When the conclusion depends on the selected rank, examine a reasonable range of tolerances. Compare the retained singular values, fitted values, coefficient sizes, and relevant downstream quantities. Report the cutoff rule and its value, rather than reporting only the resulting rank. A sensitivity check is especially important when a singular value lies close to the cutoff, since a small change in the threshold can abruptly add or remove a direction.

Common Pitfall

Wrong reasoning: “The matrix has a tiny nonzero singular value, so its inverse must include that direction; omitting it changes the mathematical answer.” This confuses the exact pseudoinverse with a truncated pseudoinverse. The truncated calculation intentionally replaces $A$ with $A_\tau$, so it can stabilize coefficients at the cost of changing the fitted subspace.

Correction: State which matrix is being analyzed and which tolerance is used. If the goal is an exact algebraic statement, use exact rank and the Moore–Penrose pseudoinverse of $A$. If the goal is a numerically stable calculation, specify the truncated rank and interpret results as conditional on that cutoff. Neither choice should be presented as though it were the other.

Key takeaway: A numerical rank is a tolerance-dependent computational decision. The truncated pseudoinverse retains and inverts only singular directions above that tolerance; this determines both the fitted subspace and the amplification of response changes.

Check Your Understanding

Use the stated strict cutoff rule: retain a singular value only when it is greater than the tolerance.

A matrix has singular values $6$, $0.3$, and $0.02$. What is its numerical rank at absolute tolerance $0.1$? What is its rank at relative tolerance $0.1\sigma_1$?
Why does the truncated fitted vector equal the projection of $y$ onto the retained left-singular subspace?
For a nonzero truncated rank, what role does the smallest retained singular value play in bounding coefficient changes?
Explain why rescaling a column can change numerical rank while preserving exact rank.
What information should accompany a reported numerical rank when it is used in a statistical analysis?