Scaling Variables and Equations

Why Scale a Linear System?

A linear system may describe quantities measured in very different units or at very different typical sizes. One unknown might be measured in thousands, while another is close to one; one equation might have coefficients in the thousands, while another has coefficients near one. Those differences can make a system harder to read or work with. Scaling changes the representation so the quantities are more comparable, while keeping careful track of how the transformed problem relates to the original.

Write a square system as \(\mathbf{A}\mathbf{x}=\mathbf{b}\), where \(\mathbf{A}\in\mathbb{R}^{n\times n}\), \(\mathbf{x}\in\mathbb{R}^n\) is the column of unknowns, and \(\mathbf{b}\in\mathbb{R}^n\) is the right-hand side. The scaling arguments below also apply to rectangular systems. An equation can be rescaled by multiplying both sides by a nonzero number. An unknown can be rescaled by expressing it as a nonzero scale factor times a new coordinate. These are different operations: the first changes the equations’ representation, while the second changes the coordinates used for the unknowns.

The diagonal entries of \(\mathbf{S}\) and \(\mathbf{D}\) are the scale factors. They must all be nonzero: this makes \(\mathbf{S}\) and \(\mathbf{D}\) invertible. Positive scale factors are usual when scales represent units or typical magnitudes, though the algebraic equivalence only requires nonzero entries. Multiplying on the left scales rows, because it multiplies each equation by its corresponding factor. Multiplying on the right scales columns, because it multiplies each unknown’s column of coefficients by its factor.

What the Transformations Preserve

Proof. Suppose \(\mathbf{x}\) solves \(\mathbf{A}\mathbf{x}=\mathbf{b}\), and define \(\mathbf{y}=\mathbf{D}^{-1}\mathbf{x}\). Then \(\mathbf{D}\mathbf{y}=\mathbf{x}\), so \(\mathbf{S}\mathbf{A}\mathbf{D}\mathbf{y}=\mathbf{S}\mathbf{A}\mathbf{x}=\mathbf{S}\mathbf{b}\). Conversely, suppose \(\mathbf{y}\) solves the transformed system and set \(\mathbf{x}=\mathbf{D}\mathbf{y}\). Then \(\mathbf{S}\mathbf{A}\mathbf{x}=\mathbf{S}\mathbf{b}\). Since \(\mathbf{S}\) is invertible, multiply by \(\mathbf{S}^{-1}\) to obtain \(\mathbf{A}\mathbf{x}=\mathbf{b}\). The assignments \(\mathbf{x}=\mathbf{D}\mathbf{y}\) and \(\mathbf{y}=\mathbf{D}^{-1}\mathbf{x}\) undo each other, so this is a one-to-one correspondence between the two solution sets. \(\square\)

In particular, these transformations preserve whether a system has no solution, exactly one solution, or more than one solution. They do not make a nonexistent solution appear. If a unique solution is obtained in scaled coordinates, it must still be converted back to the original unknowns before interpretation.

Scaling Equations: Changing Row Magnitudes

Equation scaling is useful when equations have different units or widely different coefficient sizes. For instance, an equation measured in millimetres might be rescaled to metres to match the units of other equations. But a smaller row is not automatically a more informative row: rescaling changes the numerical size of its residual as well. If the original residual is \(\mathbf{r}=\mathbf{b}-\mathbf{A}\mathbf{x}\), then the scaled residual is \(\mathbf{S}\mathbf{r}\). Comparisons of residual sizes therefore depend on the scaling chosen.

Scaling Variables: Changing Coordinates

The choice \(x=1000z\) expresses the original unknown in units of a thousand. More generally, if typical values of the original unknowns are \(d_1,\dots,d_n\), it can be useful to write \(x_j=d_jy_j\). Values of the new coordinates near one then represent unknowns near their chosen scales. Such a choice can make a model easier to read and can reduce large differences among column magnitudes.

Combining Equation and Variable Scales

This example illustrates two distinct choices. The row factors bring the equations to more comparable sizes, while the variable factor expresses \(y\) in units of a thousand. The resulting matrix has moderate, easy-to-read entries. That is a practical benefit, not a guarantee that every computation becomes more accurate or that every measure of conditioning improves. The effect depends on the system, the scales, and how numerical sizes are measured.

Common Pitfall: Returning the Scaled Coordinates

Suppose \(2x+y=5\) and \(x-y=1\), and choose \(x=10z\). Since \(x=10z\), the correct transformed coefficient matrix is obtained by multiplying the first column by \(10\): \(\begin{bmatrix}20&1\\10&-1\end{bmatrix}\). The transformed equations give \(20z+y=5\) and \(10z-y=1\). Adding gives \(30z=6\), so \(z=\tfrac{1}{5}\), and then \(y=1\). The original unknown is \(x=10z=2\).

A tempting but wrong calculation is to multiply the first column by \(\tfrac{1}{10}\) instead, as if \(x=\tfrac{1}{10}z\). That would produce \(\begin{bmatrix}\tfrac{1}{5}&1\\\tfrac{1}{10}&-1\end{bmatrix}\), which does not represent the stated substitution \(x=10z\). Another error is to report \(z=\tfrac{1}{5}\) as the value of \(x\). Checking against the original equations catches it: \(2(\tfrac{1}{5})+1=\tfrac{7}{5}\ne5\). The correction is to use \(\mathbf{A}\mathbf{D}\) when \(\mathbf{x}=\mathbf{D}\mathbf{y}\), then convert the computed coordinates back with \(\mathbf{x}=\mathbf{D}\mathbf{y}\).

Choosing Scales with Care

Useful scales come from the meaning and typical size of the quantities, not from a rule that every entry must look alike. Before scaling, identify which unknowns are measured in different units or are expected to have different magnitudes. For equation scaling, identify whether multiplying an equation changes how its residual should be interpreted. After solving, check the recovered answer in the original system, not only in the scaled one.

Scaling can improve interpretation by making coordinates represent natural units, and it can make a matrix easier to inspect by reducing extreme differences in row or column sizes. It is not the same as changing the underlying model: with nonzero scale factors and the correct conversion back, the exact solutions correspond one-to-one. But a scale chosen without regard to units or meaning can obscure the model, and scaling alone does not guarantee a better-conditioned system or a more reliable computation.