Measuring Sensitivity in a Linear System
A linear system can have a unique solution and still be sensitive: a small change in its input data may cause a much larger change in the solution. The condition number of its coefficient matrix is a measure of this possibility. It gives a scale for assessing sensitivity, but it does not say that every small data change will produce a large solution change. The direction of the change matters too.
For an invertible matrix \(\mathbf{A}\in\mathbb{R}^{n\times n}\), the system \(\mathbf{A}\mathbf{x}=\mathbf{b}\) has the solution \(\mathbf{x}=\mathbf{A}^{-1}\mathbf{b}\). We will measure vector size using the infinity norm, \(\|\mathbf{v}\|_{\infty}=\max_i |v_i|\), and use its induced matrix norm. The induced matrix norm measures the largest factor by which a matrix can enlarge a vector in that vector norm. For the infinity norm, it is the maximum absolute row sum:
The norm must be specified because different norms can give different numerical values. The condition number is a property of the matrix together with the chosen norm. It is always finite for an invertible square matrix in finite dimensions. A larger value means that the matrix allows greater possible amplification in the relationship between input data and solution; a value near \(1\) indicates less amplification by this measure.
Basic Properties of the Condition Number
Proof. The induced matrix norm is submultiplicative: \(\|\mathbf{B}\mathbf{C}\|\le\|\mathbf{B}\|\|\mathbf{C}\|\) for compatible matrices. Since \(\mathbf{A}\mathbf{A}^{-1}=\mathbf{I}_n\), this gives \(\|\mathbf{I}_n\|\le\|\mathbf{A}\|\|\mathbf{A}^{-1}\|\). The induced norm of \(\mathbf{I}_n\) is \(1\), because \(\|\mathbf{I}_n\mathbf{v}\|=\|\mathbf{v}\|\) for every vector \(\mathbf{v}\). Thus \(\kappa(\mathbf{A})\ge1\). Next, \((c\mathbf{A})^{-1}=c^{-1}\mathbf{A}^{-1}\), and absolute homogeneity of the norm gives \(\|c\mathbf{A}\|=|c|\|\mathbf{A}\|\) and \(\|c^{-1}\mathbf{A}^{-1}\|=|c|^{-1}\|\mathbf{A}^{-1}\|\). Multiplying these equalities proves \(\kappa(c\mathbf{A})=\kappa(\mathbf{A})\). \(\square\)
The second property is useful when a whole system is rescaled: multiplying every coefficient by the same nonzero constant does not change this condition number. It also warns against interpreting the size of \(\|\mathbf{A}\|\) alone as a measure of conditioning. The inverse norm matters as well.
Computing Condition Numbers
Worked Example: A Triangular Matrix
Find the infinity-norm condition number of \(\mathbf{A}=\begin{bmatrix}1&1\\0&1\end{bmatrix}\).
First find the inverse. Direct multiplication verifies the candidate:
The absolute row sums of \(\mathbf{A}\) are \(2\) and \(1\), so \(\|\mathbf{A}\|_{\infty}=2\). The absolute row sums of \(\mathbf{A}^{-1}\) are \(2\) and \(1\), so \(\|\mathbf{A}^{-1}\|_{\infty}=2\). Therefore
Check: the inverse product shown above equals \(\mathbf{I}_2\), and the two maximum row sums are \(2\). Their product, \(4\), is the condition number.
For diagonal matrices, the row sums make the calculation especially direct.
Proof. Each row of \(\mathbf{D}\) has exactly one possibly nonzero entry, so its absolute row sum is \(|d_i|\). Hence \(\|\mathbf{D}\|_{\infty}=\max_i|d_i|\). The inverse is diagonal with entries \(1/d_i\), so \(\|\mathbf{D}^{-1}\|_{\infty}=\max_i|1/d_i|=1/\min_i|d_i|\). The last equality uses the assumption that every \(d_i\) is nonzero. Multiplying the two norms gives the stated formula. \(\square\)
Worked Example: A Diagonal System with a Large Condition Number
Consider \(\mathbf{D}=\begin{bmatrix}1&0\\0&\tfrac{1}{100}\end{bmatrix}\). Its inverse is \(\mathbf{D}^{-1}=\begin{bmatrix}1&0\\0&100\end{bmatrix}\), and multiplication confirms \(\mathbf{D}\mathbf{D}^{-1}=\mathbf{I}_2\). The maximum absolute row sums are \(1\) for \(\mathbf{D}\) and \(100\) for \(\mathbf{D}^{-1}\). Thus \(\kappa(\mathbf{D})=100\).
Here is a particular change in the right-hand side. Let \(\mathbf{b}=(1,\tfrac{1}{100})^{\top}\). Solving gives \(\mathbf{x}=(1,1)^{\top}\), since
Change only the second entry of the right-hand side by \(\tfrac{1}{100}\), so that \(\widetilde{\mathbf{b}}=(1,\tfrac{1}{50})^{\top}\). The new solution is \(\widetilde{\mathbf{x}}=(1,2)^{\top}\), as
The input change is \(\widetilde{\mathbf{b}}-\mathbf{b}=(0,\tfrac{1}{100})^{\top}\). Its infinity norm is \(\tfrac{1}{100}\), while \(\|\mathbf{b}\|_{\infty}=1\), so the relative input change is \(\tfrac{1}{100}\). The solution change is \(\widetilde{\mathbf{x}}-\mathbf{x}=(0,1)^{\top}\). Its infinity norm is \(1\), while \(\|\mathbf{x}\|_{\infty}=1\), so the relative solution change is \(1\). In this example, a relative input change of \(\tfrac{1}{100}\) produces a relative solution change of \(1\), a factor of \(100\).
Check: both displayed matrix-vector products reproduce their stated right-hand sides. The input and solution changes therefore follow from the two exact solutions, rather than from an approximation.
What a Large Value Does—and Does Not—Tell You
The diagonal example shows how a large condition number can arise: a small change in a right-hand-side component corresponding to a small diagonal coefficient can cause a much larger change in the corresponding solution component. More generally, the equation \(\mathbf{x}=\mathbf{A}^{-1}\mathbf{b}\) explains why the inverse is central to sensitivity. A condition number combines the sizes of \(\mathbf{A}\) and \(\mathbf{A}^{-1}\), giving a scale-aware measure rather than looking at either matrix alone.
A large condition number indicates the possibility of substantial sensitivity, not a guarantee that every data change will be amplified by that amount. For instance, a change in \(\mathbf{b}\) may point in a direction that \(\mathbf{A}^{-1}\) does not enlarge much. The condition number describes a worst-case scale across directions. To assess one specific change, compare the actual solutions or examine how that particular change passes through \(\mathbf{A}^{-1}\).
Conditioning and algorithmic accuracy are also different questions. Conditioning concerns the mathematical problem: how solution changes may respond to data changes. Algorithmic accuracy concerns how well a chosen computation solves that problem. A stable calculation cannot remove sensitivity that is inherent in the data-to-solution relationship, and a well-conditioned problem does not by itself guarantee that every computation was carried out accurately.
Worked Example: Rescaling Does Not Improve the Condition Number
Take the triangular matrix \(\mathbf{A}=\begin{bmatrix}1&1\\0&1\end{bmatrix}\) and rescale it to \(\mathbf{B}=5\mathbf{A}=\begin{bmatrix}5&5\\0&5\end{bmatrix}\). Its inverse is \(\mathbf{B}^{-1}=\begin{bmatrix}\tfrac15&-\tfrac15\\0&\tfrac15\end{bmatrix}\). Multiplication verifies this inverse:
The maximum absolute row sum of \(\mathbf{B}\) is \(10\); that of \(\mathbf{B}^{-1}\) is \(\tfrac25\). Hence \(\kappa(\mathbf{B})=10\cdot\tfrac25=4\), the same value as for \(\mathbf{A}\). Scaling makes one norm larger and the other smaller by reciprocal factors; it does not make the system better conditioned by this measure.
Check: the product is \(\mathbf{I}_2\), and the computed norm product is \(4\). This agrees with the scalar-rescaling property.
Common Pitfall: Using Only One of the Two Norms
A common error is to report \(\|\mathbf{A}^{-1}\|\) as the condition number. For the triangular matrix in the first example, this would give \(2\), but the definition requires multiplying by \(\|\mathbf{A}\|_{\infty}=2\), giving \(\kappa(\mathbf{A})=4\). The inverse norm alone measures amplification in absolute size; the product with \(\|\mathbf{A}\|\) makes the measure relative to the scale of the original matrix.
The opposite shortcut—looking only at \(\|\mathbf{A}\|\)—can fail too. For \(\mathbf{C}=100\mathbf{I}_2\), the infinity norms are \(\|\mathbf{C}\|_{\infty}=100\) and \(\|\mathbf{C}^{-1}\|_{\infty}=\tfrac{1}{100}\), so \(\kappa(\mathbf{C})=1\), not \(100\). The inverse product is \(\mathbf{I}_2\), and the factors cancel in the condition number. Always calculate both norms, using the same chosen matrix norm for each.
Check Your Understanding
Use the infinity norm unless a question specifies otherwise.
- For \(\mathbf{A}=\begin{bmatrix}2&0\\0&1\end{bmatrix}\), find \(\mathbf{A}^{-1}\), verify it by multiplication, and compute \(\kappa(\mathbf{A})\).
- For \(\mathbf{D}=\operatorname{diag}(4,\tfrac12)\), use the diagonal-matrix formula to find \(\kappa(\mathbf{D})\).
- Explain why \(\kappa(c\mathbf{A})=\kappa(\mathbf{A})\) for any nonzero real scalar \(c\).
- In the diagonal worked example, state the relative input change and relative solution change for the displayed right-hand sides.
- In one or two sentences, distinguish a large condition number from an inaccurate numerical algorithm.
