From Transition Matrices to Long-Run Probabilities
A three-state Markov model updates a probability distribution by multiplying it by a transition matrix. If the matrix has a dominant eigenvalue of \(1\), the corresponding probability eigenvector can describe the distribution that remains unchanged under every update. When the other eigenvalues have magnitude less than \(1\), their contributions decay under repeated multiplication, so the dominant eigenvector also predicts the long-run probabilities. The calculations below show how to identify and check this behavior in explicit positive models.
We use column probability vectors: \(\mathbf{x}=(x_1,x_2,x_3)^{\top}\), where \(x_i\) is the probability of being in state \(i\), \(x_i\geq 0\), and \(x_1+x_2+x_3=1\). A transition matrix \(\mathbf{P}\) is column-stochastic if its entries are nonnegative and each column sums to \(1\). Its entry \(p_{ij}\) is the probability of moving from state \(j\) to state \(i\). It is positive if every entry is strictly greater than \(0\). The next state distribution is \(\mathbf{x}_{k+1}=\mathbf{P}\mathbf{x}_k\).
Column-stochasticity guarantees that probability vectors remain probability vectors: for any \(\mathbf{x}\), the sum of the entries of \(\mathbf{P}\mathbf{x}\) is the sum of the entries of \(\mathbf{x}\), because each column of \(\mathbf{P}\) sums to \(1\). Nonnegative entries also make each new coordinate nonnegative whenever \(\mathbf{x}\) has nonnegative coordinates. A stationary distribution, if found, is therefore a valid probability state that remains unchanged. Whether iteration converges to it depends on the other eigenvalues.
Why a Dominant Stationary Eigenvector Predicts a Limit
Proof. Since \(\mathbf{q},\mathbf{v}_2,\mathbf{v}_3\) form a basis, write \(\mathbf{x}_0=a\mathbf{q}+b\mathbf{v}_2+c\mathbf{v}_3\) for scalars \(a,b,c\). Let \(\mathbf{1}=(1,1,1)^{\top}\). Column-stochasticity gives \(\mathbf{1}^{\top}\mathbf{P}=\mathbf{1}^{\top}\). For either \(i=2\) or \(i=3\), \(\mathbf{1}^{\top}\mathbf{P}\mathbf{v}_i=\mathbf{1}^{\top}\mathbf{v}_i\), while the eigenvector equation gives \(\mathbf{1}^{\top}\mathbf{P}\mathbf{v}_i=\lambda_i\mathbf{1}^{\top}\mathbf{v}_i\). Thus \((1-\lambda_i)\mathbf{1}^{\top}\mathbf{v}_i=0\). Since \(|\lambda_i|<1\), we have \(\lambda_i\ne1\), so \(\mathbf{1}^{\top}\mathbf{v}_i=0\). Now \(\mathbf{1}^{\top}\mathbf{x}_0=1\), \(\mathbf{1}^{\top}\mathbf{q}=1\), and the expansion of \(\mathbf{x}_0\) implies \(a=1\). Applying \(\mathbf{P}\) repeatedly therefore gives \(\mathbf{P}^k\mathbf{x}_0=\mathbf{q}+b\lambda_2^k\mathbf{v}_2+c\lambda_3^k\mathbf{v}_3\). Both powers tend to \(0\), so the iterates tend to \(\mathbf{q}\). For uniqueness, any other stationary distribution has an expansion \(a\mathbf{q}+b\mathbf{v}_2+c\mathbf{v}_3\); stationarity forces \(b=\lambda_2b\) and \(c=\lambda_3c\), hence \(b=c=0\). Its entries sum to \(1\), so \(a=1\), and it equals \(\mathbf{q}\). \(\square\)
This proposition gives a precise condition to check: the stationary vector must be part of an eigenvector basis, and the other eigenvalues must have magnitude less than \(1\). In the examples, we verify those conditions directly. Positivity makes every transition possible in one step, but the calculations—not positivity alone—will establish the eigenvalues and the convergence conclusion for each particular matrix.
A Positive Model with Three Distinct Eigenvalues
Worked Example: A Uniform Stationary Distribution
Consider the positive column-stochastic matrix \(\mathbf{P}=\frac{1}{48}\begin{bmatrix}23&11&14\\11&23&14\\14&14&20\end{bmatrix}\). Find its stationary distribution, determine the long-run probabilities from \(\mathbf{x}_0=(1,0,0)^{\top}\), and check the first two iterations.
Every entry is positive. The column sums are \(23+11+14=48\), \(11+23+14=48\), and \(14+14+20=48\), so \(\mathbf{P}\) is column-stochastic. Use the vectors \(\mathbf{e}=(1,1,1)^{\top}\), \(\mathbf{v}=(1,-1,0)^{\top}\), and \(\mathbf{w}=(1,1,-2)^{\top}\). Direct multiplication gives \(\mathbf{P}\mathbf{e}=\mathbf{e}\), \(\mathbf{P}\mathbf{v}=\frac14\mathbf{v}\), and \(\mathbf{P}\mathbf{w}=\frac18\mathbf{w}\). For example, the first of these follows from the row sums of the displayed matrix, each of which is \(48\); for the second, the numerator matrix sends \(\mathbf{v}\) to \((12,-12,0)^{\top}\); for the third, it sends \(\mathbf{w}\) to \((6,6,-12)^{\top}\).
These vectors are independent: if \(a\mathbf{e}+b\mathbf{v}+c\mathbf{w}=\mathbf{0}\), subtracting the second coordinate equation from the first gives \(2b=0\). The first coordinate then gives \(a+c=0\), while the third gives \(a-2c=0\), so \(a=c=0\) as well. They are consequently an eigenvector basis. The stationary distribution is \(\mathbf{q}=\frac13\mathbf{e}\), and the other eigenvalues have magnitude less than \(1\), so the proposition shows that every probability state converges to \(\mathbf{q}\).
To find the iterates from the specified initial state, first express it in the eigenvector basis: \(\mathbf{x}_0=\frac13\mathbf{e}+\frac12\mathbf{v}+\frac16\mathbf{w}\). Checking coordinates gives \((\frac13+\frac12+\frac16,\frac13-\frac12+\frac16,\frac13-\frac{2}{6})=(1,0,0)\). Thus \(\mathbf{x}_k=\frac13\mathbf{e}+\frac12(\frac14)^k\mathbf{v}+\frac16(\frac18)^k\mathbf{w}\).
For \(k=1\), this gives \(\mathbf{x}_1=(\frac{23}{48},\frac{11}{48},\frac{14}{48})^{\top}\). Directly, \(\mathbf{P}\mathbf{x}_0\) is the first column of \(\mathbf{P}\), which is exactly that vector. For \(k=2\), the eigenvector formula gives \(\mathbf{x}_2=(\frac{47}{128},\frac{39}{128},\frac{21}{64})^{\top}\). Multiplying the first iterate back confirms it: \[ \mathbf{P}\mathbf{x}_1 =\frac{1}{2304} \begin{bmatrix} 23(23)+11(11)+14(14)\\ 11(23)+23(11)+14(14)\\ 14(23)+14(11)+20(14) \end{bmatrix} =\frac{1}{2304}\begin{bmatrix}846\\702\\756\end{bmatrix} =\begin{bmatrix}47/128\\39/128\\21/64\end{bmatrix}. \] The decaying eigenvector terms explain why these iterates approach the uniform distribution.
A Nonuniform Stationary Distribution
Worked Example: A Repeated Subdominant Eigenvalue
Let \(\mathbf{q}=(\frac12,\frac13,\frac16)^{\top}\) and define \(\mathbf{P}=\frac14\mathbf{I}_3+\frac34\mathbf{q}\mathbf{1}^{\top} =\begin{bmatrix}5/8&3/8&3/8\\1/4&1/2&1/4\\1/8&1/8&3/8\end{bmatrix}\). All entries are positive and each column sums to \(1\). Find the long-run state from \(\mathbf{x}_0=(0,1,0)^{\top}\).
Since \(\mathbf{1}^{\top}\mathbf{q}=1\), multiplication gives \(\mathbf{P}\mathbf{q}=\frac14\mathbf{q}+\frac34\mathbf{q}(\mathbf{1}^{\top}\mathbf{q})=\mathbf{q}\). More generally, for any vector \(\mathbf{x}\), \(\mathbf{P}\mathbf{x}=\frac14\mathbf{x}+\frac34\mathbf{q}(\mathbf{1}^{\top}\mathbf{x})\). If \(\mathbf{x}\) has entries summing to zero, then \(\mathbf{P}\mathbf{x}=\frac14\mathbf{x}\). The vectors \((1,-1,0)^{\top}\) and \((1,0,-1)^{\top}\) are independent and both have entries summing to zero. Together with \(\mathbf{q}\), they form a basis: the first two span the vectors with coordinate sum zero, whereas \(\mathbf{q}\) has coordinate sum \(1\). Thus the eigenvalues in this basis are \(1,\frac14,\frac14\), and the proposition applies despite the repeated subdominant eigenvalue.
For a probability vector \(\mathbf{x}\), the update formula reduces to \(\mathbf{P}\mathbf{x}=\mathbf{q}+\frac14(\mathbf{x}-\mathbf{q})\). Repeating this identity yields \(\mathbf{x}_k=\mathbf{q}+(\frac14)^k(\mathbf{x}_0-\mathbf{q})\). In particular, \(\mathbf{x}_1=(\frac38,\frac12,\frac18)^{\top}\), which is the second column of \(\mathbf{P}\). The next iterate is \(\mathbf{x}_2=(\frac{15}{32},\frac38,\frac{5}{32})^{\top}\).
Check: Direct multiplication gives \[ \mathbf{P}\mathbf{x}_1= \begin{bmatrix} (5/8)(3/8)+(3/8)(1/2)+(3/8)(1/8)\\ (1/4)(3/8)+(1/2)(1/2)+(1/4)(1/8)\\ (1/8)(3/8)+(1/8)(1/2)+(3/8)(1/8) \end{bmatrix} = \begin{bmatrix}15/32\\3/8\\5/32\end{bmatrix}. \] Each iterate stays a probability vector, and the deviation from \(\mathbf{q}\) is multiplied by \(\frac14\) at every step. Hence the long-run probabilities are \((\frac12,\frac13,\frac16)^{\top}\), not the uniform distribution.
Finding the Stationary Distribution by the Eigenvectors
Worked Example: Three Distinct Modes with Unequal State Probabilities
Consider \(\mathbf{P}=\begin{bmatrix}1/2&1/6&1/4\\1/6&1/2&1/4\\1/3&1/3&1/2\end{bmatrix}\). Its entries are positive, and its columns each sum to \(1\). Find the stationary distribution and predict the long-run state from \(\mathbf{x}_0=(0,0,1)^{\top}\).
Let \(\mathbf{q}=(3/10,3/10,2/5)^{\top}\), \(\mathbf{v}=(1,-1,0)^{\top}\), and \(\mathbf{w}=(1,1,-2)^{\top}\). Direct multiplication verifies \(\mathbf{P}\mathbf{q}=\mathbf{q}\): the coordinates are \(3/20+1/20+1/10=3/10\), \(1/20+3/20+1/10=3/10\), and \(1/10+1/10+1/5=2/5\). It also gives \(\mathbf{P}\mathbf{v}=\frac13\mathbf{v}\) and \(\mathbf{P}\mathbf{w}=\frac16\mathbf{w}\). The vectors are independent: \(a\mathbf{q}+b\mathbf{v}+c\mathbf{w}=\mathbf{0}\) implies, by subtracting the first two coordinate equations, \(2b=0\); the remaining equations are \(3a/10+c=0\) and \(2a/5-2c=0\), which force \(a=c=0\). Thus the eigenvalues are \(1,\frac13,\frac16\), with \(\mathbf{q}\) the stationary distribution and the other two eigenvalues strictly smaller than \(1\) in magnitude.
The initial state has the decomposition \(\mathbf{x}_0=\mathbf{q}-\frac{3}{10}\mathbf{w}\), as its coordinates are \(3/10-3/10=0\), \(3/10-3/10=0\), and \(2/5+6/10=1\). Therefore \(\mathbf{x}_k=\mathbf{q}-\frac{3}{10}(\frac16)^k\mathbf{w}\), which tends to \(\mathbf{q}\). For \(k=1\), this predicts \(\mathbf{x}_1=(1/4,1/4,1/2)^{\top}\), the third column of \(\mathbf{P}\). For \(k=2\), it predicts \(\mathbf{x}_2=(7/24,7/24,5/12)^{\top}\). Direct multiplication checks the second update: \[ \mathbf{P}\mathbf{x}_1= \begin{bmatrix} 1/8+1/24+1/8\\ 1/24+1/8+1/8\\ 1/12+1/12+1/4 \end{bmatrix} = \begin{bmatrix}7/24\\7/24\\5/12\end{bmatrix}. \] The limiting probability of state \(3\) is \(2/5\), while each of states \(1\) and \(2\) has limiting probability \(3/10\).
Common Pitfall: Multiplying in the Wrong Direction
With column probability states, the update is \(\mathbf{x}_{k+1}=\mathbf{P}\mathbf{x}_k\). Using the third example and \(\mathbf{x}_0=(0,0,1)^{\top}\), this gives \(\mathbf{P}\mathbf{x}_0=(1/4,1/4,1/2)^{\top}\), the third column of \(\mathbf{P}\). A wrong calculation is to multiply \(\mathbf{x}_0^{\top}\mathbf{P}\), which returns the third row, \((1/3,1/3,1/2)\). Those entries sum to \(7/6\), so they cannot be probabilities. The correction is to keep the state as a column and multiply on the left by the transition matrix.
A second error is to assume that any eigenvector for eigenvalue \(1\) is automatically a stationary distribution. It must also be scaled so its coordinates sum to \(1\), and it must have nonnegative coordinates. In the third example, \((3,3,4)^{\top}\) is an eigenvector, but the probability vector is obtained by dividing by its coordinate sum, \(10\), to get \((3/10,3/10,2/5)^{\top}\).
Interpreting the Eigenvalue Calculation
The examples illustrate a useful sequence of checks. First confirm that the transition matrix is column-stochastic and positive. Next find an eigenvector for eigenvalue \(1\), and normalize it to make a probability vector. Then identify a basis of the other eigenmodes and verify that their eigenvalues have magnitude less than \(1\). Each of those modes is multiplied by its eigenvalue at every step, so its effect shrinks under repeated iteration. A mode with a smaller eigenvalue magnitude shrinks faster; a negative eigenvalue also alternates its sign while shrinking.
When the nonstationary eigenvalues have magnitude below \(1\), the stationary eigenvector determines the limiting distribution, provided the eigenvectors give a basis as in the proposition. Direct iteration is a valuable check: it confirms the matrix convention and catches arithmetic mistakes, but a few iterates alone do not prove convergence. The eigenvalue calculation explains why the prediction holds for all later steps.
Check Your Understanding
Use column probability vectors throughout, and verify numerical answers by multiplication.
- For the first example, calculate \(\mathbf{x}_3\) from its eigenvector formula and state the limiting distribution.
- In the second example, explain why the repeated eigenvalue \(1/4\) still allows convergence to the stated stationary distribution.
- For the third example, verify that the entries of \(\mathbf{q}\) sum to \(1\) and that \(\mathbf{P}\mathbf{q}=\mathbf{q}\).
- Why does an eigenvector of eigenvalue \(1\) need an additional normalization before it can represent probabilities?
- For a column-stochastic matrix, which column gives the next state when the current state is \((0,1,0)^{\top}\), and why?
