Common Errors in Transition-Matrix Models

A Transition Matrix Is More Than a Table of Numbers

A transition matrix is a compact description of how a discrete-time process moves between states. Errors in its construction can look deceptively plausible: a matrix may contain probabilities but have a row that does not sum to one, or it may be stochastic while its rows and columns are attached to the wrong state labels. Even a correctly normalized matrix can describe the wrong process if its entries reverse the conditioning or if the process is time-dependent or does not have the Markov property.

We use the row-stochastic convention throughout. For a process \((X_n)_{n\ge0}\) on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal F,\mathbb P)\), its history through time \(n\) is \(\mathcal F_n=\sigma(X_0,\ldots,X_n)\). A time-homogeneous Markov chain has a single transition matrix \(P=(p_{ij})\) such that, for every \(n\ge0\), \(i,j\in S\), and history event \(H\in\mathcal F_n\) with \(\mathbb P(H\cap\{X_n=i\})>0\),

\mathbb{P}(X_{n+1}=j\mid H\cap\{X_n=i\})=p_{ij}.

Equivalently, conditional on the history, the distribution of \(X_{n+1}\) depends on that history only through \(X_n\), and the probabilities do not depend on \(n\). Each \(p_{ij}\) is therefore the probability of moving from \(i\) to \(j\): it is conditional on the current state. The initial distribution \(\mu\) is a row vector in the same state order as the rows and columns of \(P\).

Definition: A candidate transition matrix on a finite state space is row-stochastic if \(p_{ij}\ge0\) for every \(i,j\) and \(\sum_{j\in S}p_{ij}=1\) for every \(i\). Nonnegativity and row sums are necessary for a transition matrix, but they do not by themselves establish that a proposed process is Markov or time-homogeneous.

A Basic Validity Check

Proposition: If \(P\) describes the one-step transitions of a finite-state, time-homogeneous Markov chain, then \(P\) is row-stochastic. Conversely, row-stochasticity makes each row a probability distribution for a one-step move, but an additional model assumption is needed to identify those rows with the conditional transitions of a particular process.

Proof. Fix a state \(i\) for which the one-step conditional distribution is being specified. The events \(\{X_{n+1}=j\}\), \(j\in S\), are disjoint and their union is the whole sample space. Conditional on \(X_n=i\), their probabilities are nonnegative and sum to the probability of that whole space, which is \(1\). Thus \(p_{ij}\ge0\) and \(\sum_{j\in S}p_{ij}=1\). Conversely, if those inequalities and equalities hold, the entries in row \(i\) define a probability distribution on the finite set \(S\). This establishes a possible one-step probability rule, but it does not prove that the rule matches the conditional probabilities of a given process or that it remains unchanged over time. \(\square\)

Worked Example: A Row That Does Not Sum to One

On \(S=\{A,B\}\), suppose a proposed matrix is

P_{\mathrm{wrong}}= \begin{bmatrix} \tfrac23 & \tfrac12\\ \tfrac13 & \tfrac12 \end{bmatrix},

with rows and columns ordered \(A,B\). Check whether it can be a transition matrix. The first row sums to \(\tfrac23+\tfrac12=\tfrac76\), while the second sums to \(\tfrac13+\tfrac12=\tfrac56\). Both differ from \(1\), so the matrix is not row-stochastic.

A common wrong calculation is to start in \(A\), use \(\mu=(1,0)\), and compute \(\mu P_{\mathrm{wrong}}=(\tfrac23,\tfrac12)\) as the next-state distribution. Its entries sum to \(\tfrac76\), not \(1\), so it is not a probability distribution. This is a useful diagnostic: multiplying by an invalid matrix does not repair its invalid row.

Suppose the intended probabilities from \(A\) were \(\tfrac23\) to \(A\) and \(\tfrac13\) to \(B\), while those from \(B\) were \(\tfrac12\) to each state. Then the corrected matrix is

P= \begin{bmatrix} \tfrac23 & \tfrac13\\ \tfrac12 & \tfrac12 \end{bmatrix}.

Its row sums are \(\tfrac23+\tfrac13=1\) and \(\tfrac12+\tfrac12=1\). Starting in \(A\), \(\mu P=(1,0)P=(\tfrac23,\tfrac13)\), whose entries sum to \(1\). The correction is valid as a one-step probability rule; whether it models a process still depends on the conditional-transition assumptions.

State Labels and Conditional Direction

A matrix’s numerical entries have no state meaning without an ordering convention. Rows identify current states and columns identify next states; the initial distribution must use that same ordering. Reordering the states is harmless only if the rows, columns, and distribution are all reordered consistently.

Proposition: Let \(s_1,\ldots,s_r\) be an ordering of \(S\), and let \(P\) be the transition matrix in this ordering. If a new ordering is \(s'_a=s_{\sigma(a)}\), where \(\sigma\) is a permutation of \(\{1,\ldots,r\}\), then the same chain has matrix \(P'\) and initial distribution \(\mu'\) given by \(p'_{ab}=p_{\sigma(a),\sigma(b)}\) and \(\mu'_a=\mu_{\sigma(a)}\). This change of ordering does not change any transition probabilities attached to the named states.

Proof. In the new ordering, row \(a\) represents current state \(s'_a=s_{\sigma(a)}\), and column \(b\) represents next state \(s'_b=s_{\sigma(b)}\). The probability of that named transition is therefore the old entry \(p_{\sigma(a),\sigma(b)}\), which gives the stated formula for \(P'\). The probability of starting in \(s'_a\) is the old probability of starting in \(s_{\sigma(a)}\), giving \(\mu'_a=\mu_{\sigma(a)}\). Thus each probability is attached to the same named state or transition as before; only its position in the table changes. \(\square\)

Worked Example: A Mismatched State Order

Let \(S=\{A,B,C\}\), with order \(A,B,C\), and let

P= \begin{bmatrix} \tfrac12 & \tfrac12 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0 \end{bmatrix}.

The row sums are \(\tfrac12+\tfrac12+0=1\), \(0+0+1=1\), and \(1+0+0=1\). In this order, state \(A\) moves to \(A\) or \(B\), each with probability \(\tfrac12\). State \(B\) moves to \(C\), and state \(C\) moves to \(A\).

Now reorder the states as \(B,A,C\). The rows and columns must both be reordered. The \(B\) row, read in columns \(B,A,C\), is \((0,0,1)\); the \(A\) row is \((\tfrac12,\tfrac12,0)\); and the \(C\) row is \((0,1,0)\). Hence

P'= \begin{bmatrix} 0&0&1\\ \tfrac12&\tfrac12&0\\ 0&1&0 \end{bmatrix}.

Its row sums are \(1\), \(1\), and \(1\). Starting in \(A\) has distribution \(\mu'=(0,1,0)\) in the new order, so \(\mu'P'=(\tfrac12,\tfrac12,0)\): the next state is \(B\) or \(A\), each with probability \(\tfrac12\). As a concrete wrong calculation, using \((0,1,0)\) with the old matrix gives \((0,0,1)\), falsely assigning probability \(1\) to \(C\). The calculation multiplies correctly, but the vector and matrix use different state orders.

Another direction error is to treat \(p_{ij}\) as the probability of being in \(i\) now given that the next state is \(j\). That reverses the conditioning. In general, the reversed conditional probability requires a joint probability and the probability of the conditioning event; it is not obtained by simply reading the same matrix entry backward.

Worked Example: Reversing the Conditional Probability

Let \(S=\{A,B\}\), with initial distribution \(\mu=(\tfrac13,\tfrac23)\) and transition matrix, in order \(A,B\),

P= \begin{bmatrix} \tfrac34&\tfrac14\\ \tfrac12&\tfrac12 \end{bmatrix}.

The row sums are \(\tfrac34+\tfrac14=1\) and \(\tfrac12+\tfrac12=1\). The entry \(p_{AB}=\tfrac14\) means \(\mathbb P(X_{n+1}=B\mid X_n=A)=\tfrac14\), not \(\mathbb P(X_n=A\mid X_{n+1}=B)\).

To calculate the latter for one step, first find the probability of next state \(B\):

\mathbb P(X_1=B) =\tfrac13\cdot\tfrac14+\tfrac23\cdot\tfrac12 =\tfrac1{12}+\tfrac13 =\tfrac5{12}.

The joint probability of starting in \(A\) and moving to \(B\) is \(\tfrac13\cdot\tfrac14=\tfrac1{12}\). Since \(\mathbb P(X_1=B)=\tfrac5{12}>0\), the reverse conditional is defined and equals

\mathbb P(X_0=A\mid X_1=B) =\frac{\tfrac1{12}}{\tfrac5{12}} =\tfrac15.

Thus the two conditional probabilities are \(\tfrac14\) and \(\tfrac15\), not the same number. Check the denominator by also calculating \(\mathbb P(X_1=A)=\tfrac13\cdot\tfrac34+\tfrac23\cdot\tfrac12=\tfrac14+\tfrac13=\tfrac7{12}\); the two next-state probabilities sum to \(\tfrac5{12}+\tfrac7{12}=1\).

Time Dependence and Missing State Information

One matrix \(P\) asserts that the one-step conditional probabilities are the same at every time. A process can have the Markov property while its transition probabilities change with time. In that case the transition rule is a sequence of matrices, not one time-homogeneous matrix. Separately, even a time-independent mechanism may fail to be Markov when the recorded state omits information needed to predict the next observation.

Worked Example: Valid Rows, but a Time-Dependent Rule

Let \(S=\{0,1\}\), and define a process on a probability space with \(X_0\) equally likely to be \(0\) or \(1\). Suppose the one-step transition matrix at time \(0\) is \(P_0\), while at time \(1\) it is \(P_1\):

P_0= \begin{bmatrix} \tfrac12&\tfrac12\\ \tfrac12&\tfrac12 \end{bmatrix}, \qquad P_1= \begin{bmatrix} 1&0\\ 0&1 \end{bmatrix}.

Every row of \(P_0\) sums to \(\tfrac12+\tfrac12=1\), and every row of \(P_1\) sums to \(1+0=1\). At time \(0\), \(\mathbb P(X_1=0\mid X_0=0)=\tfrac12\). Both states have probability \(\tfrac12\) at time \(1\), because the rows of \(P_0\) are identical, so the conditional probability at the next step is meaningful for either state. At time \(1\), \(\mathbb P(X_2=0\mid X_1=0)=1\).

The rule is Markov at each step as specified, but it is not time-homogeneous: the probability of a move from \(0\) to \(0\) is \(\tfrac12\) at time \(0\) and \(1\) at time \(1\). No single fixed matrix can represent both conditional probabilities. A row-sum check alone would miss this modeling error.

Worked Example: An Observed Process That Omits Its Phase

Take the probability space \(\Omega=\{0,1,2\}\), with each outcome having probability \(\tfrac13\). Let \(Z_0(\omega)=\omega\), and let \(Z_n\) move deterministically around the cycle \(0\to1\to2\to0\). Its transition matrix, in order \(0,1,2\), is

P_Z= \begin{bmatrix} 0&1&0\\ 0&0&1\\ 1&0&0 \end{bmatrix}.

Each row sums to \(1\). Now record only \(Y_n=\mathbf{1}_{\{Z_n=0\}}\), so \(Y_n\) has state space \(\{0,1\}\). The event \(\{Y_0=1,Y_1=0\}\) occurs exactly when \(Z_0=0\), so it has probability \(\tfrac13\); on that event \(Z_2=2\), hence \(Y_2=0\). Therefore \(\mathbb P(Y_2=1\mid Y_1=0,Y_0=1)=0\).

The event \(\{Y_0=0,Y_1=0\}\) occurs exactly when \(Z_0=1\): then \(Z_1=2\), so both observations are \(0\). Its probability is also \(\tfrac13\), and on it \(Z_2=0\), hence \(Y_2=1\). Thus \(\mathbb P(Y_2=1\mid Y_1=0,Y_0=0)=1\). Both conditioning events have positive probability, and both histories end at \(Y_1=0\), but they give different next-state probabilities. The observed process \(Y_n\) is not Markov. The omitted phase—whether the hidden state is \(1\) or \(2\)—matters for predicting the next observation.

A Practical Audit

When checking a transition-matrix model, separate the numerical checks from the modeling checks. First verify that every entry is nonnegative and every row sums to one. Then confirm that row \(i\) really describes transitions out of state \(i\), and that column \(j\) describes transitions into state \(j\). Check that the initial row vector uses exactly the same ordering. Finally ask whether the stated probabilities condition on the current state, whether they are constant across time, and whether the recorded state contains enough information for the Markov property.

Common mistake. A matrix with nonnegative entries and rows summing to one is not automatically the correct matrix for a given process. For example, the matrices \(P_0\) and \(P_1\) in the time-dependent example both pass the row-sum check, but their different \(0\)-to-\(0\) probabilities rule out a single time-homogeneous matrix. Likewise, rearranging the initial distribution without rearranging the matrix can produce a normalized result that still assigns probabilities to the wrong named states. Every numerical check must be paired with a check of what the entries mean.

Check Your Understanding

For each question, identify which modeling check is relevant before carrying out any calculation.

A proposed two-state matrix has rows \((\tfrac25,\tfrac35)\) and \((\tfrac14,\tfrac12)\). Is it row-stochastic? What must be checked or corrected?
In a matrix ordered \(A,B,C\), what conditional probability does the entry \(p_{BC}\) represent?
A chain is reordered from \(A,B,C\) to \(C,A,B\). Which parts of the model must be reordered along with the state labels?
At two different times, the conditional probability of moving from state \(i\) to state \(j\) is \(\tfrac13\) and \(\tfrac23\), and both conditioning events have positive probability. Can one time-homogeneous transition matrix describe these rules?
Two positive-probability histories end in the same observed state but give different conditional probabilities for the next observation. What does this imply about the Markov property of the observed process?