Conditional Probabilities in Transition Matrices

From a Conditional Probability to a Matrix Entry

A transition matrix records probabilities of moving from each current state to each possible next state. Its entries are conditional probabilities: the row identifies the state at the current time, and the column identifies the state one step later. This interpretation is simple, but it depends on keeping both the conditioning event and the stated order of the states straight.

Let \((X_n)_{n\ge0}\) be a discrete-time process on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with finite state space \(S\). For states \(i,j\in S\), if \(\mathbb{P}(X_n=i)>0\), the conditional probability of being at \(j\) at the next time, given the current state \(i\), is

For a time-homogeneous Markov chain, this conditional probability does not depend on \(n\). The transition probability from \(i\) to \(j\) is denoted by \(p_{ij}\), and the transition matrix \(P=(p_{ij})\) is defined by

Here \(i\) is the source state and \(j\) is the destination state. Thus, to find \(p_{ij}\), condition on \(X_n=i\) and ask for the probability that \(X_{n+1}=j\). The order of the subscripts matters: \(p_{ij}\) and \(p_{ji}\) generally describe different conditional probabilities.

If transition probabilities depend on time, the entry at time \(n\) is instead written \(p_{ij}(n)=\mathbb{P}(X_{n+1}=j\mid X_n=i)\), and one uses a matrix \(P_n\) for that time. In either case, first identify the current state in the condition and the next state in the event; only then place the probability in the matrix.

Conditional Probabilities Given a State or a History

For a Markov chain, knowing the current state is sufficient for the one-step transition law: additional information about the past does not change that law. More precisely, let \(\mathcal{F}_n^X=\sigma(X_0,\ldots,X_n)\), the information generated by the states up to time \(n\). The Markov property for a homogeneous chain says that, for every \(j\in S\),

The conditional probability given the state \(X_n=i\) is only an ordinary ratio of probabilities when \(\mathbb{P}(X_n=i)>0\). If that event has probability zero, the ratio is undefined. A transition rule for the chain may nevertheless specify a row for state \(i\); that row is part of the model, even if the particular distribution of the process never visits \(i\). The distinction is between a model’s stated transition law and a conditional probability that can be calculated from a particular observed distribution.

Proof. By the Markov property, the conditional expectation of \(\mathbf{1}_{\{X_{n+1}=j\}}\) given \(\mathcal{F}_n^X\) equals \(p_{X_nj}\) almost surely. On \(H\), \(X_n=i\), so \(p_{X_nj}=p_{ij}\). Taking expectations after multiplying by \(\mathbf{1}_H\) gives \(\mathbb{P}(H\cap\{X_{n+1}=j\})=p_{ij}\mathbb{P}(H)\). Divide by the positive number \(\mathbb{P}(H)\) to obtain the claimed conditional probability. \(\square\)

This result explains why a transition row applies not only when conditioning on the current state alone, but also when conditioning on a positive-probability history that ends in that state. It does not say that conditioning on a different state, or on a future state, gives the same probability.

Worked Examples: Filling the Matrix in the Given Order

Probabilities of Sets of Destinations

Sometimes a question asks whether the next state belongs to a set, rather than asking for one particular destination. If the current state is \(i\) and \(A\subseteq S\), the probability of moving into \(A\) is obtained by adding the entries in row \(i\) whose destination states belong to \(A\).

Proof. The event \(\{X_{n+1}\in A\}\) is the disjoint union of the events \(\{X_{n+1}=j\}\) over \(j\in A\). Conditional probability given the positive-probability event \(\{X_n=i\}\) is additive over this finite disjoint union. Each term is \(\mathbb{P}(X_{n+1}=j\mid X_n=i)=p_{ij}\), so their sum is the required conditional probability. If \(A\) is empty, the event \(\{X_{n+1}\in A\}\) is empty and its probability is \(0\); the sum over no states is also \(0\). \(\square\)

For example, in the weather matrix, the probability of Rain or Cloudy tomorrow given Clear today is \(p_{12}+p_{13}=\tfrac13+0=\tfrac13\). This is a sum of entries from one row because the conditioning state remains Clear while the possible destination states are combined.

A Common Reversal: Conditioning on the Wrong Time

A transition entry is not generally a probability of the current state given the next state. The difference is the conditioning event. In the weather example, \(p_{12}=\tfrac13\) means \(\mathbb{P}(X_1=\text{Cloudy}\mid X_0=\text{Clear})=\tfrac13\). It does not mean \(\mathbb{P}(X_0=\text{Cloudy}\mid X_1=\text{Clear})=\tfrac13\).

To see the numerical difference, suppose the weather state at time zero is equally likely to be Clear, Cloudy, or Rain. The joint probability of Cloudy at time zero and Clear at time one is \(\tfrac13\cdot\tfrac14=\tfrac1{12}\). The probability of Clear at time one is \(\tfrac13\cdot\tfrac23+\tfrac13\cdot\tfrac14+\tfrac13\cdot\tfrac16=\tfrac{13}{36}\). Therefore, reversing the conditioning gives \(\mathbb{P}(X_0=\text{Cloudy}\mid X_1=\text{Clear})=(\tfrac1{12})/(\tfrac{13}{36})=\tfrac3{13}\), not \(\tfrac13\). The wrong calculation copies \(p_{12}\) into a question that conditions on the future; the correction is to use the joint and marginal probabilities for that reversed conditional event.

The row-sum checks in the examples are useful consistency checks: each row lists the possible next states conditional on one current state. They do not replace careful indexing. When constructing a matrix, write down the state ordering, fill one row at a time, and read every entry aloud as “probability of the column state next, given the row state now.”