Initial Distributions and Fixed Starting States

Two Ways to Specify Where a Chain Starts

A Markov chain has two kinds of starting information that are easy to confuse. Its transition matrix describes the rules for moving between states; its initial distribution describes where the process is at time zero. The starting state may be random, or it may be fixed in advance. Both cases fit the same row-vector notation, which makes their relationship precise.

Throughout, take a finite nonempty state space $S=\{s_1,\ldots,s_m\}$ in a specified order, and a discrete-time process $(X_n)_{n\ge0}$ on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. When a filtration is needed, use the natural filtration $\mathcal{F}_n=\sigma(X_0,\ldots,X_n)$. The initial distribution concerns $X_0$ alone. The Markov property and any transition matrix describe the process's transition behavior; they do not, by themselves, say how $X_0$ is chosen.

Definition: The initial distribution of $X_0$, in the ordering $s_1,\ldots,s_m$, is the row vector $\mu=(\mu_1,\ldots,\mu_m)$, where $\mu_r=\mathbb{P}(X_0=s_r)$. It is a probability vector: each $\mu_r\ge0$ and $\sum_{r=1}^m\mu_r=1$. If the chain starts at a fixed state $s_k$, its initial distribution is the point mass $\delta_{s_k}$, whose $k$th coordinate is $1$ and whose other coordinates are $0$.

The ordering of the states matters because it determines which state each coordinate represents. For example, if the ordering is $(a,b,c)$, then $(\tfrac12,\tfrac13,\tfrac16)$ means probabilities $\tfrac12,\tfrac13,\tfrac16$ for $a,b,c$, respectively. It is a row vector by convention. The point mass at $b$ in this ordering is $(0,1,0)$.

When the starting state is fixed at $i$, write $\mathbb{P}_i$ and $\mathbb{E}_i$ for probabilities and expectations under that starting condition. In vector notation, this means the initial distribution is $\delta_i$. For a random start with initial distribution $\mu$, a quantity involving $X_0$ can be computed by weighting its value at each state by $\mu$. The examples below use only this initial-state calculation; they do not use the transition matrix to predict later states.

Basic Facts About Initial Laws

Proposition: For a random variable $X_0$ taking values in a finite nonempty state space $S=\{s_1,\ldots,s_m\}$, the vector with coordinates $\mu_r=\mathbb{P}(X_0=s_r)$ is a probability vector. Conversely, every probability vector $\mu$ on $S$ specifies a possible law for $X_0$. A fixed start at $s_k$ has distribution $\delta_{s_k}$.

Proof. The events $\{X_0=s_r\}$, for $1\le r\le m$, are disjoint and their union is the whole sample space, because $X_0$ takes a value in $S$. Each probability $\mathbb{P}(X_0=s_r)$ is nonnegative, and finite additivity over this partition gives $\sum_{r=1}^m\mathbb{P}(X_0=s_r)=1$. Thus $\mu$ is a probability vector. Conversely, given nonnegative numbers $\mu_1,\ldots,\mu_m$ summing to $1$, one can choose $X_0=s_r$ with probability $\mu_r$; these probabilities define a distribution on the finite set. If $X_0=s_k$ with probability $1$, its coordinate probabilities are $1$ at $s_k$ and $0$ elsewhere, which is $\delta_{s_k}$. $\square$

Proposition: For any real-valued function $f$ on the finite state space, if $X_0$ has initial distribution $\mu$, then $$ \mathbb{E}[f(X_0)]=\sum_{r=1}^m\mu_r f(s_r). $$ In particular, for any subset $A\subseteq S$, $\mathbb{P}(X_0\in A)=\sum_{s_r\in A}\mu_r$.

Proof. Since $X_0$ takes exactly one of the states, $f(X_0)=\sum_{r=1}^m f(s_r)\mathbf{1}_{\{X_0=s_r\}}$. Taking expectations and using linearity for this finite sum gives $\mathbb{E}[f(X_0)]=\sum_{r=1}^m f(s_r)\mathbb{P}(X_0=s_r)=\sum_{r=1}^m\mu_r f(s_r)$. For the event $X_0\in A$, its indicator is $\sum_{s_r\in A}\mathbf{1}_{\{X_0=s_r\}}$, since the state events are disjoint. Taking expectations gives the stated probability formula. If $A$ is empty, the sum is empty and equals $0$, as required. $\square$

These facts do not require a Markov property or a transition matrix. They apply to any random variable on a finite state space. A transition matrix becomes relevant when describing how the process evolves after its initial state is specified.

Worked Example: A Random Start on Three States

Let $S=(a,b,c)$, and suppose $X_0$ has probabilities $\tfrac12,\tfrac13,\tfrac16$ at $a,b,c$, respectively. Record its initial distribution, find the probability that $X_0\in\{a,c\}$, and compute $\mathbb{E}[f(X_0)]$ for $f(a)=2,\ f(b)=0,\ f(c)=5$.

The vector in the stated ordering is $\mu=(\tfrac12,\tfrac13,\tfrac16)$. Check that its coordinates are nonnegative and sum to one: $$ \tfrac12+\tfrac13+\tfrac16=\tfrac36+\tfrac26+\tfrac16=1. $$ The event contains states $a$ and $c$, so its probability is the sum of those two coordinates: $$ \mathbb{P}(X_0\in\{a,c\})=\tfrac12+\tfrac16=\tfrac23. $$ For the expectation, weight each function value by its state's probability: $$ \mathbb{E}[f(X_0)] =\tfrac12(2)+\tfrac13(0)+\tfrac16(5) =1+0+\tfrac56 =\tfrac{11}{6}. $$

Check: The same expectation follows by listing the possible values of $f(X_0)$: the values $2,0,5$ occur with probabilities $\tfrac12,\tfrac13,\tfrac16$, so the weighted sum is again $\tfrac{11}{6}$. The initial probabilities also assign probability $\tfrac13$ to $b$, and $\tfrac12+\tfrac16+\tfrac13=1$.

Worked Example: A Fixed Starting State

Use the ordering $S=(1,2,3)$. Suppose the chain starts at state $2$ with probability $1$. Give its initial vector, and compute the probability that $X_0\in\{1,2\}$ and the expectation of $g(X_0)$, where $g(1)=4,\ g(2)=-1,\ g(3)=7$.

A fixed start at state $2$ is represented by the point mass $$ \mu=\delta_2=(0,1,0). $$ Its entries are nonnegative and $0+1+0=1$, so it is a probability vector. The probability of the stated event is the sum of the coordinates for states $1$ and $2$: $$ \mathbb{P}(X_0\in\{1,2\})=0+1=1. $$ The function is evaluated at the only state with positive initial probability: $$ \mathbb{E}[g(X_0)]=0(4)+1(-1)+0(7)=-1. $$

Check: Since $\mathbb{P}(X_0=2)=1$, $g(X_0)=-1$ with probability $1$; its expectation is therefore $-1$, matching the vector calculation. The other two state values contribute zero because their initial probabilities are zero.

Combining Possible Sources of a Starting State

A random initial state can itself be chosen in stages. For instance, a population might first be assigned to one of several groups, and the starting-state distribution might depend on the group. The overall initial distribution is obtained by weighting each group's distribution by the probability of selecting that group.

Theorem (Mixture of initial distributions): Suppose a finite label $Z$ takes values $z_1,\ldots,z_\ell$ with probabilities $\alpha_1,\ldots,\alpha_\ell$, where $\alpha_q\ge0$ and $\sum_{q=1}^{\ell}\alpha_q=1$. Suppose that, for each label with $\alpha_q>0$, the distribution of $X_0$ given $Z=z_q$ is the probability vector $\nu^{(q)}$ on $S$. Then the unconditional initial distribution is $$ \mu=\sum_{q=1}^{\ell}\alpha_q\nu^{(q)}. $$ Labels with probability zero contribute nothing and need no conditional distribution to be specified.

Proof. Fix a state $s_r$. The events $\{Z=z_q\}$ with positive probability partition the possibilities for the label. By the law of total probability, $$ \mathbb{P}(X_0=s_r) =\sum_{q:\,\alpha_q>0}\mathbb{P}(Z=z_q)\mathbb{P}(X_0=s_r\mid Z=z_q) =\sum_{q=1}^{\ell}\alpha_q\nu_r^{(q)}, $$ where a zero-weight term is taken to be zero. This is exactly the $r$th coordinate of $\sum_q\alpha_q\nu^{(q)}$, for every state $s_r$, proving the vector identity. Each coordinate of the mixture is nonnegative. Its coordinates sum to $\sum_q\alpha_q\sum_r\nu_r^{(q)}=\sum_q\alpha_q=1$, so the result is a probability vector as well. $\square$

Worked Example: Choosing Between Two Starting Populations

Let $S=(1,2,3)$. A source $A$ is selected with probability $\tfrac34$ and has initial distribution $\nu^{(A)}=(\tfrac12,\tfrac12,0)$. Source $B$ is selected with probability $\tfrac14$ and has initial distribution $\nu^{(B)}=(0,\tfrac13,\tfrac23)$. Find the overall initial distribution and the probability that $X_0\in\{2,3\}$.

First check the source probabilities: $\tfrac34+\tfrac14=1$. Each source distribution is valid: $$ \tfrac12+\tfrac12+0=1,\qquad 0+\tfrac13+\tfrac23=1. $$ Weight each source vector by its selection probability and add coordinate by coordinate: $$ \begin{aligned} \mu &=\tfrac34\bigl(\tfrac12,\tfrac12,0\bigr) +\tfrac14\bigl(0,\tfrac13,\tfrac23\bigr)\\ &=\bigl(\tfrac38,\tfrac38,0\bigr) +\bigl(0,\tfrac1{12},\tfrac16\bigr)\\ &=\bigl(\tfrac38,\tfrac{11}{24},\tfrac16\bigr). \end{aligned} $$ In the second coordinate, $\tfrac38+\tfrac1{12}=\tfrac9{24}+\tfrac2{24}=\tfrac{11}{24}$. The resulting vector sums to $$ \tfrac38+\tfrac{11}{24}+\tfrac16 =\tfrac9{24}+\tfrac{11}{24}+\tfrac4{24} =1. $$ Therefore the probability of states $2$ or $3$ is $$ \mathbb{P}(X_0\in\{2,3\})=\tfrac{11}{24}+\tfrac16=\tfrac{15}{24}=\tfrac58. $$

Check by the source calculation: In source $A$, the probability of states $2$ or $3$ is $\tfrac12+0=\tfrac12$, contributing $\tfrac34\cdot\tfrac12=\tfrac38$. In source $B$, that probability is $\tfrac13+\tfrac23=1$, contributing $\tfrac14\cdot1=\tfrac14$. The total is $\tfrac38+\tfrac14=\tfrac38+\tfrac28=\tfrac58$, as before.

Keep the Initial Distribution Separate from the Transition Matrix

A transition matrix and an initial distribution answer different questions. In the row-stochastic convention, a transition matrix $P=(p_{ij})$ records one-step transition probabilities, with each row summing to $1$. The initial distribution $\mu$ records probabilities at time zero. Specifying one does not generally specify the other: a chain may use the same transition rules with different starting distributions, or the same initial distribution with different transition rules.

For example, consider the ordered state space $(1,2,3)$ and the matrix $$ P= \begin{bmatrix} \tfrac12&\tfrac12&0\\ \tfrac14&\tfrac12&\tfrac14\\ 0&\tfrac13&\tfrac23 \end{bmatrix}. $$ Before using it as a transition matrix, check its rows: $\tfrac12+\tfrac12+0=1$, $\tfrac14+\tfrac12+\tfrac14=1$, and $0+\tfrac13+\tfrac23=1$. This matrix does not tell us whether the chain starts from $(1,0,0)$, $(0,1,0)$, or a random initial vector such as $(\tfrac12,\tfrac13,\tfrac16)$. Those are separate choices. Calculating the distribution at a later time requires combining initial information with transition information, a distinct task from identifying the initial law.

Common mistake. Suppose the ordered states are $(a,b,c)$ and $\mu=(\tfrac12,\tfrac13,\tfrac16)$. A coordinate-order error might assign $\tfrac13$ to $c$ and then calculate $\mathbb{P}(X_0\in\{a,c\})=\tfrac12+\tfrac13=\tfrac56$. The second coordinate belongs to $b$, not $c$. The correct calculation uses the first and third coordinates: $$ \mathbb{P}(X_0\in\{a,c\})=\tfrac12+\tfrac16=\tfrac23. $$ Always write down the state ordering before interpreting coordinates. A related check is to verify that all coordinates are nonnegative and sum to $1$; passing that check does not, by itself, fix a state-ordering error.

Check Your Understanding

Use the stated state ordering in each question, and check that every proposed initial vector is a probability vector.

For $S=(u,v,w)$ and $\mu=(\tfrac14,\tfrac12,\tfrac14)$, find $\mathbb{P}(X_0\in\{u,w\})$.
Write the point-mass vector for a fixed start at state $3$ when the ordering is $(1,2,3,4)$.
If $S=(a,b)$, $\mathbb{P}(X_0=a)=\tfrac25$, and $f(a)=3,\ f(b)=8$, compute $\mathbb{E}[f(X_0)]$.
Two equally likely sources have initial vectors $(1,0)$ and $(0,\tfrac12,\tfrac12)$. Is this a valid mixture as stated? Identify any issue before combining vectors.
In one sentence, distinguish the information in an initial distribution from the information in a transition matrix.

Initial Distributions and Fixed Starting States

What You'll Learn