Probability Flow Balance in Each State

Why Balance Probability State by State?

A forward equation describes how the probability distribution over a continuous-time Markov chain changes. The total probability across all states remains one, but that fact alone does not tell us whether probability is moving into or out of a particular state. For that, construct a balance for each state: probability arrives along incoming transitions and departs along outgoing transitions.

This viewpoint is useful both for writing equations from a transition diagram and for checking a proposed system of equations. The key is to attach each transition rate to the probability of occupying its source state, not its destination. Rates are typically measured per unit time, so each product of a state probability and a transition intensity is a probability flow per unit time.

Definition: Consider a finite-state continuous-time Markov chain with states $1,\ldots,n$. Let $p_j(t)=P(X(t)=j)$ be the probability of occupying state $j$ at time $t$, and let $q_{ij}(t)$ be the transition intensity from state $i$ to a different state $j$. The total rate out of state $j$ is $q_j(t)=\sum_{k\ne j}q_{jk}(t)$. The state-by-state forward balance is $$ p_j'(t)=\sum_{i\ne j}p_i(t)q_{ij}(t)-p_j(t)\sum_{k\ne j}q_{jk}(t). $$ The first term is total incoming probability flow; the second is total outgoing probability flow.

This formula assumes a Markov model with transition intensities defined over the time interval of interest. In a small interval of length $h$, conditional on being in state $i$ at time $t$, the probability of a single transition to $j\ne i$ is $q_{ij}(t)h+o(h)$. With finite, locally well-behaved intensities, the probability of two or more transitions in that interval is of smaller order than $h$. These small-interval probabilities are the basis for the balance equation.

Use a directed arrow $i\to j$ in a transition diagram to represent an allowed transition with intensity $q_{ij}(t)$. An absent arrow means that the corresponding direct transition is not allowed in the model. For a state $j$, include every arrow ending at $j$ in its incoming flow and every arrow starting at $j$ in its outgoing flow. If a state has no outgoing arrows, its outgoing flow is zero; such a state is absorbing.

From a Diagram to the Forward Equations

A reliable procedure is to write one balance for each state directly from the arrows. This avoids the need to guess signs or rely on a remembered matrix entry. The state probabilities $p_i(t)$ are occupancy probabilities at the source of each arrow; the intensities describe the rate of leaving that source for the destination.

List the incoming arrows.
For the state being balanced, record each arrow that ends there and multiply its intensity by the probability of occupying its source state.

List the outgoing arrows.
Record each arrow that starts at the state and multiply its intensity by the probability of occupying that state.

Subtract outgoing flow from incoming flow.
The result is the derivative of that state’s occupancy probability at the time in question.

Check the full system.
When the state probabilities sum to one, the derivatives should sum to zero: every departure from one state is an arrival in another.

For a time-homogeneous model, the intensities $q_{ij}$ are constant and the same balance applies at every time. For a time-inhomogeneous model, use the intensities at the current time $t$. In matrix form, with a row vector of probabilities $p(t)$ and generator $Q(t)$, the equations are $p'(t)=p(t)Q(t)$. The state-by-state balances are the component form of this forward equation.

Worked Example: A Three-State Flow Balance

A hypothetical process has states $A,B,C$, with arrows $A\to B$ at rate $0.4$ per year, $A\to C$ at rate $0.1$ per year, and $B\to C$ at rate $0.3$ per year. State $C$ is absorbing. At time $t$, suppose the occupancy probabilities are $p_A(t)=0.50$, $p_B(t)=0.30$, and $p_C(t)=0.20$. Find the derivatives of all three probabilities.

Method. Balance the incoming and outgoing flows separately for each state. The probabilities and rates are specified at the same time $t$, and all rates are per year.

For $A$, there are no incoming arrows. Its two outgoing arrows have total intensity $0.4+0.1=0.5$ per year, so

p_A'(t)=-(0.50)(0.4+0.1)=-0.25\text{ per year}.

For $B$, the incoming flow is from $A$, while the outgoing flow is to $C$:

p_B'(t)=(0.50)(0.4)-(0.30)(0.3)=0.20-0.09=0.11\text{ per year}.

For $C$, there are two incoming arrows and no outgoing arrows:

p_C'(t)=(0.50)(0.1)+(0.30)(0.3)=0.05+0.09=0.14\text{ per year}.

The derivatives sum to $-0.25+0.11+0.14=0$, as required. Thus probability is leaving $A$, while the net changes in $B$ and $C$ are positive. Common mistake avoided: the $A\to B$ flow is $p_A(0.4)$, not $p_B(0.4)$; the intensity is weighted by occupancy of the arrow’s source.

Why the Balance Formula Holds

The formula follows by considering the possible ways to be in a chosen state $j$ a short time later. Over a short interval, probability already in $j$ can remain there, while probability in another state can arrive through a direct transition. Dividing the resulting change by the interval length gives the derivative.

More precisely, let $h>0$ be small. Conditional on occupying $j$ at time $t$, the probability of leaving $j$ during $[t,t+h]$ is $q_j(t)h+o(h)$. Conditional on occupying $i\ne j$, the probability of a direct transition from $i$ to $j$ is $q_{ij}(t)h+o(h)$. Therefore, to first order in $h$,

p_j(t+h)=p_j(t)\{1-q_j(t)h\}+\sum_{i\ne j}p_i(t)q_{ij}(t)h+o(h).

Subtract $p_j(t)$, divide by $h$, and let $h$ decrease to zero. The terms $o(h)/h$ tend to zero, leaving

p_j'(t)=-p_j(t)q_j(t)+\sum_{i\ne j}p_i(t)q_{ij}(t).

This is incoming flow minus outgoing flow. The argument is a local one: it yields the instantaneous derivative from the current probabilities and intensities, rather than directly giving probabilities at a later time.

Worked Example: Balancing Four Connected States

A hypothetical four-state process has states $J,K,L,M$. Its only arrows are $J\to K$ at rate $0.20$ per year, $K\to J$ at rate $0.30$ per year, $K\to L$ at rate $0.30$ per year, and $L\to M$ at rate $2/3$ per year. State $M$ is absorbing. At the time of interest, the probabilities are $p_J=0.10$, $p_K=0.20$, $p_L=0.30$, and $p_M=0.40$. Find the derivative vector in the order $(J,K,L,M)$.

Method. For each state, use the probabilities at the source of its incoming arrows and subtract all flows out of that state.

State $J$ receives flow from $K$ and sends flow to $K$:

p_J'=(0.20)(0.30)-(0.10)(0.20)=0.06-0.02=0.04\text{ per year}.

State $K$ receives flow from $J$. It sends flow to both $J$ and $L$, so its total outgoing flow is the sum of those two terms:

p_K'=(0.10)(0.20)-(0.20)(0.30)-(0.20)(0.30) =0.02-0.06-0.06=-0.10\text{ per year}.

This result follows from the stated rates: the outgoing terms are $0.06$ and $0.06$, whose sum is $0.12$, so the derivative is $0.02-0.12=-0.10$ per year. In particular, the $K\to L$ rate is $0.30$, giving flow $(0.20)(0.30)$.

State $L$ receives flow from $K$ and sends flow to $M$:

p_L'=(0.20)(0.30)-(0.30)\left(\frac{2}{3}\right) =0.06-0.20=-0.14\text{ per year}.

State $M$ receives flow from $L$, with no outgoing flow:

p_M'=(0.30)\left(\frac{2}{3}\right)=0.20\text{ per year}.

The derivative vector is $(0.04,-0.10,-0.14,0.20)$ per year, and its components sum to zero. Common mistake avoided: do not omit one of $K$'s outgoing arrows, and do not use $p_L$ to weight the $K\to L$ transition. The source probability for that arrow is $p_K$.

Time-Dependent Intensities and Instantaneous Balance

If an intensity changes with time, its value in the balance is the value at the time where the derivative is being evaluated. Do not substitute an average rate over a longer interval unless the problem specifically asks for an approximation over that interval. The state-by-state structure is unchanged; only the rates now depend on $t$.

Worked Example: Using Intensities at the Current Time

A hypothetical process has states $U,V,W$ and arrows $U\to V$ at intensity $\alpha(t)=0.2+0.1t$ per year, $V\to U$ at intensity $0.1$ per year, $V\to W$ at intensity $0.3$ per year, and $W\to V$ at intensity $0.2$ per year. Here $t$ is measured in years. At $t=2$, the occupancy probabilities are $p_U=0.40$, $p_V=0.35$, and $p_W=0.25$. Find the derivatives at $t=2$.

Method. First evaluate the time-dependent intensity: $\alpha(2)=0.2+0.1(2)=0.4$ per year. Then balance flows using the probabilities at $t=2$.

For $U$, the incoming flow is from $V$, and the outgoing flow is to $V$:

p_U'=(0.35)(0.1)-(0.40)(0.4)=0.035-0.160=-0.125\text{ per year}.

For $V$, the incoming flows are from $U$ and $W$; the outgoing arrows are to $U$ and $W$:

p_V'=(0.40)(0.4)+(0.25)(0.2)-(0.35)(0.1+0.3) =0.160+0.050-0.140=0.070\text{ per year}.

For $W$, flow arrives from $V$ and departs to $V$:

p_W'=(0.35)(0.3)-(0.25)(0.2)=0.105-0.050=0.055\text{ per year}.

The components sum to $-0.125+0.070+0.055=0$. Common mistake avoided: using $\alpha(0)=0.2$ or treating the rate as constant over time would give the wrong instantaneous flow at $t=2$. The relevant rate is $\alpha(2)=0.4$.

Checks and Common Exam Traps

A useful check is that all state derivatives sum to zero when the probabilities describe the entire state space. In the state-by-state formula, each transition $i\to j$ contributes $-p_iq_{ij}$ to the derivative for $i$, and $+p_iq_{ij}$ to the derivative for $j$. When the equations are added, those two contributions cancel. This cancellation is the local accounting reason total probability is conserved.

That check can catch a missing arrow or a sign error, but it cannot prove that every individual equation is correct: two mistakes can cancel in the total. Verify the source and destination for each arrow before relying on the sum.

Weighting an arrow by its destination probability. For $i\to j$, the flow is $p_iq_{ij}$, not $p_jq_{ij}$.
Counting only one outgoing arrow. If a state has multiple destinations, add all outgoing intensities before multiplying by its occupancy probability, or calculate each outgoing flow separately.
Reversing the sign convention. Incoming flow is positive and outgoing flow is negative in the derivative for the state being balanced.
Mixing time units. If $t$ is in years, an intensity per month must be converted to a per-year rate before combining it with other per-year intensities.
Using an interval average for an instantaneous derivative. With time-dependent intensities, evaluate each rate at the specified time.

Key takeaway: For each state, write $p_j'(t)$ as the sum of $p_i(t)q_{ij}(t)$ over incoming arrows minus $p_j(t)$ times the total intensity of outgoing arrows. Track each arrow from its source to its destination, then check whether all state derivatives sum to zero.

Check Your Understanding

For each problem, construct the state-by-state balances from the stated arrows. Derivatives are evaluated at the time and probabilities given.

A process has arrows $A\to B$ at rate $0.5$, $A\to C$ at rate $0.2$, and $B\to C$ at rate $0.4$, all per year. State $C$ is absorbing. If $(p_A,p_B,p_C)=(0.60,0.25,0.15)$, find $(p_A',p_B',p_C')$. (Answer: $(-0.4200, 0.2000, 0.2200)$ per year.)
A process has arrows $A\to B$ at rate $0.3$, $B\to A$ at rate $0.1$, $B\to C$ at rate $0.2$, and $C\to A$ at rate $0.4$, all per year. For probabilities $(0.20,0.50,0.30)$, find each derivative. (Answer: $(0.1100,-0.0900,-0.0200)$ per year.)
Two states $R,S$ have arrows $R\to S$ at intensity $\lambda(t)=0.1+0.05t$ per year and $S\to R$ at rate $0.2$ per year, where $t$ is in years. At $t=2$, let $(p_R,p_S)=(0.40,0.60)$. Find the derivatives at that time. (Answer: $(0.0400,-0.0400)$ per year.)
A process has arrows $U\to V$ at rate $0.2$, $U\to W$ at rate $0.1$, $V\to U$ at rate $0.3$, $V\to W$ at rate $0.4$, and $W\to Z$ at rate $0.5$, all per year; $Z$ is absorbing. If $(p_U,p_V,p_W,p_Z)=(0.20,0.30,0.40,0.10)$, find the derivative vector in that order. (Answer: $(0.0300,-0.1700,-0.0600,0.2000)$ per year.)

What You'll Learn