From Small-Time Probabilities to Rate Constraints
For a continuous-time Markov chain, a transition matrix at a short time interval \(h\) describes probabilities, while a Q-matrix records their first-order rates of change at \(h=0\). Probability constraints therefore impose sign and row-sum restrictions on the rates. On a finite state space, these restrictions follow directly by taking limits of finite sums. On a countable state space, infinitely many destinations create an additional issue: probability can, in principle, move to states that recede as \(h\) shrinks. The limits for each fixed destination need not capture that probability.
Let \((X_t)_{t\ge 0}\) be a time-homogeneous continuous-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Write \(p_{ij}(h)=\mathbb{P}(X_h=j\mid X_0=i)\) for its transition probabilities over an interval of length \(h\). For each \(h\ge 0\), \(P(h)=(p_{ij}(h))\) is row-stochastic: \(p_{ij}(h)\ge 0\) and \(\sum_{j\in S}p_{ij}(h)=1\). We use \(p_{ij}(0)=\mathbf{1}_{\{i=j\}}\).
The definition concerns the rate constraints, not a construction of a chain from a Q-matrix. The essential small-time fact is that \(p_{ij}(h)\) is a nonnegative probability, and the diagonal probability cannot exceed one. These facts constrain the limits even before any further structure of the chain is used.
Proof. Fix a state \(i\). For \(j\ne i\), \(q_{ij}=\lim_{h\downarrow0}p_{ij}(h)/h\), and every term in this limit is nonnegative, so \(q_{ij}\ge0\). Also \(q_{ii}=\lim_{h\downarrow0}(p_{ii}(h)-1)/h\le0\), since \(p_{ii}(h)\le1\). For any finite set \(F\subseteq S\setminus\{i\}\), row-stochasticity gives \(\sum_{j\in F}p_{ij}(h)\le1-p_{ii}(h)\). Dividing by \(h>0\) and taking the limit yields \(\sum_{j\in F}q_{ij}\le -q_{ii}\). For countable \(S\), the sum of nonnegative terms \(\sum_{j\ne i}q_{ij}\) is the supremum of these finite partial sums. It is therefore at most \(-q_{ii}\), which is finite. If \(S\) is finite, the row-sum identity gives \(\sum_{j\in S}(p_{ij}(h)-\mathbf{1}_{\{i=j\}})/h=0\). This is a finite sum, so taking limits term by term gives \(\sum_{j\in S}q_{ij}=0\). \(\square\)
This proposition makes an important distinction. On a finite state space, the zero row sum follows from the rate limits themselves. On a countable state space, the limits ensure a finite total rate of the fixed-destination rates, but initially give only \(\sum_{j\ne i}q_{ij}\le -q_{ii}\). An additional condition is needed to rule out probability escaping, at rate scale, to destinations that depend on \(h\).
Proof. Write \(R=S\setminus F\), where \(F\) is chosen as in the condition. For each \(h>0\), row-stochasticity gives \((1-p_{ii}(h))/h=\sum_{j\in F\setminus\{i\}}p_{ij}(h)/h+\sum_{j\in R}p_{ij}(h)/h\). The left side tends to \(-q_{ii}\). The finite sum over \(F\setminus\{i\}\) tends to \(\sum_{j\in F\setminus\{i\}}q_{ij}\); the limsup of the remaining nonnegative sum is at most \(\varepsilon\). Hence \(-q_{ii}\le\sum_{j\in F\setminus\{i\}}q_{ij}+\varepsilon\le\sum_{j\ne i}q_{ij}+\varepsilon\). The proposition already gives \(\sum_{j\ne i}q_{ij}\le -q_{ii}\). Since this argument holds for every \(\varepsilon>0\), the two quantities are equal. Together with finiteness from the proposition, this is the conservative row constraint at \(i\). \(\square\)
Worked Rate Calculations
Worked Example: Recovering a Finite-State Q-Matrix
Let \(S=\{1,2,3\}\), and for \(0\le h\le\tfrac14\) define \(P(h)=I+hQ\), where
First check the rate constraints: the off-diagonal entries are nonnegative, and the row sums are \(-3+2+1=0\), \(0-2+2=0\), and \(4+0-4=0\). The resulting transition matrix is
For \(0\le h\le\tfrac14\), all entries are nonnegative: in particular, \(1-4h\ge0\), and \(1-3h\ge\tfrac14\), \(1-2h\ge\tfrac12\). Each row sums to one: \(1-3h+2h+h=1\), \(0+1-2h+2h=1\), and \(4h+0+1-4h=1\). The rate limits recover the entries. For example, \((p_{12}(h)-0)/h=2\), while \((p_{11}(h)-1)/h=-3\); applying the same calculation to every entry gives the displayed \(Q\). The diagonal entries are the negatives of their respective off-diagonal totals: \(-3=-(2+1)\), \(-2=-(0+2)\), and \(-4=-(4+0)\).
Worked Example: Finite Exit Rates on a Countable Space
Take \(S=\{0,1,2,\ldots\}\), and for \(0\le h\le1\) define, for each \(i\in S\), \(p_{ii}(h)=1-h/(i+1)\), \(p_{i,i+1}(h)=h/(i+1)\), with all other entries in row \(i\) equal to zero. Each row sums to \(1-h/(i+1)+h/(i+1)=1\). Since \(i+1\ge1\), both nonzero entries are nonnegative for \(0\le h\le1\).
For every fixed \(i\), the diagonal rate is \(\lim_{h\downarrow0}[-h/(i+1)]/h=-1/(i+1)\), and the only nonzero off-diagonal rate is \(q_{i,i+1}=\lim_{h\downarrow0}[h/(i+1)]/h=1/(i+1)\). Thus \(q_i=1/(i+1)<\infty\), \(q_{ii}=-1/(i+1)\), and the row sum is zero. For example, at \(i=2\), \(p_{22}(h)=1-h/3\), \(p_{23}(h)=h/3\), and the rates are \(q_{22}=-1/3\), \(q_{23}=1/3\); their sum is zero. The exit rate is finite in every row, although there are infinitely many rows.
Worked Example: Why Countable Row Sums Need Care
Here is a family of stochastic rows that illustrates why fixed-destination limits alone do not justify summing infinitely many limits. Let \(S=\{0,1,2,\ldots\}\). For \(0<h<\tfrac12\), let \(k(h)=\lceil1/h\rceil\), set \(p_{00}(h)=1-h\), \(p_{0,k(h)}(h)=h\), and set every other entry in row \(0\) to zero. For rows \(i\ne0\), let \(p_{ii}(h)=1\) and all other entries be zero. Since \(1/h>2\), \(k(h)\ge3\), so the two nonzero entries in row \(0\) are distinct. That row sums to \(1-h+h=1\), and every other row sums to \(1\); all entries are nonnegative.
For each fixed \(j\ne0\), once \(h\) is small enough, \(k(h)\ne j\). Consequently \(p_{0j}(h)=0\) for all sufficiently small \(h\), and its rate limit is \(q_{0j}=0\). But the diagonal rate is \(q_{00}=\lim_{h\downarrow0}[(1-h)-1]/h=-1\). Thus \(\sum_{j\ne0}q_{0j}=0\), which is not \(-q_{00}=1\). The probability \(h\) has gone to a destination that changes with \(h\), so no fixed off-diagonal rate records it. This family is a collection of stochastic matrices, not a transition semigroup; it illustrates the limitation of the rate-limit argument, not a construction of a continuous-time Markov chain.
What the Constraints Do—and Do Not—Say
A conservative Q-matrix is the appropriate rate description when each state has a finite total exit rate and the diagonal rate accounts for all the off-diagonal rates. It is useful to check these conditions before interpreting or using a proposed rate table. A negative off-diagonal entry cannot be a transition rate, a positive diagonal entry contradicts the small-time probability bound, and an infinite sum of off-diagonal rates does not meet the finite-exit-rate condition in the definition used here.
For a finite state space, the proof of zero row sums needs no extra tail condition: there are only finitely many terms, so the limit passes through the sum. For a countable state space, a sufficient additional condition is rate-tightness. The comparison is at the same small threshold \(h\): the total probability outside one fixed finite set, divided by \(h\), must become arbitrarily small as the set is enlarged. Without such control, it is not valid to infer that an infinite sum of rate limits equals the limit of the sums.
Common mistake. A tempting countable-state calculation is to write \(\lim_{h\downarrow0}\sum_{j\ne i}p_{ij}(h)/h=\sum_{j\ne i}\lim_{h\downarrow0}p_{ij}(h)/h\) solely because the row probabilities sum to one. The countable-state example shows the error: for row \(0\), the left side equals \(\lim_{h\downarrow0}h/h=1\), whereas every fixed-destination limit on the right is zero, so the right side is zero. The correction is to prove a condition that controls the tails, or to use only the inequality \(\sum_{j\ne i}q_{ij}\le -q_{ii}\) that follows from finite partial sums. On a finite state space, term-by-term limiting is valid because the sum has finitely many terms.
| Constraint | What the rate limits imply | What is needed for equality |
|---|---|---|
| Off-diagonal sign | \(q_{ij}\ge0\) for \(j\ne i\) | No extra condition |
| Diagonal sign | \(q_{ii}\le0\) | No extra condition |
| Exit-rate bound | \(\sum_{j\ne i}q_{ij}\le -q_{ii}<\infty\) | Follows from finite partial sums |
| Zero row sum | Always on a finite state space | On a countable space, a condition such as rate-tightness suffices |
Check Your Understanding
Use the transition-rate definition and distinguish finite sums from countable sums in your reasoning.
- Why must \(q_{ij}\ge0\) when \(j\ne i\), and why must \(q_{ii}\le0\)?
- For a finite state space, explain why taking the rate limit in the row-sum identity gives \(\sum_jq_{ij}=0\).
- On a countable state space, what inequality relates \(\sum_{j\ne i}q_{ij}\) to \(-q_{ii}\), and why is the sum finite when \(q_{ii}\) is finite?
- State the rate-tightness condition in words and explain how it supports equality of the countable row sum.
- In the countable-state example with destination \(k(h)=\lceil1/h\rceil\), compute the diagonal rate and the sum of the fixed-destination off-diagonal rates. Why are they different?