Worked Bus Engine Replacement Model

Why Bus Replacement Is a Dynamic Choice

A bus operator deciding whether to keep an engine in service or replace it must weigh today’s costs against future consequences. Continuing to maintain an older engine may be relatively inexpensive now, but it can leave the bus in a high-mileage state with greater future operating costs. Replacing the engine is costly today, but resets the bus to a low-mileage state. A useful model must represent both the current choice and the way that choice changes future mileage.

This worked model is hypothetical. Mileage states are deliberately grouped into a small number of categories so the choice calculation is transparent. The same logic extends to a finer mileage grid, more actions, or stochastic transitions with additional possible next states.

Model, Costs, and Transitions

Let $x\in\{0,1,2\}$ be the bus’s mileage state at the start of a period. State 0 represents a new or recently replaced engine, state 1 represents approximately 10,000 miles, and state 2 represents approximately 20,000 miles or more. The operator chooses maintenance, $M$, or replacement, $R$. The period cost depends on the current state and action.

Maintenance costs $c_x$ include the period’s operating and maintenance expense. Replacement costs $C_R$ include the replacement and installation expense as well as the period’s operating expense after replacement. Thus, the replacement cost is incurred in the period of the choice, and the replacement engine is in state 0 in the next period. For this example, maintenance has the following transition probabilities:

Current state	Next state after maintenance	Probability
0	0	0.8
0	1	0.2
1	1	0.8
1	2	0.2
2	2	1.0

These probabilities describe how mileage evolves when the operator maintains the engine. Replacement instead sends the bus to state 0 with probability 1, regardless of its current state. In a more detailed application, transitions could also vary with factors such as routes or time in service.

Model assumptions: The operator is risk-neutral over discounted utility and discounts future periods by $\beta=0.9$. The mileage state contains the information needed to predict future costs and transitions. Conditional on the current state and action, transitions follow the probabilities specified above. The operator observes action-specific choice shocks before choosing; the analyst does not. Shocks are independent across actions and periods and follow a Type I extreme-value distribution with location 0 and scale 1. Period utility is negative period cost plus the choice shock.

Write the choice-specific value before the current period’s shock as $q_a(x)$. It equals the negative current cost of action $a$, plus the discounted expected value of the next state. Let $V(x)$ be the value at state $x$, integrating over the current shocks. With the stated shock normalization, the Bellman equation is:

q_M(x)=-c_x+\beta\sum_{x'}P(x'\mid x,M)V(x'), \qquad q_R(x)=-C_R+\beta V(0), \qquad V(x)=\gamma+\log\!\left(\exp(q_M(x))+\exp(q_R(x))\right),

Here $P(x'\mid x,M)$ is the probability that maintenance in state $x$ leads to next state $x'$, and $\gamma\approx0.5772$ is the mean of the standard Type I extreme-value distribution. The replacement continuation value is $V(0)$ because replacement resets mileage to state 0. The integrated value includes the current period’s choice shocks; $q_M(x)$ and $q_R(x)$ do not.

From Choice-Specific Values to Choice Probabilities

Because the operator observes the shocks, the chosen action is the one with the greatest choice-specific value plus its shock. Under independent, standard Type I extreme-value shocks, the conditional choice probability of action $a$ at state $x$ is:

P(a\mid x)= \frac{\exp(q_a(x))} {\exp(q_M(x))+\exp(q_R(x))}.

The probability depends on the difference in choice-specific values, not on their common level. For two actions, the replacement odds are $\exp(q_R(x)-q_M(x))$. This is a probability conditional on the state and the model’s parameters; it is not a claim that every operator in a given state must choose the same action.

Derivation of the two-action probability: Let $d=q_R(x)-q_M(x)$, and let $\varepsilon_R,\varepsilon_M$ be independent standard Type I extreme-value shocks with cumulative distribution function $F(u)=\exp[-\exp(-u)]$. Replacement is chosen when $\varepsilon_M<\varepsilon_R+d$. Integrating over $\varepsilon_R=u$, and substituting $z=\exp(-u)$, gives $$ P(R\mid x)=\int_{-\infty}^{\infty} f(u)F(u+d)\,du =\int_0^\infty \exp[-z(1+\exp(-d))]\,dz =\frac{1}{1+\exp(-d)}. $$ This is equivalent to the choice-probability formula above.

Worked Example: Interpreting the Mileage Transitions

Suppose a bus is currently in state 1, approximately 10,000 miles. Under maintenance, the model assigns probability 0.8 to remaining in state 1 and probability 0.2 to moving to state 2. State indices are expressed in units of 10,000 miles for this illustrative calculation.

The expected next-state index after maintenance is:

E[x'\mid x=1,M]=(1)(0.8)+(2)(0.2)=0.8+0.4=1.2.

So the expected next-state index is 1.2, or approximately 12,000 miles on this simplified scale. This expectation does not mean that the bus literally moves to a fractional mileage category: the model assigns probability 0.8 to state 1 and 0.2 to state 2. If the operator replaces the engine instead, the next state is 0 with probability 1. The difference between these transition outcomes is one reason replacement can be attractive even when it is more expensive in the current period.

Worked Model and Dynamic Choices

For numerical calculations, suppose the maintenance costs and replacement cost are as follows. Costs are expressed in hypothetical monetary units per period; all figures are illustrative, not observations from a real fleet.

Current state	Maintenance cost $c_x$	Replacement cost $C_R$
0	8.777	35.600
1	8.877	35.600
2	16.870	35.600

The model is solved by finding values $V(x)$ that satisfy the Bellman equation in all three states. A value-iteration algorithm starts from a guess for $V$, applies the Bellman equation repeatedly, and updates the values until they converge. The resulting values here are approximately $V(0)=-100.000$, $V(1)=-110.000$, and $V(2)=-125.000$. Negative values are consistent with the model’s convention of negative costs as utility.

Worked Example: Checking the High-Mileage Value and Replacement Probability

Use the converged values to check the choice calculation in state 2. Under maintenance, the bus remains in state 2 with probability 1, so the expected continuation value is $V(2)=-125.000$. The maintenance choice-specific value is therefore:

$$ q_M(2)=-16.870+0.9(-125.000)=-16.870-112.500=-129.370. $$

Replacement costs 35.600 now and leads to state 0 next period. Its choice-specific value is:

$$ q_R(2)=-35.600+0.9(-100.000)=-35.600-90.000=-125.600. $$

Replacement has the higher choice-specific value by $q_R(2)-q_M(2)=3.770$. The probability of replacement is:

P(R\mid 2)=\frac{\exp(-125.600)} {\exp(-129.370)+\exp(-125.600)} =\frac{1}{1+\exp(-3.770)} \approx0.9775.

Thus, under the specified shock distribution and model parameters, the predicted probability of replacement in state 2 is about 97.75%. The probability is high, not equal to 1, because the operator’s action-specific shocks can sometimes make maintenance preferable.

Worked Example: Checking Values Across All Three States

The candidate values should satisfy the Bellman equation in every state, not just the high-mileage state. In state 0, maintenance leads to states 0 and 1 with probabilities 0.8 and 0.2. Its choice-specific value is:

$$ q_M(0)=-8.777+0.9[0.8(-100)+0.2(-110)] =-8.777+0.9(-102) =-100.577. $$

In state 1, maintenance leads to states 1 and 2 with probabilities 0.8 and 0.2:

$$ q_M(1)=-8.877+0.9[0.8(-110)+0.2(-125)] =-8.877+0.9(-113) =-110.577. $$

In all three states, replacement has value $q_R(x)=-125.600$, because its current cost is constant and its next state is always 0. Applying $V(x)=\gamma+\log[\exp(q_M(x))+\exp(q_R(x))]$ gives approximately $V(0)=-100.000$, $V(1)=-110.000$, and $V(2)=-125.000$. For example, the replacement probability in state 0 is approximately $1.36\times10^{-11}$; in state 1 it is approximately $2.99\times10^{-7}$. These small probabilities reflect the much lower maintenance costs in those states, while retaining the possibility that shocks affect the choice.

Replacement Threshold and Interpretation

A useful way to interpret the high-mileage comparison is to ask how large the current replacement cost could be before the two actions have equal choice-specific values. In state 2, equality requires:

-c_2+\beta V(2)=-C_R+\beta V(0).

Rearranging gives the replacement-cost threshold:

C_R^*(2)=c_2+\beta[V(0)-V(2)] =16.870+0.9[-100-(-125)] =16.870+22.500 =39.370.

Worked Example: Interpreting the Replacement-Cost Threshold

The model’s replacement cost is $35.600$, which is $39.370-35.600=3.770$ units below the equal-value threshold in state 2. Accordingly, replacement exceeds maintenance by 3.770 in choice-specific value, and the replacement probability is approximately 0.9775. If the replacement cost equalled 39.370, the two choice-specific values would be equal and the two actions would each have probability 0.5 under the symmetric shock specification. This comparison concerns the current state and the model’s specified continuation values; it does not say that replacing is always optimal for every bus or every parameter configuration.

Common Pitfall: Comparing Only Current Costs

A common mistake is to compare the current maintenance cost with the current replacement cost and conclude that maintenance must be preferred because $16.870<35.600$. That comparison omits the future consequences of the actions. In state 2, maintenance leads back to state 2, while replacement leads to state 0. The relevant comparison is between $q_M(2)=-129.370$ and $q_R(2)=-125.600$, which includes both the current cost and the discounted continuation value.

Another mistake is to treat a high replacement probability as a deterministic rule. The probability $P(R\mid x)$ averages over the model’s unobserved shocks, conditional on the state and parameters. It allows different choices in the same state without changing the mileage state or the cost structure. Finally, the transition probabilities must match the action: using the maintenance transition row after replacement would incorrectly remove the engine-reset consequence from the model.

Key takeaway: A bus engine replacement model combines current costs, action-specific mileage transitions, discounted continuation values, and choice shocks. Replacement decisions depend on the full choice-specific value, not on current costs alone.

Check Your Understanding

Use the model definitions and calculations above to answer these questions.

For a bus in state 1, what is the probability of moving to state 2 after maintenance, and what is the next state after replacement?
Why does replacement’s choice-specific value use $V(0)$ as its continuation value?
For state 2, calculate the maintenance choice-specific value using $c_2=16.870$, $\beta=0.9$, and $V(2)=-125.000$.
What does a replacement probability of approximately 0.9775 mean in context? What does it not mean?
Why can replacement be preferred in state 2 even though its current cost is greater than the maintenance cost?