Verification as a Test of the Implemented Rules
An agent-based model can produce plausible-looking trajectories while still implementing the wrong rules. A cell may move through an impermeable boundary, interact with the same neighbor twice in one update, or disappear because a counter was not updated. Verification asks whether the code implements its specified model correctly. It is distinct from asking whether the model is a good representation of a biological system.
A useful verification test is narrow: it specifies an input state and controlled conditions, then checks a result that follows from the model rules. For stochastic models, controlled conditions matter. A test can supply fixed random numbers to exercise a particular branch, or set a random seed to make a run repeatable. A fixed seed helps reproduce a failure, but does not by itself show that the code is correct.
Before testing a rule, write down its contract: what state is read, what state may change, what is held fixed, and what must remain unchanged. This is particularly important for update scheduling. As established in the earlier material on stochastic events and update scheduling, sequential and simultaneous updates can represent different rules. Tests must enforce the specified schedule rather than assume one implicitly.
Test Movement at Ordinary Locations and Boundaries
For a movement rule, test both a location where all permitted moves are available and locations where boundaries or obstacles constrain movement. On a lattice, specify the candidate moves, how a move is selected, and what happens when a proposal is invalid. For an off-lattice rule, test the displacement and any constraint applied to the resulting position. In either case, check that one update changes only the intended position and leaves agent identity and unrelated state intact.
Worked Example: Testing a Reflecting Lattice Boundary
Suppose a cell is on a square lattice with coordinates from 0 to 4 in each dimension. At each movement update, it proposes one of four cardinal directions with equal probability. A proposal outside the lattice is rejected, leaving the cell in place. Test a cell at \((0,2)\), and use controlled inputs to force each proposal in turn.
Expected cases. An east proposal moves the cell to \((1,2)\); a north proposal moves it to \((0,3)\); a south proposal moves it to \((0,1)\). A west proposal would give the invalid coordinate \((-1,2)\), so the position must remain \((0,2)\). These four tests exercise each branch directly.
Under the stated equal-probability rule, the probability of moving east is \(1/4\), as it is for north and south. The probability of remaining at the boundary is also \(1/4\), because the west proposal is rejected. Thus the expected one-step displacement in the horizontal coordinate is \( (1/4)(1) + (1/4)(0) + (1/4)(0) + (1/4)(0)=0.25\) lattice spacings. This is a useful consistency check, but the four controlled-input tests are stronger checks of the individual branches than a single average over simulated runs.
The test also checks a population invariant: the cell remains present after a rejected move. If the model instead specifies that a blocked proposal is resampled until a valid move occurs, the expected outcomes differ. The test must follow the chosen rule, not silently substitute another boundary convention.
A movement test should also check limiting cases. If movement is disabled, no position should change. If every proposed displacement is zero, positions should remain unchanged even when the random-number generator advances. If the model uses periodic boundaries, a move beyond one edge should wrap to the opposite edge; that is a different rule from rejection and needs its own test.
Test Interactions Against Explicit Scheduling Rules
Interaction tests need a precise definition of eligibility and attempt frequency. For example, a rule might consider each eligible unordered pair once per time step, while another might let each agent select a neighbor in sequence. These are not interchangeable. A test should specify which agents are eligible, which pair or pairs are examined, and whether an interaction changes eligibility for later events in the same step.
Worked Example: Controlled Tests for Pairwise Interactions
Consider three agents, A, B, and C, for which all three unordered pairs are eligible. The rule examines each pair once, in the order AB, AC, BC. Each pair interacts if its supplied uniform random number \(u\) is less than \(p=0.20\). For this unit test, supply \(u_{AB}=0.10\), \(u_{AC}=0.35\), and \(u_{BC}=0.18\).
The decisions are \(0.10<0.20\), so AB interacts; \(0.35\not<0.20\), so AC does not; and \(0.18<0.20\), so BC interacts. The expected number of pair events is therefore two, with interacting pairs AB and BC. The test should verify those pair identities, not merely that two events occurred: an implementation that mistakenly attempted AB twice could also report two events.
This example assumes each pair is tested once and that the test interaction does not remove an agent or otherwise alter later eligibility. If an interaction changes the state, add a separate test for how that change affects subsequent pairs under the specified schedule. A failure can reveal a duplicated pair, an incorrect inequality at the threshold, or an unintended change in update order.
The comparison boundary is worth testing explicitly. If the implementation uses \(u\leq p\) rather than \(u<p\), the difference occurs only when \(u=p\). A continuous uniform draw has probability zero of equaling a particular value in ideal mathematics, but a unit test can deliberately supply \(u=p\) to confirm that the code matches its stated convention. This is a branch test, not a claim that equality commonly occurs in simulation.
Check Event Probabilities and Limiting Cases
If an event is defined by a constant hazard rate \(\lambda\) per unit time over a time step of length \(\Delta t\), and event waiting times follow an exponential distribution, the probability of at least one event during that step is
This formula depends on the constant-hazard assumption during the step. It gives \(p(0)=0\), \(p(\Delta t)\to 1\) as \(\Delta t\to\infty\) for \(\lambda>0\), and \(p(\Delta t)\approx\lambda\Delta t\) when \(\lambda\Delta t\) is small. The approximation follows from \(\exp(-x)\approx 1-x\) near \(x=0\). A probability implemented as \(\lambda\Delta t\) without a cap can exceed 1 for large steps; that cannot be a valid event probability. Whether multiple events within a step are possible is a separate modeling decision: the formula above gives the probability of at least one event, not a count of all events.
Worked Example: Testing a Rate-to-Probability Conversion
Suppose a cell has a constant event hazard of \(\lambda=0.30\) per day, and the time step is \(\Delta t=2\) days. The correct probability of at least one event during the step is
For a controlled event test, supply uniform inputs \(0.20,\ 0.45,\ 0.4513,\ 0.80,\ 0.01\), using the rule “event if \(u<p\).” The first, second, and fifth inputs trigger an event; the third and fourth do not. The expected test result is three events among these five deliberately selected inputs. This count is not an estimate of the long-run event frequency, because the inputs were selected to test both sides of the threshold.
Useful limiting checks are \(\lambda=0\), which must give \(p=0\), and \(\Delta t=0\), which must also give \(p=0\). For positive \(\lambda\) and \(\Delta t\), the probability must lie strictly between 0 and 1. These checks catch sign errors, unit mismatches, and incorrect use of \(1-\exp(\lambda\Delta t)\).
Population Accounting and a Short Justification
Population accounting provides tests that apply across many model components. Let \(N_t\) be the number of agents at the start of a step, \(B_t\) the number born during it, and \(D_t\) the number removed by death during it. If these are the only population-changing events, then
This identity follows by counting agents: each of the \(N_t\) initial agents remains unless counted among the \(D_t\) removals, and each of the \(B_t\) births adds one agent. If agents can enter or leave a spatial region, add the corresponding entries and exits. If agents merely change labels or move between compartments, the total across those categories should not change unless the rules explicitly add or remove agents.
Worked Example: Auditing a Step with Births and Deaths
A hypothetical model begins a step with 12 cells. During the step, two cells divide once each, adding two cells, and three cells die. No other population-changing events occur. The accounting identity gives
The implementation should finish with 11 cells. If cells are stored in mutually exclusive states, such as viable and dead, the sum of those state counts should also equal the total represented population, provided dead cells remain represented. If dead cells are removed from the active-agent list, that list's count should instead exclude them; the model must define which quantity is being counted. Distinguishing these quantities prevents a correct biological event from being mislabeled as a bookkeeping error.
An invariant can be checked after every update or at selected checkpoints. Examples include nonnegative counts, unique agent identifiers, positions within allowed bounds, and equality between a stored total and the sum of mutually exclusive state counts. A check should reflect the model's representation: requiring a constant population is wrong when births or deaths are permitted.
Reproducible Seeds, Diagnostics, and Limitations
A reproducible seed initializes a pseudorandom number generator so that the same code, seed, inputs, and random-number call sequence can reproduce a run. When a test fails, record the seed, model configuration, software version if relevant, and the first step at which an invariant or expected result fails. Then reduce the failing case to the smallest state that still reproduces the error.
Seeded tests are most useful as integration tests: they can exercise multiple components together and make a failure repeatable. Their limitation is that a test may pass for one seed while a bug remains in an unvisited branch. Changes to code can also change the order of random-number calls, so the same seed need not produce the same trajectory after a code change. For a precise unit test of a stochastic branch, supplying a controlled random variate is usually more diagnostic than relying on a particular seed.
When a simulation appears wrong, check the earliest divergence rather than only the final outcome. Useful diagnostics include logging the event proposal, eligibility set, random draw, decision threshold, resulting state, and population counts for a small test run. Then compare that trace with the written rule. A large discrepancy in an ensemble summary may arise from a single local implementation error, but a plausible ensemble summary does not rule one out.
| Approach | Best use | Main limitation |
|---|---|---|
| Controlled-input unit test | Checking a specific branch, threshold, or state transition | Only covers the cases explicitly constructed |
| Limiting-case check | Checking whether a simplified parameter setting gives the required behavior | Does not establish correctness away from that limit |
| Invariant or accounting check | Detecting impossible states, lost agents, or inconsistent counters | A model can preserve invariants while implementing the wrong biological rule |
| Seeded integration run | Reproducing interactions among components and diagnosing a trajectory | Passing one seed does not establish correctness across random outcomes |
No single test method is sufficient. Unit tests target specific logic, limiting cases test behavior where the answer is constrained, and invariants catch broad classes of errors. Seeded runs help connect those components in a reproducible trace. Together, they provide stronger evidence of correct implementation than visual inspection or a single plausible simulation.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A lattice model rejects out-of-bounds movement proposals. What should a test check when a cell at the left edge proposes a move to the left?
- An interaction occurs when \(u<p\). For \(p=0.4\), what should happen for controlled inputs \(u=0.39\) and \(u=0.40\)?
- For a constant hazard \(\lambda=0.10\) per day and \(\Delta t=1\) day, calculate the probability of at least one event. Give a limiting-case check for the implementation.
- A model starts with 20 agents, has 4 births and 7 deaths, and no other population changes. What is the expected final population?
- Why does repeating a run with the same seed not prove that a stochastic implementation is correct?
Solutions. (1) The position should remain at the edge, and the agent should not be lost or duplicated. (2) Since \(0.39<0.4\), the event occurs for the first input; since \(0.40\not<0.4\), it does not occur for the second under the stated strict inequality. (3) \(p=1-\exp[-(0.10)(1)]=1-\exp(-0.10)\approx 0.09516\). Setting \(\lambda=0\) should produce \(p=0\). (4) \(N_{t+1}=20+4-7=17\). (5) A seed reproduces a particular sequence of pseudorandom inputs and tests only the branches reached in that run; other inputs or branches may still expose errors.