Why Parallel Randomness Needs More Than a Seed
A single MICE run may involve many random draws: model parameters are sampled, missing values are generated, and chains proceed through repeated updates. When independent imputation jobs are run in parallel, their execution order can vary with worker availability and task duration. A global seed set once before parallel work does not, by itself, ensure that each job receives the same random numbers every time.
The key distinction is between which task receives a random-number stream and which worker executes that task. A reproducible design assigns streams to stable task identifiers before dispatch. Workers then use the stream belonging to the task they receive, rather than relying on whichever random-number state happens to be current on that worker.
This tutorial concerns reproducibility with respect to worker scheduling and task completion order. It does not establish that an imputation model is appropriate, that its chains have converged, or that separate chains are statistically independent in every relevant sense. Separate, well-designed streams address the allocation of pseudorandom numbers; they do not replace statistical diagnostics.
Conditions for Schedule-Invariant Results
A pseudorandom-number generator is deterministic: given the same generator, initial state, and sequence of requests, it produces the same values. Parallel reproducibility therefore depends on making the requests for random values independent of worker scheduling. In practice, this requires the following conditions.
- Stable task identities. Each planned job has a unique identifier that does not change when jobs are reordered. For example, use an identifier for a planned imputation replicate or sensitivity scenario, not “the third job to finish.”
- Stable task-to-stream assignment. The stream is chosen from the task identifier before work begins. A worker must not determine the stream merely because it has a particular worker number or starts first.
- Separate streams from a suitable generator. Use a generator and stream-splitting procedure designed for parallel use. Merely choosing several nearby integer seeds does not prove that the resulting sequences are independent or non-overlapping.
- Deterministic computation within each task. Given the same input, software environment, and task stream, the task must consume random values in the same order. Data-dependent branching is compatible with this condition if the branch decisions themselves are reproducible.
- Results collected by task identity. Store each result with its task identifier and restore a canonical order before comparing or combining outputs. Completion order is not a reliable identifier.
The stream-splitting method matters. Some generators provide formal stream or substream facilities; for example, R’s L’Ecuyer-CMRG generator supports streams designed for parallel computation. Use the documented mechanism of the chosen software and record the generator and stream-allocation method. The earlier material on Managing Random Seeds in MICE covers the role of a seed in a reproducible run; the additional requirement here is a stable allocation of random-number sequences across parallel tasks.
Why Task-Indexed Streams Work
Let \(F_j\) denote the deterministic computation for task \(j\), including its MICE procedure, and let \(S_j\) be the stream assigned to that task. Its result can be written \(R_j=F_j(S_j)\), with the input data and settings understood as fixed. A scheduler chooses a worker and an execution time, but neither appears in this expression. Consequently, for any two schedules \(a\) and \(b\),
for every task \(j\). The complete result collection, when arranged by task identifier, is therefore the same. This argument does not apply if the stream is selected by worker: then task \(j\) may receive \(S_1\) under one schedule and \(S_2\) under another. Nor does it apply if the program uses an untracked global random-number state shared among tasks.
The guarantee is reproducibility of each task’s random input under the stated conditions. A stream-splitting method is designed to produce sequences suitable for parallel use, but “different streams” should not be interpreted as a universal mathematical proof of independence for every generator and every implementation. Follow the generator’s documented guarantees. In addition, parallel floating-point calculations can sometimes differ slightly if sums are reduced in a different order, even when the imputed values are identical. Compare exact imputed data when appropriate, and use a stated numerical tolerance for derived floating-point summaries.
Worked Example: Same Tasks, Different Completion Order
Suppose a hypothetical analysis runs three independent MICE jobs, labelled A, B, and C. Each job imputes one missing measurement in a fixed dataset. Before dispatch, the analyst assigns stream \(S_A\) to A, \(S_B\) to B, and \(S_C\) to C. For illustration, suppose those streams produce imputed values of 12, 16, and 20 units, respectively.
In the first run, the completion order is A, B, C, so the recorded sequence of values is \(12,16,20\). In a second run, the completion order is C, A, B, so the raw completion sequence is \(20,12,16\). Once results are put back into task order, both runs give A = 12, B = 16, and C = 20 units.
For a simple summary across these three jobs, the mean is
Reordering the same three values does not alter this particular mean. The more important check is that each value remains attached to its intended job: if A and C were accidentally interchanged, the average would still be 16 units while the task-specific results would be wrong. Thus, reproducibility requires stable stream assignment and task-labelled result collection.
Common Failure: Seeding Workers Instead of Tasks
A tempting strategy is to give each worker a different seed. This can make a run reproducible when the same tasks always go to the same workers in the same way. It does not generally make results invariant to scheduling. If task assignments change, a task may receive a different worker’s random-number sequence. Reusing one seed on every worker is worse: workers may generate identical or overlapping sequences, depending on the generator and how it is initialized.
Worked Example: A Worker-Based Assignment Changes a Job
Consider two hypothetical MICE jobs, A and B, and two workers. Suppose worker 1’s initialized sequence makes job A’s imputed mean 8 units, while worker 2’s sequence makes that same job’s imputed mean 20 units. These numbers are illustrative outputs, not a claim about a particular generator.
In schedule 1, A runs on worker 1, so its result is 8 units. In schedule 2, A runs on worker 2, so its result is 20 units. The change in A’s result is
The input data and job definition did not change; only the worker assignment changed. The worker-based scheme therefore fails the required invariance check. With task-indexed allocation, A receives the same preassigned stream in both schedules, so its result remains the same even if it moves between workers.
A reliable implementation creates the task identifiers and their streams before launching workers. A worker receives a task together with that task’s stream state, initializes or restores the generator state as documented, runs the task, and returns both the task identifier and result. Avoid having workers draw their stream assignments from a shared random-number state as tasks arrive: the order of arrival can then determine the allocation.
List the independent jobs and assign each a stable identifier. Decide the task granularity in advance, such as one complete imputation job per task.
Use a documented parallel stream-splitting facility to give each task a distinct stream. Save the association between task identifiers and stream states.
Do not select a stream from worker number, task start time, or completion order. Keep task inputs and settings fixed.
Store outputs by task identifier, then compare results across schedules in canonical task order.
Verification and Failure Diagnostics
A useful test deliberately changes the schedule while keeping the task list, inputs, stream mapping, generator, and software environment fixed. For example, compare a run with two workers to a run with four workers, or introduce delays so tasks finish in a different order. The comparison should be made by task identifier, not by position in the returned list.
For each task, compare the completed datasets or a suitable exact representation of them. If exact equality is expected, test exact equality. If only derived floating-point summaries are compared, specify a numerical tolerance and explain why it is adequate; a tolerance must not conceal changed imputations. A compact diagnostic table can include task identifier, stream identifier, completion order, and an output checksum. A checksum helps flag a changed output but does not replace comparing the underlying data when a discrepancy needs investigation.
Worked Example: Checking Results in Canonical Task Order
Suppose two hypothetical jobs return completed measurements. Job A imputes 6.2 and 7.1 units; job B imputes 5.8 and 6.9 units. In one run, the completion order is A then B; in another it is B then A. After sorting by task identifier, compare A’s two values with A’s values from the second run, and do the same for B.
As a simple overall check, the mean of all four values is
This summary alone is not a sufficient test: a swap between A and B could preserve the overall mean. A stronger check is that the task-labelled vectors match exactly: A is \((6.2,7.1)\) in both runs and B is \((5.8,6.9)\) in both runs. If task-specific outputs differ, investigate whether stream allocation depended on workers, whether a global random-number state was shared, or whether another source of nondeterminism changed the computation.
Other warning signs include duplicated stream identifiers, results that change when the number of workers changes, and output files named by completion position rather than task ID. Also check whether a task launches its own parallel work: nested random-number use needs a documented allocation strategy too. A correctly assigned outer stream does not automatically control random draws made by an unconfigured inner process.
Choosing a Parallel Strategy and Knowing Its Limits
| Approach | Strength | Limitation |
|---|---|---|
| Task-indexed parallel streams | Allows task-specific results to remain stable across worker schedules when the stated conditions hold. | Requires supported stream allocation, stable task IDs, and correct state handling. |
| Worker-based seeds | Simple to set up for a fixed worker arrangement. | A task can receive a different sequence when reassigned; reused worker seeds can also cause sequence overlap. |
| Serial execution | A fixed sequence of tasks is straightforward to reproduce and can serve as a reference check. | May be slower; changing the serial task order can still change outputs if tasks draw from one continuing global state. |
Parallelize jobs that are genuinely separate, such as independently specified imputation runs, rather than interrupting the dependent update sequence within a MICE chain. Within a chain, each conditional-model update uses the current state of the other variables; arbitrary concurrent updates can change the algorithm, not merely its scheduling. If an implementation supports parallel chains or jobs, use its documented random-stream controls and verify what unit of work receives each stream.
Task-indexed streams also do not make results invariant to every change. Changing the generator, package version, model settings, data order, task partition, or the number and identity of planned tasks may change outputs. The reproducibility claim should be stated narrowly: results are unchanged under different worker schedules for a fixed set of tasks and a fixed computational environment. Nor do distinct streams diagnose convergence, model misspecification, or sensitivity to missing-data assumptions.
Common Mistakes
- Assuming one global seed is enough. In parallel work, the sequence consumed by a task can depend on scheduling. Assign streams to tasks explicitly.
- Assuming different integer seeds guarantee independent streams. Use a generator’s documented parallel stream mechanism rather than relying on seed values alone.
- Comparing results by return position. Completion order can change. Compare by stable task identifier.
- Treating stream separation as a convergence check. Stream design concerns reproducible random-number allocation; assess the imputation procedure and its diagnostics separately.
- Claiming bit-for-bit reproducibility across environments. The same streams may not yield identical results after software, generator, numerical library, or model implementation changes.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why can setting one global seed before launching several workers fail to give task-specific results that are invariant to scheduling?
- Two tasks use streams selected by worker number. What can happen to task A’s output if it runs on a different worker in a later run?
- In a validation run, task B finishes first, but task A is listed first in the saved results. Is that necessarily an error? What should be checked?
- Does assigning distinct parallel streams prove that MICE chains have converged? Explain.
- A task’s imputed values match across schedules, but a pooled floating-point summary differs in its last decimal place. Give one possible explanation and a suitable check.
Solutions. (1) Workers may consume a shared or worker-specific random-number sequence in an order affected by scheduling, so a task can receive different draws. Assign streams to task IDs instead. (2) A can receive another stream and produce different imputations; stream selection should follow A, not its worker. (3) Not necessarily. Results should be associated with stable task IDs, so a saved canonical order may differ from completion order; verify that each task’s output is correctly labelled and matches its assigned stream. (4) No. Separate streams address random-number allocation, not convergence or model adequacy; those need their own diagnostics. (5) Floating-point reductions can depend on summation order. Compare task-level imputed data and recompute the summary in a fixed order, using a stated tolerance if appropriate.