Flat Trace Plots That Conceal Poor Exploration

Why a Flat Trace Needs a Closer Look

A trace plot displays a monitored quantity against FCS iteration. A plot that appears stable can be reassuring: it may show that the quantity is no longer changing systematically. But visual flatness alone does not show that the chain has explored the relevant range of imputed data. The plot may repeat the same imputed state many times, remain inside a narrow band, or summarize changing data with a statistic that hides those changes.

This tutorial focuses on that diagnostic limitation. It does not treat flatness as proof of failure: a well-mixing chain can have a relatively steady trace, and a narrow range can be appropriate for a particular variable and conditional model. The question is whether the plotted quantity is informative enough, and whether other evidence is consistent with adequate exploration.

Definition: Let $S_t$ denote the state of the imputed data at FCS iteration $t$, represented here by the ordered vector of current imputed values for all missing cells. For $T\ge 2$, the repeated-state rate is $$ R_{\text{repeat}} = \frac{\sum_{t=1}^{T-1} I(S_{t+1}=S_t)}{T-1}, $$ where $I(S_{t+1}=S_t)=1$ when the two consecutive imputed-value vectors are identical and $0$ otherwise. For $T=1$, this rate is undefined because there are no transitions.

The state definition matters. Observed values are fixed, so the comparison concerns the current imputations in the same cells, in the same order. A repeated imputed-value vector is evidence that the recorded imputations did not change at that transition. It does not necessarily mean that every internal random draw or model parameter in an implementation was identical; those quantities may not be included in $S_t$. Consequently, record what the state contains before interpreting repeated-state counts.

What a Trace Plot Can Hide

A typical trace plots a scalar summary $Q_t$, such as the mean of an imputed variable at iteration $t$, rather than the full state $S_t$. This is a useful reduction: a full state can contain many missing values, making it impossible to display directly. The reduction also loses information. Different states can have the same summary, and a summary can vary little even when other features of the imputations change.

A trace can also look flat because its vertical scale is wide, because the plotted statistic has been rounded, or because the chain has only explored a restricted part of a range that would be plausible under the model and data. Flatness is therefore a statement about the visible plotted values, not a general certificate that the imputed states are representative or well explored.

Worked Example: Counting Consecutive Repeated States

Suppose a hypothetical FCS run has five recorded states for four missing cells. The states are $S_1=S_2=S_3=(11,12,13,14)$, $S_4=(11,12,13,15)$, and $S_5=(11,12,13,14)$. Calculate the repeated-state rate.

There are $T=5$ states, so there are $T-1=4$ transitions. The first transition repeats because $S_2=S_1$. The second repeats because $S_3=S_2$. The third does not repeat because the fourth cell changes from 14 to 15. The fourth does not repeat because $S_4=(11,12,13,15)$, whereas $S_5=(11,12,13,14)$. Thus two of the four transitions repeat:

R_{\text{repeat}}=\frac{2}{4}=0.5.

Half of the observed transitions returned exactly the same recorded imputed values as the immediately preceding iteration. This is a reason to inspect the chain and the update procedure more closely, not by itself a universal threshold for rejecting a run. The interpretation depends on the state being recorded and on whether the variables and update mechanism make exact repeats plausible.

Worked Example: A Flat Mean with Changing Imputed Values

Consider two hypothetical states for four imputed cells: $S_1=(10,20,30,40)$ and $S_2=(11,19,30,40)$. Calculate the mean in each state and assess what the corresponding trace shows.

For $S_1$, the mean is $(10+20+30+40)/4=100/4=25$. For $S_2$, it is $(11+19+30+40)/4=100/4=25$. The mean trace is flat at 25 across this transition, although the states are different: the first imputation increased by 1 and the second decreased by 1.

The reason is that the mean compresses a vector into a single number. In general, different vectors can have the same sum and therefore the same mean. A flat trace of the mean cannot establish that individual imputed values were repeated, nor can it show that other features of the imputed data were stable. Inspecting repeated values, selected cell-level traces, or additional summaries can reveal changes that the mean conceals.

Restricted Ranges and the Scale of Inspection

Repeated states are only one way a visually flat trace can be misleading. A chain may change at every iteration while staying within a very narrow range. Whether that range is concerning depends on context: a narrow range may be exactly what the conditional distribution implies, or it may indicate that the chain is exploring only a small part of the values that should be plausible.

The plotted scale can obscure the difference. If the vertical axis spans a wide range, small but meaningful movement can appear flat. If the axis is tightly cropped, small changes may look substantial. Always inspect axis limits and numerical summaries rather than judging variability from the line’s appearance alone. Compare the explored range with the variable’s allowable values and the values plausible given the observed data and the conditional models. Do not assume that the observed values alone define every plausible imputed value.

Worked Example: Checking a Narrow Trace Numerically

Suppose a hypothetical trace contains these ten iteration summaries for an integer-valued imputed quantity: 42, 43, 43, 42, 44, 43, 42, 44, 43, 42. The substantive context indicates that values from 30 to 60 could be plausible, although that interval is not a required range for every imputation. Calculate the mean and range, then explain what can be concluded.

The sum is $42+43+43+42+44+43+42+44+43+42=428$. Therefore the average of the ten summaries is $428/10=42.8$. The minimum is 42 and the maximum is 44, so the observed range is $44-42=2$.

The trace varies over only three distinct values, 42, 43, and 44, despite a broader contextually plausible interval. This is a signal to investigate why the sampled summaries occupy such a narrow band—for example, by examining more than one chain, looking at cell-level imputations, and checking whether rounding or the summary itself explains the limited variation. It does not prove poor exploration: the conditional distributions might genuinely concentrate near these values. The range is diagnostic evidence to interpret, not a pass-or-fail rule.

A Practical Diagnostic Sequence

When a trace looks flat, use checks that add information beyond the appearance of one line. The following sequence is a structured investigation, not an automatic stopping rule. It assumes that iterations and chains have been recorded consistently and that the monitored quantities correspond to the variables and cells of interest.

Read the trace scale and definition.
Identify exactly what $Q_t$ summarizes, whether values were rounded, and the vertical-axis limits. Check whether an apparently flat line could result from compression by the plotting scale or loss of detail in the summary.

Check for repeated states.
Compare consecutive vectors of imputed values for a suitably chosen set of cells or, where feasible, the full recorded imputed state. Report the number of repeats and the denominator $T-1$; do not report a rate when $T=1$.

Quantify variation and range.
Report numerical summaries such as the minimum, maximum, number of distinct values, and range for the trace statistic. Consider whether the observed values are narrow relative to the relevant substantive and model-based context.

Compare complementary views.
Inspect other relevant summaries, selected cell-level traces, and independent chains. Ask whether they tell a consistent story; a single flat summary does not represent every aspect of the imputed states.

Investigate a mismatch.
If the plot looks stable but repetitions or restricted ranges raise concern, check initialization, variable ordering, conditional dependence, and implementation details. Use the evidence to guide further diagnosis rather than treating a visually flat line as proof of convergence.

These checks rely on appropriate recording and comparable chains. A cell-level trace of one selected missing value may reveal movement in that cell but cannot establish adequate exploration of all cells. Likewise, a summary across many cells can hide behavior concentrated in a subgroup or in a particular variable. Choose diagnostics to match the question, and be explicit about what each one represents.

How This Differs from Other Diagnostics

Trace plots are a visual check of a monitored quantity across iterations. Repeated-state counts ask a narrower question: how often does the recorded imputed vector remain exactly unchanged from one iteration to the next? Range checks quantify how widely a chosen quantity varied. These approaches are complementary because none captures every property of FCS behavior.

Autocorrelation and effective sample size describe dependence and information in a monitored sequence; they do not by themselves show whether the plotted quantity hides changes in the underlying imputed cells. Comparing independent chains can reveal differences in the regions they visit, but chains with similar trace summaries can still have distinct states. Monte Carlo error concerns simulation precision for a specified summary and does not certify that the summary is a sufficient representation of the chain. Use these diagnostics together as evidence, without interpreting any one of them as a complete guarantee.

The main alternative is not to discard trace plots, but to supplement them. Monitoring multiple summaries, inspecting values at the cell or variable level, and comparing chains can expose problems a single scalar trace misses. Each additional view also has limits: examining only a few cells may miss a problem elsewhere, while a very large collection of plots can be difficult to interpret and may encourage selective attention. State which views were checked and why they are relevant.

Common Mistakes and Limitations

Calling a flat trace proof of convergence. A flat plotted summary only shows little visible change in that summary; it does not establish broad exploration of the imputed states.
Equating every repeated state with a frozen chain. Repeated imputed values are a warning to examine, but a recorded state may exclude sampled parameters or other internal quantities. State clearly what was compared.
Treating any narrow range as failure. A narrow range is concerning only in relation to what is plausible under the data, conditional models, and variable support. It is not a universal error threshold.
Assuming different states must produce different summaries. As the worked mean example shows, distinct vectors can have identical averages or other aggregate statistics.
Reporting a repeated-state rate for a single iteration. When $T=1$, there are no transitions and the rate is undefined, not zero.

These diagnostics cannot prove that an FCS chain has visited every relevant part of its state space. They are also sensitive to the choice of monitored cells, summaries, scale, and recording precision. Their purpose is to identify evidence that a reassuring-looking plot may be incomplete and to motivate targeted investigation.

Key takeaway: A flat trace is evidence only about the quantity plotted. Check what it summarizes, whether states repeat, how restricted the observed range is, and whether other summaries and chains support the same interpretation.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A trace has $T=8$ recorded states and three consecutive transitions repeat exactly. What is the repeated-state rate?
Why can two different imputed states give the same trace value when the trace displays a mean?
A trace varies over a narrow range, but all observed values lie within the scientifically plausible range. Is poor exploration established? What should be checked next?
For $T=1$, should the repeated-state rate be reported as zero or undefined? Explain.

Solutions. (1) There are $T-1=7$ transitions, so $R_{\text{repeat}}=3/7\approx0.429$. This describes the proportion of consecutive recorded transitions with identical states; it is not, by itself, a universal failure threshold. (2) A mean retains only the sum divided by the number of values. Different vectors can have the same sum, as $(10,20,30,40)$ and $(11,19,30,40)$ both have mean 25. (3) No. A narrow range within plausible values is not proof of poor exploration, and it may be consistent with the conditional distribution. Inspect the trace definition and scale, repeated states, other relevant summaries, and independent chains. (4) It is undefined because there are zero transitions; the rate’s denominator $T-1$ is zero.