Why the Visit Schedule Matters
Panel data record a process at scheduled visits, not continuously. Between two visits, a person may remain in the same state or make one or more unobserved transitions. Consequently, the observed data can determine probabilities of states at visits without determining the underlying timing or route of transitions. This is an issue of identifiability, not merely of computational difficulty.
The question is not simply whether an estimation algorithm returns a number. It is whether two different parameter values or models could generate exactly the same distribution of all observations allowed by the visit schedule. If so, the observations alone cannot distinguish them, even with an arbitrarily large sample under that same design.
For a time-homogeneous Markov model with transition matrix \(P(\Delta;\theta)\) over an interval of length \(\Delta\), and observed states \(x_0,\ldots,x_m\) at visits \(t_0,\ldots,t_m\), the probability of that sequence is
Thus the schedule determines which transition matrices enter the observed-data distribution. A parameter may affect the continuous-time paths while leaving every matrix used by the design unchanged. The formula assumes the process is Markov, the recorded state at each visit is correctly observed, and the visit times are specified independently of the unobserved path. If visits depend on evolving health or transition history, that observation mechanism must also be modeled.
Identifiability of a Hazard From One Visit Interval
Consider a two-state process: a person begins event-free in state 0 and, after one event, remains in absorbing state 1. Suppose only baseline and a visit one year later are observed. Under a possibly time-varying hazard \(\lambda(t)\), the probability of remaining in state 0 for one year is determined by the cumulative hazard, not by the full shape of the hazard within the year:
Worked Example: Different Hazard Shapes, Same Visit Probabilities
Suppose hypothetical panel data record each person’s state at time 0 and again at time 1 year, with no visit in between. Compare two event-hazard models. Model A has a constant hazard of 0.4 events per year. Model B has hazard 0.2 per year during the first half-year and 0.6 per year during the second half-year.
For Model A, the cumulative hazard is \(0.4 \times 1=0.4\). For Model B, it is \(0.2 \times 0.5+0.6 \times 0.5=0.4\). Therefore both models give
The observed probabilities are identical: about 67.03% remain event-free at the one-year visit and 32.97% are in the absorbing state. Yet the hazard shapes differ within the interval. With only these visits, the data identify the cumulative hazard over the year (or its equivalent survival probability), not which of the two hazard patterns occurred. An intermediate visit could provide additional information, although it would not by itself guarantee identification of an unrestricted hazard function.
This example separates two questions that are easy to conflate. The one-year event probability can be identifiable while a finer feature, such as the hazard in each half-year, is not. If a model imposes a constant hazard, the single-interval probability can identify that model’s constant rate in this simple setting. But that conclusion follows from the restriction to a constant hazard; it is not evidence that the data identified the hazard’s shape without that restriction.
Distinct Continuous-Time Markov Models With the Same Matrix
The same issue occurs even within a finite-state, time-homogeneous continuous-time Markov model. Let \(Q\) be its generator: the off-diagonal entry \(q_{ij}\) is the instantaneous transition intensity from state \(i\) to state \(j\), and each diagonal entry is \(q_{ii}=-\sum_{j\ne i}q_{ij}\). The transition matrix over time \(\Delta\) is \(P(\Delta)=\exp(Q\Delta)\). Observations at a particular interval length reveal this matrix, not necessarily a unique generator.
Worked Example: Two Three-State Generators Agree at One Visit Gap
Consider three states labelled 0, 1, and 2 arranged in a cycle. From each state, transitions occur to the next state around the cycle at rate \(a\), and to the previous state at rate \(b\). For example, from state 0 the rates to states 1 and 2 are \(a\) and \(b\), respectively. Suppose the only interval between visits is one time unit.
In both models below, let \(a+b=10\). Model A has \(a=b=5\). Model B has \(a-b=4\pi/\sqrt{3}\approx 7.255\), so \(a=(10+4\pi/\sqrt{3})/2\approx 8.628\) and \(b=(10-4\pi/\sqrt{3})/2\approx 1.372\). Both models have positive rates and are distinct.
The eigenvalues of the cyclic generator are \(0\) and \(-3(a+b)/2 \mathbin{\pm} i\sqrt{3}(a-b)/2\), where \(i^2=-1\). For Model A, the two nonzero eigenvalues are both \(-15\). For Model B, they are \(-15\mathbin{\pm} i2\pi\). Exponentiating over one time unit gives \(1\) for the zero eigenvalue and \(e^{-15}\) for each of the other two eigenvalues in both models, because \(e^{\pm i2\pi}=1\). The generators have the same eigenvectors, so their matrix exponentials at that interval are identical:
where \(I\) is the \(3\times3\) identity matrix and \(J\) is the \(3\times3\) matrix of ones. For example, a diagonal transition probability is \(1/3+(2/3)e^{-15}\approx0.3333335\); an off-diagonal probability is \(1/3-(1/3)e^{-15}\approx0.3333332\). The tiny difference from \(1/3\) is the same under both models. Thus, observations only at gaps of one time unit cannot distinguish these two different sets of instantaneous rates.
How a Different Schedule Can Help
Observational equivalence is schedule-specific. A visit at a different time can reveal transition probabilities at another interval length and separate models that agree at the original one. It may resolve a particular ambiguity without establishing that every parameter in a broader model is identifiable.
Worked Example: An Intermediate Visit Separates the Cyclic Models
Use the two hypothetical cyclic models from the previous example, but now observe the process at times 0, 0.5, and 1 instead of only at times 0 and 1. Over a half-unit interval, Model A has nonzero generator eigenvalues \(-15\), so the corresponding transition-matrix eigenvalues are \(e^{-7.5}\). Model B has eigenvalues \(-15\mathbin{\pm} i2\pi\), giving transition-matrix eigenvalues
The two half-unit transition matrices are therefore different. For instance, their traces differ: the trace is \(1+2e^{-7.5}\) under Model A and \(1-2e^{-7.5}\) under Model B. Since the trace is a matrix summary, unequal traces prove the matrices are unequal. A visit at time 0.5 can therefore distinguish this pair of models in principle, provided the sample contains enough information to estimate the relevant transition probabilities. It does not establish that every possible pair of generators can be separated by this schedule.
Justification: What the Data Can Distinguish
The logic behind these examples can be stated directly. Let \(\mathcal{O}\) denote the mapping from a model parameter \(\theta\) to the probability distribution of the recorded visit data under a fixed schedule. Identifiability requires that this mapping be one-to-one for the parameter of interest. If \(\mathcal{O}(\theta_1)=\mathcal{O}(\theta_2)\) for \(\theta_1\ne\theta_2\), no statistic calculated solely from those observations can reliably choose between \(\theta_1\) and \(\theta_2\) in general: the observations have exactly the same distribution under both.
For a homogeneous Markov model, the schedule supplies matrices \(P(\Delta_r;\theta)\) for its observed gaps \(\Delta_r\). If two parameter values give the same matrices at every gap used, the product formula for each observed state sequence gives the same probability under both parameters, assuming the same initial-state distribution. Hence the entire distribution of observed sequences is the same. Conversely, different matrices at a used gap indicate a possible source of distinction, but whether the full parameter is identifiable still depends on which states are observed, the initial states represented, and whether the observation probabilities uniquely determine the parameter within the model family.
This is a structural argument: it concerns exact probability distributions, not whether a particular finite sample happens to look similar. Even when a model is structurally identifiable, estimates can be imprecise if two models yield probabilities that are very close or if relevant visit patterns are rare. That is practical difficulty rather than exact nonidentifiability.
Diagnostics, Alternatives, and Limitations
Identifiability should be examined before interpreting fitted continuous-time parameters. Useful checks include comparing the number and spacing of distinct visit gaps, determining whether the recorded states distinguish the transitions of interest, and searching for different parameter values that produce the same observed transition probabilities. In parametric models, numerical fits from multiple starting values can reveal flat or ridged likelihoods, but agreement between optimizers does not prove identifiability. Nor does a finite standard error or a software convergence message establish that the model’s parameterization is uniquely determined by the design.
The main alternatives answer different questions:
- Model the observed interval transitions directly. A discrete-time transition model targets probabilities between visits and need not claim that the underlying instantaneous rates have been recovered. Its interpretation is tied to the visit interval and may need to vary when gaps differ.
- Impose scientifically justified restrictions. For example, a constant hazard or a particular intensity structure may reduce ambiguity. Such assumptions can make a parameter identifiable within the restricted family, but the resulting identification is conditional on those assumptions.
- Change or enrich the observation design. Additional visits, varied interval lengths, observing more states, or recording transition causes can provide distinctions unavailable under a sparse schedule. The benefit must be checked for the model at hand; more visits do not automatically identify an unrestricted process.
The central limitation is that panel data describe states at visits, while many continuous-time quantities concern unobserved paths between visits. The introductory material on interval-censored event times and panel histories provides the setting for this distinction: identifying a visit-to-visit probability is not the same as identifying event timing, transition order, or an instantaneous intensity. Identifiability analysis does not decide which model is scientifically credible, and it cannot compensate for misspecified states, dependent observation times, or measurement error. It clarifies what the chosen design and assumptions can support.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- With only baseline and a one-year visit in the absorbing-event example, what quantity is shared by all hazard functions that give the same event probability?
- Why can the two cyclic generators in the worked example have identical transition matrices after one time unit despite having different rates?
- Does a visit at 0.5 prove that every possible continuous-time generator is identifiable? Explain.
- Give one reason a structurally identifiable model may still produce unstable estimates in a finite sample.
Solutions. (1) They share the one-year cumulative hazard, since \(P_{00}(0,1)=\exp\{-\int_0^1\lambda(u)\,du\}\). (2) Their nonzero eigenvalues differ in imaginary part by \(\mathbin{\pm}i2\pi\), which disappears on exponentiation over one time unit. (3) No. It separates the particular pair shown; identification of a full model family requires checking that the observation map is one-to-one for that family. (4) For example, two distinguishable models may nevertheless produce probabilities that are very close, so a finite sample may contain little information to separate them.