Why Visit Dependence Can Be Hard to Learn
A model can allow the visit rate to depend on a patient’s outcome and still provide little evidence about the strength of that dependence. This is not a paradox: a flexible model describes possibilities, while the observed data determine how well its parameters can be estimated. If visits are rare, outcomes barely vary, or each patient has only a few measurements, many different dependence strengths may fit almost equally well.
The issue matters because the estimated visit effect can influence how longitudinal outcomes are interpreted. The earlier material on informative visits establishes that outcome-related observation can distort estimated trajectories. Here the focus is narrower: when do the observed visit times and outcomes actually contain enough information to estimate the strength of that relationship?
A simple way to make the distinction is to consider an outcome effect in a visit-rate model. Let \(K_i\) be the number of visits for patient \(i\) during an observed period of length \(T_i\). Suppose, conditional on a recorded outcome summary \(z_i\), visits follow independent Poisson distributions with mean \(\mu_i=T_i\exp(\gamma+\alpha z_i)\). Here \(\gamma\) is the log baseline visit rate and \(\alpha\) is the change in log visit rate per one-unit increase in \(z_i\). The summary \(z_i\) might represent an outcome level, in specified measurement units. This simplified model is useful for seeing the information issue; it is not a substitute for checking whether its assumptions are suitable for a particular visit process.
The model assumes the stated conditional Poisson rate is appropriate over each observation period, the exposures \(T_i\) and summaries \(z_i\) are known, and visit counts are independent across patients conditional on the model. In particular, it does not account for extra variation or unmodeled patient-specific visit propensities. If those features are important, the information calculations below need not describe the uncertainty of a better-fitting model.
Information After Allowing for the Baseline Rate
Having many visits is not enough by itself to identify an outcome effect. The outcomes must vary in a way that separates \(\alpha\) from the baseline rate \(\gamma\). This can be shown directly from the likelihood.
Under this Poisson model, the expected information matrix is
The information about \(\alpha\) must account for the fact that \(\gamma\) is estimated too. Removing the part of the information that can be explained by changing the baseline rate gives the efficient information
The last equality follows by expanding the weighted sum: \(\sum_i\mu_i(z_i-\bar z_\mu)^2 =\sum_i\mu_i z_i^2-2\bar z_\mu\sum_i\mu_i z_i+\bar z_\mu^2\sum_i\mu_i\). Substituting \(\bar z_\mu=(\sum_i\mu_i z_i)/(\sum_i\mu_i)\) yields the preceding expression. Thus information about the visit effect depends on both the expected number of visits, represented by \(\mu_i\), and the outcome variation left after accounting for the baseline. If all patients have the same \(z_i=c\), then \(\mathcal I_{\alpha\mid\gamma}=0\): the rate depends only on \(\gamma+\alpha c\), so the data cannot separate the two parameters.
When the model is regular and the information is sufficiently large, the inverse square root of this efficient information gives a rough model-based standard error for \(\hat\alpha\). This is a local approximation, not a guarantee that a normal approximation is accurate in very sparse data. A flat likelihood can also be asymmetric or bounded in ways that make a single standard error misleading.
Worked Example: No Outcome Variation Leaves the Effect Confounded
Suppose six hypothetical patients are each observed for one year, and every patient has the same outcome summary, \(z_i=2\). The visit-rate model is \(\lambda_i=\exp(\gamma+2\alpha)\) visits per year. Even if the observed visit counts are \(0,1,0,2,1,2\), what can these data tell us separately about \(\gamma\) and \(\alpha\)?
The likelihood depends on the two parameters through the combination \(\gamma+2\alpha\), not through either parameter separately. For example, replacing \(\alpha\) by \(\alpha+1\) and \(\gamma\) by \(\gamma-2\) leaves \(\gamma+2\alpha\) unchanged: \((\gamma-2)+2(\alpha+1)=\gamma+2\alpha\). The modeled visit rate and therefore the data likelihood are unchanged.
Conclusion. The common rate combination may be estimable, but the outcome effect \(\alpha\) is structurally unidentifiable when the baseline rate is also unknown and all \(z_i\) are identical. More visits from these same outcome values do not resolve this exact confounding. Variation in \(z_i\), or an external restriction on the baseline, is needed to separate the parameters.
Worked Example: Sparse Visits Give Little Information Even With Some Variation
Consider four hypothetical patients observed for one year each. Their outcome summaries are \(-0.5,-0.1,0.1,0.5\), measured in the same units, and the model’s baseline visit rate is \(0.1\) visits per year at \(\alpha=0\). Calculate the expected information for \(\alpha\) at that parameter value and its rough information-based standard error.
At \(\alpha=0\), each patient has expected count \(\mu_i=1(0.1)\exp(0)=0.1\) visits. The weighted mean of the summaries is zero, so the efficient information is
The corresponding rough standard error is \(1/\sqrt{0.052}\approx4.3853\) log-rate units per one-unit increase in the outcome summary. This is very large: despite some outcome variation, four patient-years at a low visit rate provide little expected information. It is only a local, model-based approximation; with so few expected visits, a normal approximation to the estimator may be poor. The calculation signals weak information, not a reliable confidence interval.
Why Few Repeat Measurements Leave Latent Predictors Uncertain
In practice, the outcome that may trigger a visit is often not known continuously. It may be measured only at the visits themselves, while a visit model depends on an underlying trajectory or a patient-specific latent level. Then the outcome-related predictor is uncertain as well as the visit effect. A small number of measurements can make it difficult to tell whether a patient’s visit pattern reflects a genuinely high or low latent outcome, random measurement error, or a strong dependence parameter.
Worked Example: One Measurement Versus Four for a Latent Outcome Level
Suppose a hypothetical patient has a latent outcome level \(b_i\) with \(b_i\sim N(0,1)\). Each observed outcome is \(y_{ij}=b_i+\epsilon_{ij}\), where the independent measurement errors satisfy \(\epsilon_{ij}\sim N(0,1)\). Assume these distributions and variances are known, and explicitly set the population mean to zero. How much do one or four measurements tell us about \(b_i\)?
For \(n_i\) measurements with mean \(\bar y_i\), the normal-model posterior variance is \(\left(1+n_i\right)^{-1}\), and the posterior mean is \(\frac{n_i}{n_i+1}\bar y_i\). With one measurement \(y_{i1}=2\), the posterior mean is \(\frac{1}{2}(2-0)=1\), and the posterior variance is \(\frac{1}{2}=0.5\).
With four measurements \(1,2,1,2\), the sample mean is \(\bar y_i=(1+2+1+2)/4=1.5\). The posterior mean is \(\frac{4}{5}(1.5-0)=1.2\), and the posterior variance is \(\frac{1}{5}=0.2\). The four measurements give a more precise estimate of this patient’s latent level than one measurement does.
Interpretation. These calculations describe uncertainty about \(b_i\) under the specified normal model; they do not directly calculate the standard error of a visit-dependence estimate. They show why few measurements can leave the outcome predictor uncertain. Even with improved measurement of each patient, estimating a population-level dependence effect still requires enough patients, visits, and variation across the relevant outcome values.
The distinction between within-person and between-person information is important. Repeated measurements can clarify an individual trajectory, while the visit-effect estimate may depend on comparisons across many patients and across different outcome levels. If nearly all patients have similar outcomes, adding a few repeats for each one may reduce measurement error without providing much variation to identify the association with visit frequency.
Practical Warning Signs and Responses
Weak identification is not established by one diagnostic number. It is a reason to examine the data and model together. Useful warning signs include very few total visits, many patients with no repeat visit, little spread in the outcome summaries, and a fitted likelihood that changes only slightly over a broad range of \(\alpha\). Large standard errors, strong dependence between \(\hat\alpha\) and the fitted baseline rate, or estimates that change sharply when a few patients are removed are also evidence that the data may be doing little to determine the effect.
A model can also appear to estimate \(\alpha\) precisely because it fixes parameters, treats uncertain outcomes as known, or imposes a restrictive distribution on patient variation. Precision conditional on those assumptions is not the same as information supplied by the observations. Check whether visit counts are compatible with the assumed Poisson variation and whether the outcome summaries represent the time periods that could plausibly influence visits. A poor fit or a mismatch between the assumed and recorded outcome timing undermines the information calculation rather than proving a particular alternative dependence pattern.
| Data or modeling feature | Why it weakens interpretation | Reasonable response |
|---|---|---|
| Outcome summaries have almost no variation | The effect can be confounded with the baseline rate. | Report the lack of identifying variation; avoid presenting a stable effect estimate as data-driven. |
| Very few visits or little exposure | The likelihood contains few event contributions and may be nearly flat. | Show uncertainty transparently; use cautious inference and avoid relying on a normal approximation without support. |
| Few repeated outcomes per patient | Latent outcomes or trajectories may be imprecisely estimated. | Represent outcome uncertainty in the joint model and assess how assumptions affect the estimate. |
| Large changes under small model alterations | The reported effect may depend heavily on assumptions rather than observed information. | Compare defensible specifications and report sensitivity, rather than selecting the most favorable estimate. |
The main alternatives serve different purposes. A simpler visit model with no outcome dependence can be useful as a comparison, but it does not demonstrate that the dependence is absent. A more detailed joint model can represent uncertainty in outcomes and visit times, but extra parameters may make weak information more apparent, not make it disappear. Additional measurements or a planned schedule that supplies outcome information at times not determined by the same visit process can improve the design. Sensitivity analysis can show how conclusions change under plausible dependence strengths, but it cannot recover a uniquely data-identified strength when the observed data do not distinguish the possibilities.
A common mistake is to say, “The estimated effect is near zero, so visits are noninformative.” A near-zero estimate with a broad range of plausible values may instead mean the data cannot distinguish weak from strong dependence. The correction is to examine uncertainty and identification, state what variation and visit information are available, and avoid interpreting a lack of precision as evidence of no association. Similarly, an optimizer returning a number does not show that the likelihood contains substantial information about that number.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a log-linear visit-rate model with an unknown baseline rate, all patients have the same outcome summary. Is the outcome effect structurally identifiable from their visit counts?
- In the Poisson model above, what happens to information about the outcome effect if the expected visit counts are multiplied by four while the outcome summaries stay fixed?
- Why can four repeated measurements improve knowledge of a patient’s latent outcome without guaranteeing precise estimation of the population visit effect?
- A fitted outcome effect is near zero but has a very large standard error. Does this establish that visits are noninformative?
- Name one design change that could improve information about outcome-dependent visiting, and explain why.
Solutions. (1) No. If all summaries equal \(c\), the rate depends on \(\gamma+\alpha c\), so the baseline and effect cannot be separated without additional information or constraints. (2) Expected information is proportional to the expected counts, so it is multiplied by four; the rough standard error is divided by \(\sqrt{4}=2\), if the same model-based approximation remains appropriate. (3) Repeats can reduce uncertainty about each patient’s latent outcome, but population-effect precision also depends on the number of patients and visits and on variation in outcomes across the data. (4) No. A large standard error means the data may not distinguish a zero effect from other values; it is not evidence that the effect is absent. (5) For example, collect more outcome measurements or provide follow-up at planned times independent of symptom-triggered visits. Either can supply additional outcome information and, depending on the design, more useful variation for separating the outcome effect from the baseline visit rate.