Why Qualitative Evidence Belongs in Mechanism Comparison
Quantitative measurements are not the only observations that distinguish biological mechanisms. A study may report that a biomarker increased rather than decreased, that a response crossed a clinically meaningful threshold, or that a pathway component was detected in one condition but not another. These observations can be informative even when an exact value is unavailable. The challenge is to represent them without treating an uncertain or coarse observation as more precise than it is.
The goal is to compare how well competing mechanisms predict the evidence actually observed. A qualitative statement is not automatically a hard rule: “increased” may be uncertain, and “not detected” does not necessarily mean “absent.” Conversely, a genuine biological constraint, such as a concentration that must be nonnegative, should not be weakened into an ordinary preference.
Representing Evidence on a Common Scale
For a quantitative observation \(y\), each mechanism supplies a probability density or probability mass for the observed value. For qualitative evidence, it supplies the probability of the observed category or event. In either case, the evidence is evaluated under an explicit model for what is observed, not merely by reading a biological claim as a certainty.
Suppose \(E_q\) is a qualitative observation and \(E_c\) is a quantitative observation. If they are conditionally independent given each mechanism and its specified parameters, their joint likelihood is the product of their likelihood contributions:
It follows that the joint likelihood ratio is the product of the separate ratios. Equivalently, log-likelihood ratios add. This is a direct consequence of the factorization: dividing the joint likelihood under \(M_A\) by that under \(M_B\) gives one ratio for each conditionally independent evidence source. If the observations share an assay, subjects, or unmodelled biological variability, that factorization may be wrong; use a joint observation model instead.
A directional result can be represented by the probability of the reported direction. A threshold result such as “below 5” uses the probability of the entire event below 5, not the density at 5. A presence–absence result uses the probability of detection or non-detection, including assay sensitivity and specificity when relevant. These choices make the coarseness of the evidence explicit.
Worked Example: Combining a Biomarker Value and a Binary Assay
Hypothetical data. Two mechanisms predict a continuous biomarker measurement \(Y\) with a normal observation model and common standard deviation 2 units. Under \(M_A\), the mean is 11 units; under \(M_B\), it is 8 units. The observed value is 12 units. A separate assay is positive in 8 of 10 samples. For this assay, suppose each sample is independently positive with probability 0.8 under \(M_A\) and 0.3 under \(M_B\). Assume the continuous measurement and the assay results are conditionally independent under either mechanism.
For a normal observation with standard deviation \(\sigma\), the density at \(y\) is proportional to \(\exp(-(y-\mu)^2/(2\sigma^2))\); the common normalizing constant cancels in the ratio. Thus, for the continuous value,
The exact value 12 is half a standard deviation above the mean under \(M_A\), but two standard deviations above the mean under \(M_B\); its density is therefore higher under \(M_A\). The continuous measurement favors \(M_A\) by a likelihood ratio of about 6.521.
For the binary assay, the observed result is 8 positives and 2 negatives. Under the stated independent Bernoulli model, its likelihood ratio is
By conditional independence, the joint ratio is
The combined observations favor \(M_A\) over \(M_B\) by a likelihood ratio of approximately 1361.18 under these specified models. This does not mean that \(M_A\) has a 1361-fold probability of being true, nor does it account for prior probabilities or uncertainty in fitted parameters. It is a comparison of the likelihoods of these observations under the stated alternatives.
Directional and Threshold Evidence
A direction-only report discards the magnitude of a change. That loss of information may be unavoidable when only a qualitative summary was recorded, but the analysis should use the event that was actually observed. For example, if the report says “increased,” the likelihood contribution is the probability of an increase under each mechanism. It is not the density at an invented numerical change.
Worked Example: An Independent Directional Observation
Hypothetical data. Mechanism \(M_A\) predicts a continuous endpoint with distribution \(Y\sim N(2,1)\), and \(M_B\) predicts \(Y\sim N(0,1)\), with units of change in a biomarker per hour. A quantitative assay records \(Y=3\). A separate blinded tissue assessment reports that a second marker increased; the probability of that directional result is 0.9 under \(M_A\) and 0.2 under \(M_B\). Assume the assays are conditionally independent under each mechanism.
For the quantitative value, the common standard deviation is 1, so
The separate directional assessment contributes \(0.9/0.2=4.5\). Therefore,
The combined evidence favors \(M_A\) by a likelihood ratio of approximately 245.7. The independence assumption matters: if “increased” were simply a qualitative recoding of the same \(Y=3\) measurement, multiplying by 4.5 would count the same evidence twice. In that case, use the quantitative value alone or construct a joint model that accounts for how the qualitative report was generated.
Threshold reports are handled in the same way. If a report says that \(Y\) is below a threshold \(c\), then the relevant probability under a continuous predictive distribution is \(P(Y<c\mid M_j)\). The observed event covers a range of possible values, so using a density evaluated at \(c\) would answer a different question.
Worked Example: Evidence That a Response Fell Below a Threshold
Hypothetical data. A report gives only whether a response was below 5 units. Under \(M_A\), the response is predicted to follow \(N(3,2^2)\); under \(M_B\), it follows \(N(7,2^2)\). The observed report is “below 5 units.” Assume the threshold classification is accurate and the stated distributions include the relevant predictive uncertainty.
Standardizing the threshold gives \((5-3)/2=1\) under \(M_A\) and \((5-7)/2=-1\) under \(M_B\). Therefore,
where \(\Phi\) is the standard normal cumulative distribution function. The likelihood ratio for the reported event is
This threshold observation favors \(M_A\), but it does not reveal the exact response value or justify treating the value as 5 units. If the threshold classification itself can be erroneous, its sensitivity and specificity should be included in the observation model rather than assuming a perfect classification.
Qualitative Evidence as a Biological Constraint
Some biological knowledge is not merely probabilistic evidence. A concentration cannot be negative, and a fraction such as receptor occupancy must lie between 0 and 1. These are constraints on the domain of a valid model. They differ from a noisy observation that happens to be unusual: a hard constraint can rule out a prediction that violates a well-established physical or definitional limit.
For example, suppose a candidate mechanism necessarily predicts receptor occupancy of 1.12 at a stated condition, while occupancy is defined as a fraction of occupied receptors. That prediction is outside the allowable range \(0\leq \text{occupancy}\leq 1\). The model formulation or its parameterization is invalid for that condition; the value should not be treated as an ordinary observation with a modest penalty. If the value arises only for one parameter setting and other admissible settings remain possible, it rules out that setting, not necessarily the entire mechanism. If a noisy assay reports a value above 1, the measurement error model and the physical constraint on the latent occupancy must be kept distinct.
Presence–absence results require similar care. A “not detected” report is usually an assay outcome, not proof of biological absence. If a present analyte has detection probability \(q\), then non-detection has probability \(1-q\); the mechanism’s likelihood contribution depends on its predicted abundance and the assay’s detection characteristics. Calling every non-detection a hard zero can unfairly eliminate mechanisms that predict low but nonzero abundance.
Assumptions, Diagnostics, and Alternatives
A combined comparison is only as credible as its assumptions. Before multiplying evidence contributions, specify what each observation means, the sampling or measurement model under each mechanism, and whether observations can reasonably be treated as conditionally independent. When mechanisms have estimated parameters, the predictions should reflect the chosen parameter values and their uncertainty; a plug-in likelihood at one fitted value can understate uncertainty in the mechanism’s predictions.
Useful diagnostics include checking whether predicted directions agree with the recorded directions, whether predicted probabilities for observed categories are effectively zero, and whether simulated observations reproduce the frequency of threshold crossings or detections. For quantitative components, inspect residual patterns as well as the overall fit: a model can fit an average while systematically predicting the wrong direction in a subgroup. Examine sensitivity to plausible assay error rates and threshold definitions. A large combined likelihood ratio can be misleading if it is driven by one misspecified evidence component or by an unjustified independence assumption.
| Approach | What it uses | Main caution |
|---|---|---|
| Quantitative-only comparison | Exact numerical values and their observation model | Leaves out relevant categorical or directional observations |
| Qualitative-only comparison | Probabilities of reported categories or events | Discards magnitude information when exact values are available |
| Joint comparison | Quantitative and qualitative evidence together | Requires defensible observation models and dependence assumptions |
| Hard-constraint screening | Known physical, definitional, or biological limits | Must not confuse a genuine impossibility with a low-probability observation |
These approaches answer different questions. A joint likelihood comparison can use all recorded evidence when the model is specified adequately. A qualitative summary may be appropriate when raw measurements are unavailable, but it cannot recover information that was discarded. A hard constraint is appropriate only when violation is genuinely impossible under the definitions and domain of the system. None of these comparisons alone establishes causation or guarantees that the best-fitting candidate is biologically correct; all candidates may be inadequate.
Common Mistakes and a Practical Workflow
- Treating direction as certainty. “The marker increased” does not imply probability 1 under one mechanism and 0 under another. Specify an observation probability that allows error or biological variability.
- Double-counting a result. A direction label derived from the same continuous measurement is not an independent second data point. Model their dependence or use one representation.
- Using a threshold density. For a report \(Y<c\), calculate \(P(Y<c)\), not the density at \(c\).
- Confusing non-detection with absence. Include assay detection properties unless the observation is truly a definitive statement of absence.
- Over-interpreting the likelihood ratio. A likelihood ratio compares how well specified models predict the observed evidence; it is neither a posterior probability nor a measure of biological truth.
A disciplined analysis therefore begins by translating every record into an explicit event or value, assigns its observation model under each mechanism, checks whether evidence sources overlap, and only then combines their likelihood contributions. Biological constraints should be stated separately and used to identify predictions outside the model’s valid domain.
Check Your Understanding
Answer each question using the stated models. Full solutions follow the questions.
- For an observation \(Y=2\), suppose \(Y\sim N(1,1^2)\) under \(M_A\) and \(Y\sim N(0,1^2)\) under \(M_B\). Calculate the likelihood ratio for this quantitative observation.
- A presence result has probability 0.7 under \(M_A\) and 0.2 under \(M_B\). What is its likelihood ratio, and what does it mean?
- In a hypothetical study, the report is \(Y>4\). Under \(M_A\), \(Y\sim N(5,1^2)\); under \(M_B\), \(Y\sim N(3,1^2)\). Calculate the likelihood ratio for the reported event.
- Why is it generally wrong to multiply the likelihood for a measured value by another likelihood for a direction label computed from that same value?
Solutions. (1) The common standard deviation is 1, so \(\exp(((2-0)^2-(2-1)^2)/2)=\exp(1.5)\approx 4.482\); the observation favors \(M_A\). (2) The ratio is \(0.7/0.2=3.5\). The observed presence is 3.5 times as likely under \(M_A\) as under \(M_B\), not a 3.5-fold probability that \(M_A\) is true. (3) Under \(M_A\), \(P(Y>4)=P(Z>-1)=\Phi(1)\approx0.8413\); under \(M_B\), \(P(Y>4)=P(Z>1)=1-\Phi(1)\approx0.1587\). The ratio is \(0.8413/0.1587\approx5.301\). (4) The direction is determined by the measured value, so the two records are dependent and multiplication would count overlapping evidence twice unless a joint observation model explicitly accounts for that dependence.