Event Counts Within Clusters And PQL Stability

Why Cluster Event Counts Matter

In binary clustered data, the number of events in each cluster is a compact description of how outcomes are distributed within clusters. A collection of clusters with mostly zero events can present a different estimation problem from one in which events occur across many clusters—even if the total number of events is the same. For penalized quasi-likelihood (PQL), those patterns matter because the binary data are represented through local working responses and weights, while random-intercept estimation depends on how much information each cluster contributes.

Counts are clues, not a stand-alone test of PQL quality. The working weights also depend on fitted probabilities, which depend on covariates, fixed effects, and random effects. A zero-event cluster can be unsurprising when its fitted probabilities are low; the same count may be more influential when the model predicts probabilities near one half. The relevant question is therefore not simply “How many zero-event clusters are there?” but “What do the observed counts imply alongside the fitted probabilities and working weights?”

Model and Count-Based Information

Consider \(m\) independent clusters. In cluster \(i\), there are \(n_i\) binary responses \(Y_{ij}\), where \(Y_{ij}=1\) denotes an event and \(Y_{ij}=0\) denotes no event. A binary random-intercept logit model is

Y_{ij}\mid b_i \ \text{independent}, \qquad P(Y_{ij}=1\mid b_i)=p_{ij}, \qquad \operatorname{logit}(p_{ij})=\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}+b_i, \qquad b_i\sim N(0,\sigma_b^2).

The covariate vector \(\mathbf{x}_{ij}\) includes any specified fixed-effect predictors. The cluster event count is \(K_i=\sum_{j=1}^{n_i}Y_{ij}\), taking values from 0 to \(n_i\). Conditional independence is assumed given the random intercept and covariates; marginally, responses in a cluster are associated through their shared \(b_i\).

Definition: A cluster has zero events if \(K_i=0\), one event if \(K_i=1\), and multiple events if \(K_i\ge 2\). A mixed-outcome cluster contains at least one event and at least one non-event, so \(1\le K_i\le n_i-1\). These categories describe observed outcomes; they do not by themselves establish whether a cluster is informative or whether PQL is adequate.

For a random-intercept model, the conditional log likelihood contribution from cluster \(i\), apart from terms not involving \(b_i\), is

\ell_i(b_i)=\sum_{j=1}^{n_i} \left\{Y_{ij}\eta_{ij}-\log\!\left(1+\exp(\eta_{ij})\right)\right\}, \qquad \eta_{ij}=\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}+b_i.

Differentiating with respect to \(b_i\) gives the score and curvature:

\frac{\partial \ell_i}{\partial b_i} =\sum_j(Y_{ij}-p_{ij}) =K_i-\sum_jp_{ij}, \qquad -\frac{\partial^2\ell_i}{\partial b_i^2} =\sum_jp_{ij}(1-p_{ij}).

The score compares the observed event count with the model’s expected count, conditional on the current random effect. The curvature is the cluster’s local likelihood information about \(b_i\). With the normal random-effect penalty, the negative second derivative of the penalized criterion is \(\sum_jp_{ij}(1-p_{ij})+1/\sigma_b^2\). The penalty stabilizes estimation when the likelihood contribution is weak, but it does not create data information.

This calculation gives an important qualification: at fixed fitted probabilities, the binary logit curvature does not directly depend on whether the observed responses are zero, one, or a mix. Event counts affect the score, and therefore the fitted random effect and subsequent fitted probabilities; those probabilities in turn determine the working weights. Count patterns are useful for diagnosing what may happen during fitting, not a substitute for inspecting the fitted weights.

How Counts Enter the PQL Approximation

For the logit link, PQL linearizes the link around the current fitted probability \(p_{ij}\). The working response and its weight are

z_{ij}=\eta_{ij}+\frac{Y_{ij}-p_{ij}}{p_{ij}(1-p_{ij})}, \qquad w_{ij}=p_{ij}(1-p_{ij}).

The local weighted fit uses \(z_{ij}\) as a response with weight \(w_{ij}\). Because \(w_{ij}\) is small when a fitted probability is near zero or one, the working response can be far from \(\eta_{ij}\) for an outcome that differs from that fitted probability. In particular, an event where \(p_{ij}\) is very small can produce a large positive working-response adjustment. A non-event where \(p_{ij}\) is very large can produce a large negative adjustment.

The following derivation links the event count to a local update. Suppose, for illustration, that a cluster has a common fitted probability \(p\), a common fixed linear predictor \(\eta\), and a current random intercept \(b=0\). The likelihood score for \(b\) is \(K_i-n_ip\); the penalized curvature at zero is \(n_ip(1-p)+1/\sigma_b^2\). A Newton step for the penalized criterion is therefore

\Delta b_i \approx \frac{K_i-n_ip}{n_ip(1-p)+1/\sigma_b^2}.

This is a local step, not a final random-effect estimate. It shows why two clusters with the same size and fitted probabilities can move in different directions: a count above the expected count pushes the random intercept upward, while a count below it pushes downward. When the likelihood information \(n_ip(1-p)\) is small, the prior penalty has a relatively large influence on this step.

Worked Example: Interpreting Counts Against the Expected Count

Suppose a hypothetical cluster has \(n_i=4\) observations, each with current fitted probability \(p=0.25\), and the random-effect variance is \(\sigma_b^2=1\). Compare a cluster with \(K_i=0\) to one with \(K_i=3\), using the local update above.

The expected count is \(n_ip=4(0.25)=1\). The likelihood information is \(n_ip(1-p)=4(0.25)(0.75)=0.75\), and the penalty contribution is \(1/\sigma_b^2=1\). Thus the denominator is \(0.75+1=1.75\).

For \(K_i=0\), the score is \(0-1=-1\), so \( \Delta b_i\approx -1/1.75=-0.5714\). For \(K_i=3\), the score is \(3-1=2\), so \( \Delta b_i\approx 2/1.75=1.143\).

The local update is negative for the zero-event cluster and positive for the three-event cluster. The larger positive update reflects that three events are two above the current expected count, whereas zero events are one below it. These are illustrative one-step updates, not estimates of the final random effects; after an update, probabilities and weights change.

Three Worked Checks of Event-Count Patterns

Worked Example: Same Total, Different Distribution Across Clusters

Consider two hypothetical data sets, each with six clusters of size four and six events in total. Their cluster event counts are \(1,1,1,1,1,1\) in the first data set and \(0,0,0,0,3,3\) in the second. Suppose, only for a local comparison, all observations currently have fitted probability \(p=0.25\), with \(\sigma_b^2=1\).

Each cluster has expected count \(4(0.25)=1\), likelihood information \(4(0.25)(0.75)=0.75\), and penalized denominator \(0.75+1=1.75\). In the first data set, every cluster has score \(1-1=0\), giving a zero first-order update for each cluster at this provisional fit.

In the second data set, each zero-event cluster has score \(-1\), giving a local update of \(-1/1.75=-0.5714\). Each three-event cluster has score \(2\), giving a local update of \(2/1.75=1.143\). The total event count alone hides this contrast: one data set distributes events evenly, while the other concentrates them in two clusters.

The calculation does not prove that either data set will yield a stable or unstable fit. It shows why counts should be examined by cluster, together with the probabilities after fitting. The provisional weights are identical here because the probabilities were held fixed; they need not remain identical after the random effects are updated.

Worked Example: How Often Are Zero-Event Clusters Expected?

As a simple reference calculation, suppose outcomes in a hypothetical cluster of size four are conditionally independent with common probability \(p=0.10\). Then \(K_i\) has a binomial distribution with parameters 4 and 0.10. The probabilities of zero and one event are

P(K_i=0)=(0.9)^4=0.6561, \qquad P(K_i=1)=\binom{4}{1}(0.1)(0.9)^3=0.2916.

Therefore \(P(K_i\ge2)=1-0.6561-0.2916=0.0523\). In 30 such clusters, the expected numbers are \(30(0.6561)=19.68\) zero-event clusters, \(30(0.2916)=8.748\) one-event clusters, and \(30(0.0523)=1.569\) clusters with multiple events.

These are expected counts under the stated conditional-independent reference model, not guarantees about a realized data set. A random-intercept model induces marginal dependence within clusters, so this binomial calculation is not the exact marginal distribution of \(K_i\) under a nonzero random-effect variance. It is still a useful check when assessing whether an abundance of zero counts is surprising under a specified conditional probability scenario.

Worked Example: A Rare Event and an Extreme Working Response

Suppose a hypothetical observation has fitted probability \(p=0.10\), so \(\eta=\operatorname{logit}(0.10)=\log(0.1/0.9)\approx-2.197\). Its working weight is \(w=0.10(0.90)=0.09\). If the observed response is an event, \(Y=1\), then

z=\eta+\frac{Y-p}{p(1-p)} =-2.197+\frac{1-0.10}{0.09} =-2.197+10 =7.803.

For a non-event at the same fitted probability, the working response is \(z=-2.197+(0-0.10)/0.09=-3.308\). Both observations have the same small weight, 0.09, but their working responses lie far apart. The example illustrates the local nature of the approximation: the working response can be extreme even though the observation’s weight is small. A single such value does not establish failure, but repeated extreme working responses or substantial changes across iterations warrant closer inspection.

Reading Counts as a Stability Diagnostic

The count distribution helps describe where information may be coming from. Zero- and all-event clusters can have scores far from their expected counts, especially when the current model assigns probabilities inconsistent with their outcomes. But if the model assigns probabilities near zero in a zero-event cluster, its count may be entirely expected. Mixed clusters often show within-cluster outcome variation, but their value for estimating a random intercept still depends on fitted probabilities and covariates. A cluster with mixed outcomes can contribute little local information if its fitted probabilities are near the boundaries.

A practical inspection should therefore combine the following summaries rather than treating any one as decisive:

Tabulate the numbers of clusters with zero, one, multiple, and all \(n_i\) events, preserving the cluster sizes.
Compare observed counts \(K_i\) with conditional expected counts \(\sum_jp_{ij}\).
Inspect the cluster information \(\sum_jp_{ij}(1-p_{ij})\), and note whether many clusters have very small values.
Check whether event counts are concentrated in a small number of clusters or are broadly distributed.
Look for changes in working responses, weights, and random-effect updates over PQL iterations.

Counts are especially difficult to interpret when clusters have unequal sizes or different covariate distributions. A count of two events has a different meaning in a cluster of three than in a cluster of thirty. Also, clusters with different exposure levels or predictor values can have different expected counts even when their random effects are identical. Comparisons should use observed and expected counts in context, not raw counts alone.

Alternatives, Limitations, and a Common Mistake

PQL uses a sequence of local weighted mixed-model approximations. A likelihood-based method that integrates over the random-effect distribution instead targets the marginal likelihood rather than relying on the same working-response construction. Such methods can provide a useful comparison, although their numerical cost and performance depend on the model and data. A count table alone cannot determine which method is preferable, nor can it correct a misspecified model or sparse design.

The number of clusters and the event-count pattern are related but distinct design features. Many clusters can provide information about the distribution of random effects, while large clusters can provide more within-cluster observations; neither guarantees strong information if fitted probabilities are extreme or events are concentrated. The count distribution is a diagnostic summary, not a replacement for examining the likelihood contribution, fitted weights, and model assumptions.

Common mistake: “Most clusters have zero events, so their PQL weights must be zero and they provide no information.” This is incorrect. A binary logit working weight is \(p_{ij}(1-p_{ij})\), not a function of \(Y_{ij}\) alone. A zero-event cluster can have nonzero weights if its fitted probabilities are interior, although its count affects the score and can move the fitted random effect. The correction is to inspect counts together with fitted probabilities, expected counts, and cluster-level working information.

Key takeaway: Cluster event counts reveal whether outcomes are spread across clusters or concentrated in a few, and they influence random-effect updates through the score. PQL stability, however, depends on the resulting fitted probabilities and working weights; counts are a warning signal to investigate, not a verdict.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In a cluster of size five, the current fitted probabilities are all 0.20 and the observed event count is two. What are the expected count and the random-intercept score?
For a logit random-intercept model, does the conditional likelihood curvature at fixed fitted probabilities depend directly on whether a response is zero or one? State the curvature.
If \(n_i=3\) and all three fitted probabilities equal 0.50, calculate the cluster likelihood information for the random intercept.
Why is a zero-event cluster not automatically evidence of poor PQL stability?

Solutions. (1) The expected count is \(5(0.20)=1\), and the score is \(K_i-\sum_jp_{ij}=2-1=1\). (2) No. The curvature is \(\sum_jp_{ij}(1-p_{ij})\), which at fixed probabilities does not directly involve the observed responses. (3) The information is \(3(0.50)(0.50)=0.75\). (4) Zero events may be expected when fitted probabilities are low; stability also depends on the fitted weights, covariates, and resulting updates. The count must be interpreted against the model’s expected count and probabilities.