Defining Sample Size for BIC in NLME

Why the BIC Sample Size Needs a Definition

The Bayesian information criterion is commonly written as \(\mathrm{BIC}=-2\ell(\hat{\theta})+k\log(n)\), where \(\ell(\hat{\theta})\) is the maximized log likelihood, \(k\) is the number of estimated parameters, and \(n\) is the sample size used for the penalty. In a simple setting with one independent observation per participant, the count is usually clear. In a nonlinear mixed-effects (NLME) analysis, one participant may contribute many correlated concentration measurements. The number of participants, the number of observed records, and the number of scheduled occasions are then different quantities.

The choice is consequential: changing \(n\) changes the penalty for model complexity, and therefore can change which model has the lower BIC. The relevant question is not just how many rows appear in the data file. It is what independent sampling units support the likelihood approximation used to motivate BIC.

Subjects and Records in the Marginal Likelihood

Let \(y_{ij}\) denote the response for subject \(i\) at observation \(j\), and let \(b_i\) denote that subject's random effects. The population parameters, including fixed effects, variance parameters, and residual-error parameters, are collected in \(\theta\). Under a standard NLME formulation, observations within a subject are modeled conditional on \(b_i\), and the random effects are integrated out to obtain that subject's marginal likelihood contribution:

Here \(n_i\) is the number of observed records for subject \(i\), \(f\) is the conditional response density, and \(g\) is the random-effects density. If the subject vectors are independent, the full marginal likelihood is \(L(\theta)=\prod_{i=1}^{m}L_i(\theta)\), where \(m\) is the number of subjects. Thus the log likelihood is a sum of \(m\) subject-level contributions, even though each contribution may involve several measurements.

When the number of subjects grows and the within-subject design is suitably controlled, the information about population parameters accumulates across subjects. This is the setting that supports using \(m\), rather than the total number of rows, in the standard BIC penalty. The argument depends on the independent-subject structure; it is not a claim that individual records contain no information.

Why the Subject Count Appears in the Penalty

A brief Laplace-approximation argument explains the role of the independent-unit count. Suppose the model is regular near its maximum, has \(k\) estimable parameters, and the subject contributions are independent. If there are \(m\) subjects, the log likelihood is a sum of \(m\) contributions. Its local curvature in the \(k\) parameter directions therefore grows on the order of \(m\). In the Laplace approximation to the integrated likelihood, the determinant of this curvature contributes a factor of approximately \(m^{-k/2}\).

Multiplying by \(-2\) gives the BIC form \(-2\ell(\hat{\theta})+k\log(m)\), apart from terms that do not grow with \(m\). This is a justification for the subject count when subjects are the independent units in the relevant asymptotic regime. It is not a universal proof that every NLME BIC must use subjects: a different sampling design or likelihood construction can require a different analysis.

How the Choice Can Change a Model Ranking

For two candidate models fitted to the same response data with the same likelihood convention, write their parameter counts as \(k_A\) and \(k_B\). Their BIC difference is \(\mathrm{BIC}_B-\mathrm{BIC}_A=[-2\ell_B+2\ell_A]+(k_B-k_A)\log(n)\). The likelihood improvement of the more complex model competes with a penalty that depends on both the parameter-count difference and the chosen \(n\). If both models have the same \(k\), changing \(n\) adds the same penalty to both and cannot change their ranking.

Conditions, Diagnostics, and Alternatives

The subject-count convention is most defensible when the analyzed likelihood is a marginal likelihood with independent subject contributions, the candidate models use the same subjects and response data, and the number of subjects is the relevant quantity growing in the asymptotic argument. Within-subject measurements may be correlated after integrating over random effects; this does not invalidate subject-level independence. Dependence between subjects, such as shared unmodeled clusters, can invalidate the simple factorization and requires the sampling structure to be reconsidered.

Before comparing BIC values, check the following:

• Identify the likelihood. Determine whether the reported likelihood is marginal over random effects or is conditional on subject-specific estimates. A BIC derivation for one likelihood construction does not automatically justify the same sample-size choice for another.
• Identify the independent units. Confirm whether subjects are independent, or whether the design has a higher-level cluster that induces dependence across subjects.
• Check data comparability. Models should be fitted to the same response records and under compatible likelihood definitions. Different missing-data subsets can make a numerical BIC comparison misleading.
• Inspect the implementation. Software may define \(n\) as the number of observations, records, or subjects. Confirm the definition rather than inferring it from the displayed BIC formula alone.
• Report the convention. State the actual count, such as “BIC uses \(n=75\) independent subjects,” and report enough information to reproduce the penalty.

The main alternative is to use the total number of observed response records, often because an implementation defines BIC that way or because the analyst is using a record-level asymptotic argument. That convention is not automatically justified by having many rows: records from one subject share random effects and are generally not independent marginal observations. The number of scheduled occasions is another possible count, but it is not the same as the observed sample size when visits are missed or measurements are not collected. Neither alternative should be adopted solely because it produces a preferred model.

There are important limits to the standard subject-level argument. If the number of subjects is small while the number of measurements per subject grows, the information may accumulate in a way not captured by simply treating the subject count as the only relevant scale. If the likelihood is approximate, singular, or poorly identified, regular BIC asymptotics may also be unreliable. These situations call for careful qualification of the criterion, not an unexamined switch between counts. The choice of \(n\) does not correct a misspecified model, nonconvergence, or a poor likelihood approximation.

Common Mistake

A frequent error is to say, “There are 480 rows, so \(n=480\),” without considering that those records come from repeated observations on fewer subjects. Another is to assume that BIC must use subjects in every NLME analysis, regardless of the likelihood definition or software convention. The correction is to identify the independent unit supporting the likelihood approximation, inspect how the likelihood and BIC are implemented, and state the convention explicitly. A comparison is interpretable only when its likelihood, parameter count, data, and sample-size convention are all clear.