Marginal Totals as Sufficient Statistics

Why Margins Carry Information About Loglinear Parameters

A contingency table may contain many cell counts, while a loglinear model describes those counts using a smaller collection of effects. The likelihood depends on the data through particular sums of cell counts: the margins corresponding to the effects included in the model. These margins are sufficient statistics for the model parameters. They also appear in the likelihood equations, which require the fitted margins to agree with the observed ones when a finite maximum likelihood estimate exists.

The key reason is visible in the Poisson likelihood. A loglinear model makes each cell's log mean a linear combination of model terms. Collecting the data-dependent terms in the likelihood then produces sums of counts weighted by those same terms. For categorical factors, those sums are marginal totals or contrasts of marginal totals.

Poisson Factorization and Sufficient Statistics

Suppose the table has \(m\) cells and the counts \(Y_1,\ldots,Y_m\) are independent Poisson random variables. Write their means as \(\mu_j\), and specify the loglinear model by \(\log\mu_j=x_j^{\mathsf T}\beta\), where \(x_j\) is the fixed design vector for cell \(j\) and \(\beta\) is the parameter vector. This includes a constant term when the model has an intercept. The joint probability mass function is

The data enter the parameter-dependent part only through \(T(y)=\sum_j x_jy_j\). Thus the factorization theorem identifies this vector as sufficient for \(\beta\). This conclusion holds even if some columns of the design matrix are redundant; such redundancy affects parameter identifiability, not the factorization.

To see directly why the factorization gives sufficiency, fix a possible value \(t\) of \(T(Y)\). For any table \(y\) with \(T(y)=t\), its probability has the form \(h(y)\exp\{\beta^{\mathsf T}t-\sum_j\exp(x_j^{\mathsf T}\beta)\}\). When probabilities are normalized over all tables with that same statistic \(t\), the common parameter-dependent factor cancels. The conditional probability is therefore

which does not involve \(\beta\). The denominator is a sum over a countable set of tables and is finite and positive whenever the conditioning event has positive probability. This verifies the conditional-distribution definition as well. \(\square\)

From Model Terms to Marginal Totals

For categorical factors, the design terms indicate which margins are relevant. Let the table index \(i=(i_1,\ldots,i_q)\) specify one level of each of \(q\) factors, and let \(y_i\) be its observed count. For a subset of factors \(A\), define the \(A\)-margin at level combination \(i_A\) by summing over all remaining factors:

A hierarchical loglinear model is specified by a generating class: a collection of factor subsets such that the model contains the terms for those subsets and all their subsets. For a generator \(A\), the model can include any function of the factor levels in \(A\); in a full-indicator representation, the corresponding design terms indicate each possible level combination \(i_A\). Summing each indicator times the count across cells gives exactly \(y_{i_A+}\). Therefore, the margins for the generating class provide sufficient statistics. They may contain redundant information: some margins, or totals within them, can be determined from other margins.

This formulation does not depend on a particular coding of the effects. Reference-cell or contrast coding replaces full indicators by linear combinations of them. The resulting sufficient statistics are then corresponding contrasts of margins, but the complete generator margins contain those contrasts and are a convenient way to report the information.

Likelihood Equations Match the Model Margins

The same sufficient statistics appear when differentiating the log-likelihood. Let \(X\) be the matrix whose \(j\)th row is \(x_j^{\mathsf T}\), and let \(\mu_j(\beta)=\exp(x_j^{\mathsf T}\beta)\). The log-likelihood, omitting no data-dependent terms, is

Its gradient is

Proof. At an interior maximum of a differentiable log-likelihood, every partial derivative with respect to a free parameter is zero. Applying this necessary condition to the displayed gradient gives

Now consider a design column that is the indicator of a particular level combination \(i_A\) of a generator \(A\). Its component of \(X^{\mathsf T}y\) is the sum of observed counts in cells with that combination, namely \(y_{i_A+}\). The corresponding component of \(X^{\mathsf T}\hat\mu\) is \(\hat\mu_{i_A+}=\sum_{i_{A^c}}\hat\mu_i\), the fitted margin. Equality of the vectors therefore gives equality of these observed and fitted margins for every such indicator. If effects are expressed in a reduced contrast coding, the score equations first state equality of the associated contrasts; because the model includes the lower-order terms, those equations together recover the same generator-margin equalities. The result is a necessary likelihood equation, not a claim that an estimate always exists or that matching margins alone establishes a finite maximum. \(\square\)

Common Mistakes and Practical Meaning

• Assuming the grand total is always sufficient. It is sufficient for a model with only a common mean, but an independence model also has row and column effects. The likelihood then depends on row and column totals, not only their grand total.
• Calling sufficient margins independent. Margins can share totals or determine one another in part. Sufficiency does not mean statistical independence among the summaries.
• Confusing sufficiency with fitted-cell equality. The likelihood equations match the model's sufficient margins. They do not generally set every fitted count equal to its observed counterpart.
• Using score equations without checking existence. The equations above hold at an interior finite maximum. In some data configurations a finite maximum does not exist, so one cannot assume that a finite parameter vector solving the equations is available.

Marginal sufficiency gives a compact description of how a loglinear model uses the observed table. It also explains why margins are central both for understanding parameter estimation and for checking the resulting fit: the model's fitted counts must reproduce the information in the margins associated with its generating class.