Iterative Proportional Fitting for Loglinear Models

Why Adjust Fitted Margins?

For many loglinear models, the sufficient statistics are selected marginal totals. A finite maximum-likelihood fit reproduces those observed totals; if the MLE does not exist in the model’s parameter space, the observed margins need not be attained by a finite fitted table, but the model also constrains the pattern of counts across cells. Iterative proportional fitting (IPF) constructs a fitted table by adjusting its current cell means so that one required margin at a time matches the observed margin.

The method is useful because each adjustment is simple: within a stratum of the selected margin, multiply every current fitted count by the same observed-to-fitted ratio. Repeating these updates builds a table with the structure of the model while matching its required margins. The sequence of updates matters for intermediate tables; after one margin is matched, another update can disturb it, so margins must be checked again.

Definition and Update Rule

For a hierarchical loglinear model, the selected margins correspond to its generating class: the maximal variable sets in the model. For example, the model \([AB][AC]\) has generating margins \(AB\) and \(AC\), and includes their lower-order terms. It does not include a \(BC\) interaction. The model \([AB][AC][BC]\) has all three two-factor generating margins, but no three-factor interaction.

Assume for the update rule that all cells are allowed and that the starting means and target margins are positive. Then every denominator is positive, and every update is well-defined and positive. Zero margins and structural zeros require care: a multiplier of zero can remove cells from the support, and the positive-table arguments below do not cover that case.

Why the Update Has This Form

The formula is not an arbitrary rescaling. Under independent Poisson sampling, the log-likelihood, apart from terms that do not involve the means, is $$ \ell(m)=\sum_i \{y_i\log(m_i)-m_i\}. $$ Suppose the current means have the form required by a loglinear model, and consider changing the parameter associated with margin \(S\) by adding a separate amount \(\gamma_s\) for each category combination \(s=i_S\). In the cells of stratum \(s\), this multiplies each current mean by \(e^{\gamma_s}\). Holding the other model terms fixed, the part of the log-likelihood depending on \(\gamma_s\) is $$ \ell_s(\gamma_s) = y_s\gamma_s-e^{\gamma_s}q_s+\text{constant}, $$ where \(y_s=y_S(s)\) and \(q_s=m^{\mathrm{old}}_S(s)\).

Proof. Differentiating the displayed log-likelihood contribution gives $$ \frac{d\ell_s}{d\gamma_s}=y_s-e^{\gamma_s}q_s. $$ Setting this derivative to zero yields \(e^{\gamma_s}=y_s/q_s\). The second derivative is \(-e^{\gamma_s}q_s<0\), so this stationary point is the unique maximum. Applying this calculation separately to every stratum \(s\) gives the stated cellwise multiplier. \(\square\)

The update matches the selected margin immediately: summing the new means over the cells in stratum \(s\) gives $$ m^{\mathrm{new}}_S(s) = \frac{y_S(s)}{m^{\mathrm{old}}_S(s)} m^{\mathrm{old}}_S(s) = y_S(s). $$ This identity explains the name “proportional fitting”: each stratum is scaled in proportion to its observed-to-fitted margin ratio.

Worked Example: Two-Way Independence

Worked Example: The Model \([AB][AC]\)

Worked Example: A Three-Way Model with All Pairwise Margins

Likelihood Behavior and Practical Use

For positive observed margins and positive current fitted margins, an IPF update cannot reduce the Poisson log-likelihood. To see this, consider a single stratum with observed total \(y>0\) and current fitted total \(q>0\). Its multiplier is \(r=y/q\). The change in log-likelihood from scaling the cell means in that stratum by \(r\) is $$ y\log(r)-(r-1)q = y\log(y/q)-y+q. $$ Writing \(x=y/q\), this equals \(q(x\log x-x+1)\), which is nonnegative for \(x>0\): the function \(g(x)=x\log x-x+1\) has derivative \(\log x\), has its minimum at \(x=1\), and \(g(1)=0\). Summing these nonnegative changes over the strata proves that the update does not reduce the likelihood. Equality holds when every current stratum total already matches its target. \(\square\)

In practice, cycle through the generating margins and stop when every required margin is sufficiently close to its observed counterpart. One possible check is to compute, for every target margin, the largest absolute difference between fitted and observed marginal totals. The tolerance should be chosen relative to the scale of the counts and the accuracy needed. A matched margin is not, by itself, evidence that the fit is complete: check all generating margins after a full cycle.

The examples also show why a statement such as “the last margin update always leaves earlier margins unchanged” is false in general. In the \([AB][AC]\) example, the numbers happened to preserve the \(AB\) margins after the \(AC\) update. That preservation was verified from the resulting cell values; it should not be assumed in a different table. In general, continue cycling and recheck.

Common Mistakes

• Using the wrong denominator. Divide the observed margin by the current fitted margin in the same stratum. In the \([AB][AC]\) example, the current \(AC\) margins for \(A=2\) were 7 and 7, so the update factors were \(5/7\) and \(9/7\), not ratios with denominator 5.
• Applying a margin factor to only one cell. The factor for a margin stratum multiplies every cell that contributes to that marginal total.
• Assuming a single pass always suffices. A later margin adjustment can disturb an earlier one. The required check is whether all generating margins match after the updates.
• Fitting margins that are not in the model. For \([AB][AC]\), the required margins are \(AB\) and \(AC\), not \(BC\). The generating class determines which margins IPF targets.