Normalizations And Parameter Invariance

Why Structural Models Need Normalizations

A structural model connects parameters to a probability distribution for observable data. But its parameters do not always have a unique numerical representation: a change in scale, origin, or labels can leave every observable implication unchanged. In that case, reporting a parameter value without stating its normalization can be misleading.

A normalization chooses one representation from a set of equivalent representations. It is not automatically an economic claim or a source of information in the data. The central task is to determine whether a proposed normalization merely fixes a convention, or instead removes models that make different observable or economic predictions.

Equality of observable distributions is stronger than matching one sample moment: it means the model assigns the same probabilities to all observable events. A normalization can make parameters easier to describe, but it does not by itself show that the data distinguish the normalized parameter from other economically different possibilities. The earlier material on observational equivalence provides the relevant distinction: different structural descriptions may have identical observable implications.

Scale Invariance: The Units of a Latent Factor

Suppose a model has an unobserved factor \(F\) and two observed measurements: \(Y_1=\lambda_1F+\varepsilon_1\) and \(Y_2=\lambda_2F+\varepsilon_2\). Assume \(F\), \(\varepsilon_1\), and \(\varepsilon_2\) are mutually independent, have mean zero, and are normally distributed. Let the errors have positive variances. These assumptions imply a joint normal distribution for \((Y_1,Y_2)\), so its mean vector and covariance matrix determine its observable distribution.

For any nonzero constant \(c\), define \(F^*=cF\) and \(\lambda_j^*=\lambda_j/c\) for \(j=1,2\). The measurement equations remain unchanged, since \(\lambda_j^*F^*=(\lambda_j/c)(cF)=\lambda_jF\). The latent factor's variance changes to \(c^2\operatorname{Var}(F)\), but the distribution of the observed measurements does not change. Thus, the magnitude assigned to the factor and the magnitudes assigned to its loadings depend on the chosen unit for \(F\).

A sign is also part of the scale convention: choosing \(c=-1\) reverses the sign of the factor and of every loading without changing the measurements. A common convention fixes one loading as positive, or sets the factor variance to one and fixes a loading's sign. Such a sign convention is useful for comparing parameterizations. It does not make a substantively positive relationship true if the direction of the factor has not been given an external meaning.

This argument depends on the factor being latent and its scale not being independently anchored. If an observed variable defines the scale, or if external information imposes a meaningful unit, rescaling the latent construct while holding that anchor fixed may change the model. In that setting, treating the scale as arbitrary would not be justified.

Location Invariance: The Origin of a Latent Outcome

A parallel issue arises when only a thresholded latent outcome is observed. Suppose \(Y^*=\alpha+\beta X+U\), but the recorded outcome is \(Y=1[Y^*>\tau]\). Assume \(U\) has a logistic distribution with mean zero and scale one, independently of \(X\). Then the conditional success probability depends on \(\alpha\) and \(\tau\) through their difference: \(P(Y=1\mid X=x)=\{1+\exp[-(\alpha+\beta x-\tau)]\}^{-1}\).

Adding the same constant \(d\) to \(\alpha\) and \(\tau\) leaves \(\alpha+\beta x-\tau\) unchanged for every \(x\). The model's probabilities therefore do not reveal the absolute origin of the latent outcome. Fixing the threshold at zero is one possible location normalization; an alternative parameterization could fix the intercept or otherwise set an origin.

The assumptions matter. If the threshold is externally known in a meaningful measurement system, it cannot be shifted freely while holding that external information fixed. Likewise, changing the distributional scale of \(U\) is not the location transformation just described. Which parameters are conventional depends on what is observed and what the model takes as fixed.

Label Invariance: The Names of Mixture Components

In a finite mixture, the component labels can be arbitrary. For example, suppose each observation \(Y\) comes from component 1 with probability \(\pi\) and from component 2 with probability \(1-\pi\). Assume the component distributions have densities \(f_1\) and \(f_2\). The observable density is \(\pi f_1(y)+(1-\pi)f_2(y)\). Swapping the component labels and their associated weights leaves this density unchanged. The labels “1” and “2” do not, by themselves, identify substantively different groups.

An ordering rule is only a naming convention when the component parameters are distinct and the rule can be applied. If two component means coincide, ordering by mean does not separate their labels; imposing a strict order could even exclude that parameter configuration. More generally, labeling components does not establish that a mixture is otherwise uniquely determined by its observable distribution. It resolves only the redundancy created by interchangeable labels.

Harmless Conventions Versus Substantive Restrictions

The key distinction is whether a change of parameter values preserves the model's observable implications. A pure normalization selects among descriptions that already imply the same distribution. A substantive assumption rules out some possibilities with different implications, based on economic structure, design knowledge, or maintained theory. It may be defensible, but it is not merely a choice of units or names.

A short algebraic check is often enough to recognize a proposed normalization. Write the observable distribution as \(P_\theta\), and consider a transformation \(g\) of the parameter vector. If the model allows the transformed vector and \(P_{g(\theta)}=P_\theta\) for all observable events, then the transformation describes an invariance. Restricting attention to one representative can tidy the parameterization, but cannot create information that distinguishes representatives. If the equality fails, the restriction changes observable implications and needs substantive justification rather than being presented as a harmless convention.

In practice, state the normalization alongside every parameter interpretation. “The factor loading is 4” is incomplete if the latent factor's unit is unspecified; “the loading is 4 under a unit-variance factor normalization” identifies the convention. Similarly, a mixture component's label has no inherent meaning unless an external definition attaches it to a named population or mechanism.

Common Pitfall

A frequent error is to say, “The model is identified because I fixed one parameter.” That conclusion does not follow. Fixing a scale, origin, or label may remove a redundant representation, but it does not ensure that the remaining parameters are uniquely determined by the observable distribution. Conversely, a restriction that changes the observable distribution is not a harmless normalization merely because it makes estimation easier. First establish what transformation leaves observables unchanged; then describe the convention or restriction accurately.