Joint Hypotheses and Nested Regression Models

Why Joint Hypotheses Need Two Models

A regression question may concern several coefficients at once. For example, a pricing analyst might ask whether two risk factors and their interaction contribute anything after other predictors are included. That question is not answered by testing just one coefficient: the null hypothesis specifies a collection of restrictions, and the model under those restrictions differs from the model that allows all coefficients to vary.

The key translation is from a joint null hypothesis to a restricted regression model. The unrestricted model allows the full set of coefficients; the restricted model imposes the null conditions. The restricted model is nested in the unrestricted model when every possible fitted mean response from the restricted model is also attainable in the unrestricted model. This framework is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Definition: Suppose the regression coefficient vector is \(\beta\), with \(p\) freely estimated coefficients, including an intercept when present. A joint linear hypothesis has the form \(H_0:R\beta=r\), where \(R\) is a specified matrix and \(r\) a specified vector. The unrestricted model imposes no such restrictions; the restricted model imposes them. The restrictions are independent in number \(q=\operatorname{rank}(R)\), provided they are consistent.

From Coefficient Statements to Model Equations

A restriction on a coefficient changes which mean-response functions the model can represent. If a coefficient is fixed at zero, its predictor term disappears. If two coefficients are required to be equal, their predictor terms combine under a shared coefficient. If a coefficient is fixed at a nonzero value, its term remains in the equation with that value rather than being estimated freely.

For a model with coefficient vector \(\beta=(\beta_0,\beta_1,\ldots,\beta_k)^\mathsf{T}\), several restrictions can be written together as \(R\beta=r\). Each row of \(R\) describes one linear combination of coefficients. The number of rows is not necessarily the number of independent restrictions: duplicate or linearly dependent rows do not add another restriction. When the restricted model has a full-rank parameterization, the difference between the unrestricted and restricted numbers of free coefficients is \(q\).

Worked Example: Removing Two Predictors Jointly

A hypothetical insurer models claim cost \(Y\) using a policy-age measure \(x_1\), a vehicle score \(x_2\), and a regional indicator \(x_3\): \(Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\varepsilon_i\). The analyst wants to assess the joint null that the vehicle score and regional indicator have no linear contribution, while retaining policy age. Write the joint null, give the restricted model, and count the independent restrictions.

Step 1: State the unrestricted model. Its coefficient vector is \(\beta=(\beta_0,\beta_1,\beta_2,\beta_3)^\mathsf{T}\), so there are \(p=4\) freely estimated coefficients. The intercept and policy-age coefficient are not restricted.

Step 2: Express the null. The two statements are \(\beta_2=0\) and \(\beta_3=0\). In matrix form,

H_0: \begin{pmatrix} 0&0&1&0\\ 0&0&0&1 \end{pmatrix} \begin{pmatrix} \beta_0\\ \beta_1\\ \beta_2\\ \beta_3 \end{pmatrix} = \begin{pmatrix} 0\\ 0 \end{pmatrix}.

Step 3: Apply the restrictions to the equation. Substitute \(\beta_2=0\) and \(\beta_3=0\): \(Y_i=\beta_0+\beta_1x_{i1}+\varepsilon_i\). This is the restricted model. It retains the intercept and policy-age term but removes the other two terms.

Step 4: Count restrictions. The two rows of \(R\) are independent, so \(\operatorname{rank}(R)=2\). The restricted equation has two free coefficients, \(\beta_0\) and \(\beta_1\), while the unrestricted equation has four. Thus \(q=4-2=2\), agreeing with the rank calculation.

Check. Setting the two removed coefficients to zero in the unrestricted equation gives exactly the restricted equation. The restricted model is therefore a subset of the unrestricted model. The common mistake this avoids is removing only one predictor and calling that the model under the stated joint null.

Nestedness and What It Implies

The idea of nesting can be seen directly through fitted mean responses. Let \(X\) be the design matrix for the unrestricted model, and let \(\mathcal{M}_U\) be the set of all vectors of fitted means \(X\beta\) as \(\beta\) varies freely. Let \(\mathcal{M}_R\) be the corresponding set when \(R\beta=r\) is imposed. Because every restricted coefficient vector is also an allowed unrestricted coefficient vector, \(\mathcal{M}_R\subseteq\mathcal{M}_U\). The restricted model cannot produce a fitted mean vector that the unrestricted model cannot produce.

This inclusion gives a useful check on model comparisons. For a fixed data set and least-squares fitting, the unrestricted model's residual sum of squares cannot exceed the restricted model's residual sum of squares. The reason is that least squares minimizes the residual sum of squares over the available parameter values, and the unrestricted model has all restricted choices available plus possibly more. This fact establishes the direction of the comparison; it is not, by itself, a significance test.

\min_{\beta:\,R\beta=r}\sum_{i=1}^{n}(Y_i-\widehat{Y}_{i,R})^2 \ \geq\ \min_{\beta}\sum_{i=1}^{n}(Y_i-\widehat{Y}_{i,U})^2.

For classical finite-sample regression inference, the usual model assumes \(Y=X\beta+\varepsilon\), a fixed design matrix of appropriate full column rank, and errors that are independent with distribution \(N(0,\sigma^2)\), with a common variance \(\sigma^2\). The joint hypothesis concerns the mean-model coefficients, not a claim that the observed responses themselves satisfy the restrictions exactly. The restricted model is a fitting device representing what the mean structure would be if the null were true.

A joint test is needed to assess the restrictions as a set. Testing each coefficient separately does not generally give the same inference: the estimates may be correlated, and a combination of coefficients can be uncertain even when individual estimates appear precise, or vice versa. The hypotheses and models must therefore be specified jointly before choosing a test statistic.

Worked Example: Equal Slopes and a Zero Effect

A hypothetical health plan models annual service use \(Y\) using two centered measures \(x_1\) and \(x_2\) of related exposures and a binary program indicator \(x_3\): \(Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\varepsilon_i\). The question is whether the two exposure measures have equal slopes and the program indicator has no additional linear effect. Translate this joint null and write its restricted model.

Step 1: State the unrestricted coefficient vector. It is \(\beta=(\beta_0,\beta_1,\beta_2,\beta_3)^\mathsf{T}\), with four free coefficients. The requested restrictions are \(\beta_1=\beta_2\) and \(\beta_3=0\).

Step 2: Write each restriction as a linear equation. The equality is \(\beta_1-\beta_2=0\); the second condition is \(\beta_3=0\). Thus

H_0: \begin{pmatrix} 0&1&-1&0\\ 0&0&0&1 \end{pmatrix} \beta = \begin{pmatrix} 0\\ 0 \end{pmatrix}.

The rows are independent: the first involves \(\beta_1,\beta_2\), whereas the second involves \(\beta_3\). Therefore \(\operatorname{rank}(R)=2\), and there are two independent restrictions.

Step 3: Substitute the null conditions. Write the shared exposure slope as \(\gamma\), so that \(\beta_1=\beta_2=\gamma\), and set \(\beta_3=0\). The mean equation becomes \(Y_i=\beta_0+\gamma x_{i1}+\gamma x_{i2}+\varepsilon_i =\beta_0+\gamma(x_{i1}+x_{i2})+\varepsilon_i\). The restricted model estimates \(\beta_0\) and \(\gamma\), giving two free coefficients.

Step 4: Check the count and nesting. The unrestricted model has four coefficients and the restricted model has two, so the dimension difference is \(4-2=2\), matching the rank. Any restricted fit is obtained from the unrestricted model by choosing equal exposure slopes and a zero program coefficient; hence it is nested.

Common mistake avoided. Testing only \(\beta_3=0\) leaves \(\beta_1\) and \(\beta_2\) unrelated, so it does not test the claim that the exposure slopes are equal. Conversely, imposing equal slopes but leaving \(\beta_3\) free omits the second part of the stated joint null.

Independent, Redundant, and Inconsistent Restrictions

The rank of \(R\), not the number of written equations, determines how many independent restrictions are imposed. This matters when a problem states the same condition in more than one way. If the equations are consistent but some are redundant, the restricted model is still well-defined, but its dimension reduction is less than the number of rows in \(R\). If the equations contradict one another, there is no coefficient vector satisfying the proposed null, so it does not describe a meaningful restricted model.

Worked Example: Detecting a Redundant Restriction

A hypothetical portfolio model is \(Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\varepsilon_i\). A proposed null states \(\beta_1+\beta_2=2\), \(2\beta_1+2\beta_2=4\), and \(\beta_3=0\). Find the independent restriction count and the restricted equation.

Step 1: Write the restriction matrix. With \(\beta=(\beta_0,\beta_1,\beta_2,\beta_3)^\mathsf{T}\),

R= \begin{pmatrix} 0&1&1&0\\ 0&2&2&0\\ 0&0&0&1 \end{pmatrix}, \qquad r= \begin{pmatrix} 2\\ 4\\ 0 \end{pmatrix}.

Step 2: Identify dependence. The second row of \(R\) is twice the first, and the second entry of \(r\) is also twice the first. Thus the second equation repeats the first rather than adding a new condition. The first and third rows are independent, so \(\operatorname{rank}(R)=2\), not 3.

Step 3: Substitute the independent conditions. From \(\beta_1+\beta_2=2\), write \(\beta_2=2-\beta_1\). From \(\beta_3=0\), remove the \(x_3\) term. The unrestricted equation becomes

Y_i=\beta_0+\beta_1x_{i1}+(2-\beta_1)x_{i2}+\varepsilon_i =\beta_0+2x_{i2}+\beta_1(x_{i1}-x_{i2})+\varepsilon_i.

Step 4: Count free coefficients. The restricted equation estimates \(\beta_0\) and \(\beta_1\), so it has two free coefficients. The unrestricted model has four; the reduction is \(4-2=2\), in agreement with the rank. The fixed contribution \(2x_{i2}\) is not an additional estimated coefficient.

Check. The restricted equation satisfies \(\beta_1+\beta_2=2\), its doubled version, and \(\beta_3=0\). Counting three written equations as three restrictions would be the error; the rank and dimension checks both give two.

A Reliable Translation Process

For a joint-hypothesis problem, write the requested conditions before changing the model equation. Then convert them to linear equations, check independence, and substitute them into the unrestricted equation. This order makes it less likely that a coefficient is accidentally left free or that a redundant condition is counted twice.

Record the full model.
List the predictors and coefficient vector, including the intercept if the model has one. This establishes what the unrestricted model permits.

Translate the null literally.
Write each equality or fixed-value condition as a linear equation in the coefficients, and assemble the equations as \(R\beta=r\).

Check consistency and rank.
Confirm that the equations can hold simultaneously. Use \(\operatorname{rank}(R)\) to count independent restrictions rather than merely counting rows.

Construct the restricted equation.
Substitute the null conditions and identify which terms disappear, combine, or become fixed contributions.

Verify nesting and dimensions.
Check that every restricted parameter choice is allowed by the unrestricted model and, when both parameterizations are full rank, that the reduction in free coefficients equals \(\operatorname{rank}(R)\).

Common Traps

Testing only part of the null. If the null sets two coefficients to zero, a restricted model that sets only one to zero is not the model under the joint null.
Confusing “equal” with “both zero.” The restriction \(\beta_1=\beta_2\) allows both to be nonzero. It combines predictor terms under a shared coefficient; it does not remove them.
Counting equations instead of independent restrictions. A duplicate or scalar multiple of another consistent equation adds no new restriction.
Counting fixed contributions as estimated coefficients. If \(\beta_2=2\), the term \(2x_2\) remains in the mean equation, but 2 is not a freely estimated coefficient under the null.
Replacing a joint question with separate tests. Separate coefficient tests do not, in general, determine the joint inference because the coefficient estimates can be correlated.

Key takeaway: A joint linear null \(R\beta=r\) defines a restricted model by limiting the coefficient choices allowed in the unrestricted model. Check the actual reduced equation and use \(\operatorname{rank}(R)\) to count independent restrictions.

Check Your Understanding

For each problem, identify the restricted model and the number of independent restrictions. Assume the stated unrestricted model has full column rank.

A model is \(Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\varepsilon_i\). Under \(H_0\), \(\beta_2=\beta_3=0\). Give the reduced equation and restriction count. (Answer: \(Y_i=\beta_0+\beta_1x_{i1}+\varepsilon_i\); 2 independent restrictions.)
A model is \(Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\varepsilon_i\). Under \(H_0\), \(\beta_1+\beta_2=0\) and \(\beta_3=1\). Rewrite the restricted equation using \(\beta_1\) as the free slope parameter. (Answer: \(Y_i=\beta_0+\beta_1(x_{i1}-x_{i2})+x_{i3}+\varepsilon_i\); 2 independent restrictions.)
A five-coefficient model has restrictions \(\beta_1-\beta_2=0\), \(3\beta_1-3\beta_2=0\), and \(\beta_4=0\). How many independent restrictions are imposed? (Answer: 2; the second equation repeats the first up to a nonzero multiplier.)
A model is \(Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i1}x_{i2}+\varepsilon_i\). The null sets \(\beta_2=\beta_3=0\). State the restricted equation and explain whether it is nested in the unrestricted model. (Answer: \(Y_i=\beta_0+\beta_1x_{i1}+\varepsilon_i\); yes, it is nested because it is obtained by setting two unrestricted coefficients to zero.)
A four-coefficient model is given the restrictions \(\beta_1+\beta_2=1\), \(2\beta_1+2\beta_2=3\). Are these redundant consistent restrictions, and does a restricted model exist? (Answer: No. The left side of the second condition is twice the first left side, but its right side is not twice 1; the restrictions are inconsistent, so no restricted model exists.)

What You'll Learn