Defining The Hypothesis Family For FDR Control

Why the Family Must Be Defined

A genomic analysis may test thousands of genes, several treatment contrasts, multiple tissues, or several outcomes. Before applying false discovery rate (FDR) control, the analyst must specify which null hypotheses belong to the set of claims being assessed. That set is the hypothesis family. Defining it is a scientific decision: it determines what “the FDR is at most the target” refers to.

The earlier material on false discovery proportion and false discovery rate introduced the error measure. Here we focus on its scope: how the family is formed, how alternative boundaries change the interpretation, and what must be stated so that an FDR target is meaningful.

The maximum in the FDP denominator defines the FDP as zero when there are no rejections. The quantity \(V\) counts false rejections, not all true null hypotheses in the family. Thus, the family contains both null hypotheses that are true and null hypotheses that are false; the analyst generally does not know which are which.

An FDR target, often written \(q\), concerns the expected FDP for repeated applications of the specified testing procedure to the specified family, under the procedure’s assumptions. It does not promise that the realized FDP in one data set will be at most \(q\). Nor is it the probability that any particular rejected hypothesis is false. The family and procedure together give the target its interpretation.

Make the Family Match the Claims

In a gene-expression analysis, “test each gene” is not yet a complete family description. A null hypothesis is attached to a particular scientific comparison. For example, a gene can have one null hypothesis for a treatment-versus-control contrast in one tissue and a different null hypothesis for a dose trend in another tissue. The family might therefore consist of all gene-by-contrast hypotheses for one primary experiment, or it might consist only of hypotheses for a particular contrast.

A useful specification names the tested feature, the comparisons, the population or samples, and any rules that determine which hypotheses are included. A family might be described as “the null hypotheses of no treatment effect for each of the 12,000 genes measured in the primary tumor samples for the prespecified treatment-versus-control contrast.” This description is more informative than “the genes tested”: it identifies both the claims and the boundary of the error-rate statement.

• One contrast across genes: the family can contain one null hypothesis per gene for a prespecified contrast.
• Several contrasts: the family can contain all gene-by-contrast hypotheses, or separate families can be defined for distinct scientific questions.
• Several tissues or outcomes: each tissue or outcome may define a separate family, or the combined set may be treated as one family.
• Selected genes: a family may be limited by a pre-established eligibility rule, provided the testing and error-control method remain valid for that rule.

There is no universal family boundary dictated by the fact that the data are genomic. The right boundary depends on which discoveries will be reported and interpreted together. If the scientific conclusion is “these genes are associated with treatment across all measured contrasts,” defining a family for only one contrast does not control the error rate for that broader conclusion.

Combined and Separate Families

When there are several scientific questions, analysts must decide whether their hypotheses form one combined family or multiple families. A combined family makes the FDR target refer to discoveries across the entire collection. Separate families make each target refer only to its own collection. Neither choice is merely a formatting convention: the reported scope of the error-rate claim changes.

For example, suppose a study has treatment-versus-control and dose-trend hypotheses for each gene. A combined family contains both types of hypotheses. Alternatively, each contrast can have its own family if the scientific questions and reporting are genuinely separate. If each family is analyzed at target \(q\), this gives a target for each family; it does not automatically establish that the pooled discoveries across all families have FDR at most \(q\). The pooled ratio and its expectation concern a different collection of discoveries.

Filtering, Selection, and Family Membership

Many genomic workflows remove genes that are unmeasurable or have little information before testing. The family should reflect the hypotheses that are actually eligible for the declared analysis, and the eligibility rule should be reported. Filtering based on information that does not undermine the null distribution can be compatible with valid testing, but that is a condition to justify—not an automatic consequence of calling a filter “preprocessing.”

Selection based on the same evidence later used to test hypotheses is more delicate. For example, selecting genes because they have small p-values and then treating those selected tests as if they were an independently specified family can distort the error rate. The selection changes which hypotheses are examined and can make their p-values no longer behave as required by the testing procedure. Such a workflow needs a method that accounts for the selection, or a selection rule with appropriate validity guarantees.

There is also an important distinction between changing a family and changing only its description. If a fixed set of rejections and all its false rejections remain in the family, adding unrejected hypotheses does not change the realized FDP: neither \(R\) nor \(V\) changes. It can nevertheless change the procedure’s behavior and the expected FDP across repeated data sets, because the testing procedure is being applied to a different set of hypotheses. Conversely, if a family boundary excludes some rejected hypotheses, the reported FDP is calculated using only rejections and false rejections inside that boundary.

A Short Justification: Why Membership Matters

Write \(I_i=1\) when hypothesis \(i\) is a true null and \(I_i=0\) otherwise, and let \(D_i=1\) when it is rejected. For a fixed family, \(V=\sum_{i\in\mathcal{I}} I_iD_i\) and \(R=\sum_{i\in\mathcal{I}}D_i\). Substituting these counts into the definition gives

Every term in this expression refers to a hypothesis in \(\mathcal{I}\). Changing \(\mathcal{I}\) changes the set of possible rejections and false rejections entering the ratio, and can also change the operation of the testing procedure. This is why an FDR target cannot be interpreted independently of a family definition. The equation does not imply that increasing the number of hypotheses always increases FDR; the result depends on the procedure, the null configuration, and the dependence among tests.

Diagnostics, Comparisons, and Limitations

Before reporting an FDR-controlled result, check whether the family description matches the analysis and the scientific claims. Useful questions include:

• Can every hypothesis be identified by its gene, contrast, tissue, and outcome, as applicable?
• Does the stated family include all discoveries that are being presented as one set?
• Were inclusion and filtering rules specified, and are they compatible with the p-values and procedure used?
• Were separate FDR targets reported as separate claims, rather than silently treated as a single pooled guarantee?
• Does the stated target refer to expected FDP under the method’s conditions, rather than a guaranteed realized fraction in this data set?

A family-specific FDR analysis differs from controlling the familywise error rate (FWER), which concerns the probability of at least one false rejection in the family. These measures answer different questions: FDR concerns the expected proportion of false discoveries among rejections, while FWER concerns whether any false rejection occurs. In either case, defining the family is essential. Choosing a narrower family can make a claim narrower; it does not justify presenting that claim as covering excluded tests.

The FDR target is not a guarantee about the truth of an individual gene result, a confidence level for the number of true discoveries, or a license to redefine the family after seeing results. It also does not resolve biological dependence, model misspecification, or measurement problems. Those issues affect whether a procedure’s assumptions are credible and whether the hypotheses represent the intended scientific questions. A transparent family definition makes the scope clear; it cannot repair an invalid testing analysis.