Specifying Discrete Response Models

What It Means to Specify a Discrete Response Model

A discrete response model is a rule for assigning probabilities to the possible values of a response. To state one completely, we need to say which values are possible, which parameter values are allowed, and what probability each response value receives. These pieces work together: a formula that assigns negative numbers, or whose masses add to something other than one, is not a probability model.

The response variable is denoted by $Y$. Its realized value is a particular $y$. For a discrete response, probabilities are assigned to individual values by a probability mass function. A model makes those masses depend on a parameter $\theta$. The basic task here is to check and interpret that assignment, rather than to make assumptions about how multiple responses are sampled.

Definition: A discrete response model consists of a parameter space $\Theta$ and, for each $\theta\in\Theta$, a probability mass function $f(y;\theta)$ on a discrete set of response values. For every allowed $\theta$, the masses must be nonnegative and sum to one. The support is the set of values that can receive positive probability; a stated carrier set may also include values whose mass is zero for a particular parameter value.

A support can be finite, such as $\{0,1\}$ or $\{\text{A},\text{B},\text{C}\}$, or countably infinite, such as $\{0,1,2,\ldots\}$. The values need not be numbers: categories can be labels. For a countably infinite support, “the masses sum to one” means that the infinite series of nonnegative masses has sum one.

The parameter space is part of the model, not an optional detail. For instance, a formula with a parameter called $\theta$ is incomplete until we state which $\theta$ values make every mass nonnegative and the total equal to one. Different parameter values may describe different distributions, but a model need not be identifiable: distinct parameter values can sometimes produce exactly the same masses.

Conditions for a Valid Mass Assignment

Suppose $S$ is a finite or countable carrier set, and a proposed formula $f(y;\theta)$ assigns a number to every $y\in S$. The two checks below characterize whether it defines a probability model on $S$.

Theorem: For a fixed $\theta$, the assignment $f(y;\theta)$, $y\in S$, defines a probability mass function on $S$ if and only if $$ f(y;\theta)\geq 0\quad\text{for every }y\in S, \qquad \sum_{y\in S}f(y;\theta)=1. $$ A parameterized assignment is a model on $\Theta$ only if these conditions hold for every $\theta\in\Theta$.

Proof. If $f$ is a probability mass function, then $f(y;\theta)=P(Y=y)$, so each mass is nonnegative. The events $\{Y=y\}$, as $y$ ranges over $S$, are disjoint and their union is the event that $Y$ takes a value in its carrier set. That event has probability one, so countable additivity gives $\sum_{y\in S}f(y;\theta)=1$.

Conversely, suppose the masses are nonnegative and sum to one. For any subset $A\subseteq S$, define $$ P(Y\in A)=\sum_{y\in A}f(y;\theta). $$ This is a probability: it is nonnegative, and its value for $A=S$ is one. If $A_1,A_2,\ldots$ are disjoint subsets of $S$, every mass in their union occurs in exactly one of the sums for the $A_i$. Since all terms are nonnegative, regrouping the series is valid, and $$ P\left(Y\in\bigcup_{i=1}^{\infty}A_i\right) =\sum_{y\in\bigcup_i A_i}f(y;\theta) =\sum_{i=1}^{\infty}\sum_{y\in A_i}f(y;\theta). $$ Thus the assignment is countably additive and defines a probability distribution. For a model indexed by $\theta$, the same argument must apply separately at every allowed parameter value. $\square$

This criterion also gives a direct way to find or check a parameter space: impose nonnegativity and normalization, then retain exactly the parameter values that satisfy both. If the model is intended to have exact support $S$, require in addition that $f(y;\theta)>0$ for every $y\in S$. If zero masses are allowed, $S$ is better understood as a carrier set.

Worked Discrete Response Models

Worked Example: A Binary Response

Let $Y$ record whether a patient meets a specified response criterion: $Y=1$ means yes and $Y=0$ means no. Consider the assignment $$ f(y;\theta)= \begin{cases} 1-\theta,&y=0,\\ \theta,&y=1, \end{cases} \qquad \Theta=[0,1]. $$

For every $\theta\in[0,1]$, both masses are nonnegative. Their total is $$ f(0;\theta)+f(1;\theta)=(1-\theta)+\theta=1. $$ Therefore the parameter range gives a valid model. At $\theta=0.3$, the mass at 0 is $0.7$ and the mass at 1 is $0.3$. In particular, $P(Y=1)=0.3$, while the event $Y\in\{0,1\}$ has probability $0.7+0.3=1$.

The endpoints are valid: at $\theta=0$, all mass is at 0; at $\theta=1$, all mass is at 1. If the intended model required both responses to have positive probability, the parameter space would instead be $0<\theta<1$.

Worked Example: An Unordered Categorical Response

Suppose a response records which of three treatment options a participant selected, with values $\text{A}$, $\text{B}$, and $\text{C}$. Let the parameter be a vector $\boldsymbol{\theta}=(\theta_{\text{A}},\theta_{\text{B}},\theta_{\text{C}})$, and assign $$ f(j;\boldsymbol{\theta})=\theta_j, \qquad \Theta=\left\{\boldsymbol{\theta}:\theta_j\geq 0,\ \theta_{\text{A}}+\theta_{\text{B}}+\theta_{\text{C}}=1\right\}. $$

The parameter-space constraints are exactly the nonnegativity and sum-to-one conditions. For $\boldsymbol{\theta}=(0.2,0.5,0.3)$, the three masses are $0.2$, $0.5$, and $0.3$, whose total is $1.0$. If the event of interest is selecting A or C, then $$ P(Y\in\{\text{A},\text{C}\})=f(\text{A};\boldsymbol{\theta})+ f(\text{C};\boldsymbol{\theta})=0.2+0.3=0.5. $$

The labels have no numerical order in this specification. The probabilities are attached to the categories themselves, and the model requires no arithmetic on the labels.

Worked Example: A Count Response on an Infinite Support

Let $Y$ be a count taking values $0,1,2,\ldots$. For $0\leq\theta<1$, propose $$ f(y;\theta)=(1-\theta)\theta^y,\qquad y=0,1,2,\ldots. $$ At $\theta=0$, interpret the assignment directly as $f(0;0)=1$ and $f(y;0)=0$ for $y\geq1$.

For $0\leq\theta<1$, all masses are nonnegative. For $0<\theta<1$, the geometric-series identity gives $$ \sum_{y=0}^{\infty}f(y;\theta) =(1-\theta)\sum_{y=0}^{\infty}\theta^y =(1-\theta)\frac{1}{1-\theta}=1. $$ At $\theta=0$, the masses sum to one by the stated boundary definition. Thus the entire parameter range is valid. The value $\theta=1$ must be excluded: the proposed expression would assign zero to every $y$, so the total would be zero, not one.

For $\theta=0.4$, the first three masses are $$ f(0;0.4)=0.6,\qquad f(1;0.4)=0.6(0.4)=0.24,\qquad f(2;0.4)=0.6(0.4)^2=0.096. $$ Their sum is $0.936$. The remaining event $Y\geq3$ has probability $$ \sum_{y=3}^{\infty}0.6(0.4)^y =0.6\frac{(0.4)^3}{1-0.4} =0.064. $$ The two parts total $0.936+0.064=1$, as required.

From Point Masses to Events

Once the point masses are specified, probabilities of events follow by addition. For a finite event, this is an ordinary finite sum. For a countably infinite event, it is a sum of nonnegative terms. This is the discrete version of the rule that probabilities of disjoint outcomes add.

Result: If $Y$ has mass function $f(y;\theta)$ on a countable carrier set $S$, then for every event $A\subseteq S$, $$ P(Y\in A)=\sum_{y\in A}f(y;\theta). $$

Proof. For each $y\in A$, the event $\{Y=y\}$ is disjoint from $\{Y=z\}$ whenever $z\in A$ and $z\ne y$. Their union is exactly $\{Y\in A\}$. By countable additivity, $$ P(Y\in A)=\sum_{y\in A}P(Y=y)=\sum_{y\in A}f(y;\theta). $$ For finite $A$, the sum is finite; for countably infinite $A$, it is the corresponding nonnegative series. $\square$

A model can also be valid without being uniquely parameterized. For example, on $\{0,1\}$, set $f(1;\theta)=\theta^2$ and $f(0;\theta)=1-\theta^2$, with $-1\leq\theta\leq1$. These masses are nonnegative and sum to one, but $\theta=0.5$ and $\theta=-0.5$ give the same distribution. This does not invalidate the model; it means the parameter cannot be recovered uniquely from the distribution.

Common Mistakes and Why Specification Matters

Checking only one parameter value. A parameterized model must be valid for every parameter value in its stated space. Check the constraints across the whole range.
Forgetting normalization on an infinite support. A formula can give nonnegative values at every count and still have a total other than one. Evaluate the infinite series, not just its first few terms.
Confusing a carrier set with exact support. A value can belong to the carrier set and have zero probability at a boundary parameter value. Exact support contains only values with positive mass.
Treating a formula as self-validating. Naming a parameter or writing a plausible-looking expression does not establish that it defines a probability distribution. State the parameter space and verify both required conditions.

Clear specification matters because later calculations depend on the response distribution being well-defined. The support identifies which outcomes are possible, and the parameter space states which probability assignments the model permits. These are the basic checks to make before interpreting probabilities or using a model for inference.

Check Your Understanding

Use the support, parameter range, and mass formula together when answering.

A response has support $\{0,1\}$ and masses $f(1;\theta)=\theta$, $f(0;\theta)=1-\theta$. What is the largest parameter space that makes the assignment valid?
For a three-category response, explain why nonnegative category probabilities that sum to $0.9$ do not define a probability mass function.
For the count model $f(y;\theta)=(1-\theta)\theta^y$, why is $\theta=1$ excluded?
If a carrier set includes a value with probability zero at one parameter value, is that value in the exact support at that parameter value? Explain.
Can two distinct parameter values define the same response distribution? Give the implication for identifying the parameter from the distribution.