Why Specify a Model for Two Groups?
When comparing two groups on a yes-or-no outcome, the observed data are counts of successes and group sizes. But those counts alone do not explain how the data arose. A model makes the assumptions explicit: what counts as a success, how likely it is in each group, and whether the observations can be treated as independent. Those assumptions matter because conclusions about population proportions depend on the way the groups were formed and the outcomes collected.
For example, a study might compare whether patients recover within a specified period under two treatments. “Recovered” is a binary outcome, and each treatment group has its own population probability of recovery. The independent Bernoulli model describes such data when each person’s outcome is a separate, binary observation and outcomes in the same group share a common probability.
The Independent Bernoulli Model
The definition of success depends on the question. It could mean “recovered,” “experienced an adverse event,” or “responded yes.” It must be chosen consistently so that \(p_1\) and \(p_2\) describe the same outcome in the two groups. Reversing the coding of success would replace each proportion \(p_j\) by \(1-p_j\).
In a sample, let \(X_j\) be the number of successes among the \(n_j\) individuals in group \(j\). The sample proportion is \(\hat{p}_j=X_j/n_j\), and the model’s group-specific population proportion is \(p_j\). A group’s outcomes all have the same success probability, but their realized values can differ: equal probabilities do not mean that every person has the same outcome.
What the Assumptions Mean
The model’s assumptions concern the data-generating process, not just the appearance of a table of counts. They can be considered one at a time.
- Binary outcome. Each individual contributes one observation coded as success or failure. An outcome with several categories, or a numerical measurement such as blood pressure, is not directly a Bernoulli outcome unless a meaningful binary event is defined.
- Common probability within each group. For a given group, each individual has the same probability \(p_j\) of success under the model. If probabilities vary substantially across individuals—for example, because the group contains distinct risk subgroups—the single-\(p_j\) model may be an oversimplification.
- Independence within a group. Knowing one individual’s outcome does not change the modelled probability distribution of another individual’s outcome. Shared exposure, household membership, or transmission between individuals can undermine this assumption.
- Independence between groups. The observations in one group do not determine or constrain the observations in the other. Two groups made from separate samples of people are a natural setting; the same people measured under both conditions generally are not independent groups.
- Fixed group sizes. The model describes \(n_1\) and \(n_2\) observations in the respective groups. The group sizes are set by the study design or sampling plan, while the outcomes are observed.
These assumptions are not guaranteed simply because a data set contains two columns of counts. A paired study, a cluster-based study, or a study with substantial dependence between participants may need a different model. Likewise, if the outcome is recorded inconsistently across groups, a common definition of success is absent.
Worked Example: A Randomized Treatment Study
In a hypothetical trial, 40 eligible patients are randomly assigned to one of two treatments, with 20 patients assigned to each. The outcome is whether each patient has recovered by day 14. Suppose 13 patients in Treatment A and 9 in Treatment B recover. State the model and assess what the design supports.
Code recovery by day 14 as \(1\), and no recovery by day 14 as \(0\). Let \(Y_{i1}\) be patient \(i\)’s outcome in Treatment A and \(Y_{i2}\) the corresponding outcome in Treatment B. Under the model, the outcomes in group 1 are independent Bernoulli observations with probability \(p_1\); those in group 2 are independent Bernoulli observations with probability \(p_2\); and observations across the groups are independent.
The observed proportions are \(\hat{p}_1=13/20=0.6500\) and \(\hat{p}_2=9/20=0.4500\). These describe the trial participants’ observed recovery rates; they are not, by themselves, proof that the model’s assumptions hold.
Random assignment supports a causal comparison of the treatments for the patients enrolled, because the assignment procedure makes pre-treatment characteristics comparable between treatment groups on average across repeated assignments. It does not make the enrolled patients a random sample of all patients. Therefore, extending the causal conclusion to a wider patient population requires a separate argument about how representative the participants are. Also, random assignment does not automatically guarantee independent outcomes: shared wards, treatment spillovers, or other interference could create dependence.
How Study Design Supports Inference
Two design features are especially important, but they answer different questions. Random sampling concerns which individuals enter the study. Random assignment concerns which condition or group the selected individuals receive. One does not substitute for the other.
| Design feature | What is randomized? | What it supports | What it does not establish by itself |
|---|---|---|---|
| Random sampling | Selection of individuals from a population | Using sample outcomes to learn about the population’s proportions, subject to the model and sampling design | A causal effect of a group or treatment |
| Random assignment | Allocation of study participants to groups | A causal comparison of assigned conditions for the study participants, subject to the design and outcome assumptions | That participants represent a broader population |
If two independent random samples are drawn from two target populations, random sampling gives a design-based reason to use the observed proportions to learn about those populations. The groups need to be sampled independently for the between-group independence assumption to be plausible. If people are sampled without replacement from a finite population, the draws are not exactly independent; treating them as independent is an approximation that is usually more reasonable when the sample is a small fraction of the population.
In a randomized experiment, random assignment helps make the groups comparable with respect to pre-existing characteristics, in expectation over the assignment process. This supports a causal interpretation of differences between assigned conditions, but assignment does not make the participants representative of everyone to whom the result might be applied. Conversely, a well-drawn random sample can support generalization to its source population while still being observational: if individuals chose their own groups, other differences between groups may explain an observed association.
Worked Example: Independent Random Samples From Two Populations
Suppose a hypothetical poll draws a simple random sample of 80 adults from Region A and a separate simple random sample of 100 adults from Region B. The question is whether each adult supports a proposed policy. In the samples, 31 adults in Region A and 46 in Region B support it. Describe the model assumptions and the role of sampling.
Code support as \(1\) and opposition as \(0\). The observed proportions are \(\hat{p}_1=31/80=0.3875\) and \(\hat{p}_2=46/100=0.4600\). The model uses population support probabilities \(p_1\) for Region A and \(p_2\) for Region B.
The outcome is binary and defined identically in both regions. The sampling plan gives a basis for treating each sample as representative of its own region, provided selection and response follow the stated plan. Separate, independent sampling supports independence between groups. Within each region, the common-probability assumption treats adults’ support outcomes as having the same population probability. If people’s responses are linked—for instance, if a selected household member influences another sampled household member—the within-group independence assumption may be questionable.
Because the adults were sampled, rather than assigned to live in one region or the other, the design supports inference about support in the two regions, not a claim that living in a particular region causes a person to support the policy. Sampling supports generalization; it does not turn an observational group comparison into a randomized experiment.
A Short Justification: What One Bernoulli Observation Implies
The model also tells us the mean and variance of an individual outcome. This follows directly from the two possible values, rather than from a special rule to memorize. Let \(Y\) be one Bernoulli outcome with success probability \(p\). Since \(Y=1\) with probability \(p\) and \(Y=0\) with probability \(1-p\), its expected value is
Because \(Y^2=Y\) for both possible values, \(E(Y^2)=p\). Using \(\operatorname{Var}(Y)=E(Y^2)-[E(Y)]^2\) gives
Thus \(p_j\) is the mean of a single binary outcome in group \(j\), and \(p_j(1-p_j)\) is its variance. This calculation concerns one observation. It does not yet give the sampling distribution of a difference between two sample proportions; that requires combining information across observations and groups.
Worked Example: Diagnosing Dependence in a Clustered Study
A hypothetical study randomly assigns 12 clinics to a new reminder system and 12 clinics to usual care. It then records, for every patient at each clinic, whether the patient attends a scheduled appointment. Can all patient outcomes automatically be treated as independent Bernoulli observations?
No. Each outcome is binary, but patients at the same clinic share staff, procedures, and local conditions. In addition, the reminder system is assigned to clinics, not separately to individual patients. Outcomes within a clinic may therefore be related, and patients in the same group need not share one common probability in the simple individual-level model.
The number of randomized units is 24 clinics, not the total number of patients. Random assignment of clinics can support a causal comparison of the reminder systems at the clinic level, but it does not make every patient outcome independent. An analysis would need to account for the clustered design rather than assume that the patient records are independent Bernoulli observations. Simply counting all patients as independent observations would overstate how much independent information the study contains.
Common Mistake: Treating Randomization as a Universal Guarantee
A frequent error is to reason, “The study was randomized, so the groups are independent, representative, and any difference is causal.” Each part of that statement needs its own justification. Random assignment supports causal comparison of the assigned conditions, but does not ensure population representativeness. It also does not prevent interference or clustering from making outcomes dependent. Random sampling supports generalization to a source population, but does not remove confounding from a nonrandom group comparison.
A more careful description names the actual design: who was sampled, whether the groups were sampled independently, how participants were assigned, and what features might link outcomes. The independent Bernoulli model is useful when its assumptions reasonably represent the study, not merely because it is convenient for analyzing two proportions.
Check Your Understanding
Use the model and study-design distinctions to answer these questions.
- A study records whether each participant experienced a side effect. State what must be defined before the outcome can be coded as Bernoulli.
- In the two-group model, must \(p_1\) and \(p_2\) be equal? What does the model require to be common within each group?
- Explain what random sampling supports that random assignment does not, and what random assignment supports that random sampling does not.
- A random sample is taken without replacement from a small finite population. Why might the independence assumption not hold exactly?
- Why might treating all patient records as independent be inappropriate when a treatment was assigned by clinic?