Introduction
A study’s title or abstract may call it a “trial,” “program evaluation,” or “comparison,” but those labels do not tell you exactly how participants ended up in the groups. The methods section is where that information belongs. Reading it carefully means identifying who or what was studied, what condition or exposure was compared, and how the groups were formed.
The key question about randomization is not whether the researchers used a computer, drew up a list, or made the groups look similar. It is whether a chance mechanism determined which units received which treatment. If the methods do not say how assignment occurred, the careful conclusion is that randomization cannot be determined from the description—not that it happened, and not necessarily that it did not.
A Reliable Reading Framework
Methods descriptions often mix several ideas together. Pull them apart before deciding what kind of study was conducted. For each description, look for the units, the treatment or exposure, the comparison group, and the assignment process. Then classify the study and say whether randomization is established.
Name the people, animals, schools, clinics, or other entities the study concerns. Check whether the unit receiving an assignment differs from the unit on which outcomes are measured.
Ask what condition is being compared. If investigators impose or allocate it, it is a treatment. If units already have different statuses or exposures and investigators record them, it is an observed exposure.
State which group is being compared with which other group, and describe its condition. A comparison group may receive usual care, a placebo, another treatment, or no assigned intervention.
Look for a stated allocation rule. Determine whether investigators assigned the conditions, whether assignment used chance, or whether groups arose from existing behavior, status, or circumstances.
Classify the study from the described methods. State that assignment was randomized only if the methods establish a chance-based allocation; otherwise identify precisely what remains unspecified.
The experimental unit is the smallest unit assigned to a treatment independently. The observational unit is the unit on which a measurement or outcome is recorded. These may be the same, but they need not be. For example, a clinic might be assigned one program while outcomes are measured for individual patients attending that clinic. In that case, the clinics are the experimental units and the patients are observational units.
The words “treatment” and “exposure” help describe what the researchers did, but they do not by themselves determine study type. A study of an exposure can be an experiment if investigators assign it; a study of a treatment can be observational if investigators only record who received it in ordinary practice. The decisive issue is whether investigators assigned the condition under study.
Distinguishing Assignment from Observation
In an experiment, investigators decide which units receive which conditions. In a randomized experiment, a chance process makes that allocation. Examples of chance processes include drawing assignments from a properly conducted lottery or using a random-number generator to allocate units. Merely using a computer is not enough: software can implement either a random process or a fixed, predictable rule.
In an observational study, the groups arise without the investigators assigning the exposure being studied. A methods section might compare people who already use a particular service with people who do not, or compare outcomes across existing exposure categories. Researchers may collect data prospectively or retrospectively, but the direction of data collection does not by itself change an observational study into an experiment.
Some studies involve an intervention but do not use random assignment. They are often described as nonrandomized experiments or quasi-experiments, depending on the design. For reading a methods description, report the concrete facts first: investigators assigned a program, but the methods describe a fixed allocation rule rather than a chance mechanism. This is more informative than relying on a broad label alone.
Worked Example: A Clearly Randomized Individual Trial
Suppose a hypothetical study enrolls 120 adults with recurring headaches. The methods state that each adult was assigned individually, using a random-number generator, to receive either a new treatment or a matching placebo. Researchers record each adult’s number of headache days over the next eight weeks. Identify the design features.
Units. The 120 adults are the units assigned to treatment. Since headache days are recorded for each adult, the adults are also the observational units.
Treatment and comparison group. The new treatment is the treatment condition. The placebo group is the comparison group; its members receive a matching placebo rather than the active treatment.
Study type and assignment. Investigators assigned the treatments, so this is an experiment. The methods explicitly say a random-number generator determined each individual’s assignment, so treatment assignment was randomized. The fact that outcomes are collected over the following eight weeks makes this a prospective study, but prospective data collection is not what establishes randomization.
A concise methods summary would be: “This is an individually randomized experiment in 120 adults, comparing the new treatment with placebo, with headache days measured for each adult.” Each part follows from a specific statement in the methods.
Worked Example: An Existing Exposure in a Records Study
Suppose investigators use existing clinic records to compare kidney test results for 84 patients who had already used a certain pain medicine regularly and 92 patients who had not. The clinicians—not the study investigators—made decisions about medicine use as part of ordinary care. The records include each patient’s test result and past medicine use. Identify the study design.
Units. The patients are the units, and the kidney test result is measured for each patient.
Exposure and comparison group. Regular prior use of the medicine is the exposure being studied. Patients with no regular use form the comparison group. The investigators are comparing existing exposure groups; they did not assign the medicine for this study.
Study type and assignment. This is an observational study because investigators recorded existing medicine use and outcomes rather than assigning the medicine. There was no treatment-assignment process in the study as described, so treatment was not randomized by these investigators. This does not mean that all medicine use everywhere is nonrandom; it describes how groups arose in this particular study.
The records already existed before the comparison was planned, so the study is retrospective in its data use. That feature does not alter the basic classification: the investigators observed an existing exposure rather than assigning it.
Worked Example: A Computer-Generated List with Unclear Randomization
Suppose a hypothetical methods description says that 60 adults entering a wellness program were split between a coaching group and a usual-services group “according to a computer-generated allocation list.” The description does not explain how the list was produced. Outcomes were recorded for each adult after three months. What can be concluded?
Units. The adults are assigned to groups, and outcomes are measured for each adult. Thus, the adults are both the units receiving assignment and the observational units.
Treatment and comparison group. Coaching is the assigned program, and usual services are the comparison condition.
Study type. The description says the adults were split between conditions by an allocation list. It therefore describes an intervention assigned to units, rather than merely comparing pre-existing coaching users with nonusers. On the information given, it is an experiment with an allocation process that is not fully specified.
Was assignment randomized? That cannot be determined from the statement “computer-generated allocation list.” A computer can generate a list using chance, but it can also produce a fixed allocation, such as alternating group labels or sorting by an ID number. The methods must establish that a chance mechanism generated the assignments before we can call the treatment assignment randomized. A careful report is: “The methods describe investigator allocation to coaching or usual services, but do not provide enough detail to determine whether allocation was random.”
Why the Assignment Detail Matters
The method used to form groups changes what the design description means. If investigators assign a treatment by chance, group membership is not determined by participants’ choices or by a fixed characteristic used in the allocation rule. If investigators assign treatments without chance, the methods still describe an experiment, but the groups were formed differently. If investigators do not assign the exposure at all, the comparison is observational.
This distinction is why labels should be supported with method details. “Randomized” is a claim about the allocation mechanism, not a synonym for “assigned,” “computer-generated,” “balanced,” or “chosen by the researchers.” Likewise, “control group” does not tell you whether the group was randomly formed. The description of the group and the description of its assignment are separate pieces of information.
When the methods are incomplete, preserve the uncertainty rather than filling the gap with a guess. You can still report what is known: the units, the intervention, the comparison condition, and the fact that an allocation list was used. Then specify the missing detail: whether the list was generated by chance. Good design reading separates what the text establishes from what it leaves open.
Common Mistakes
- Calling any assigned treatment randomized. An investigator can assign a treatment using a fixed rule. Assignment makes the study experimental; only a chance-based assignment makes it randomized.
- Treating “computer-generated” as proof of chance. A computer may follow a deterministic rule. Look for an explicit chance mechanism, such as random-number allocation.
- Calling an observed exposure a treatment. If participants already had different exposure histories and investigators only recorded them, describe the exposure as observed, not assigned by the study.
- Confusing the units assigned with the units measured. If whole clinics are assigned but individual patient outcomes are recorded, name the clinics as experimental units and the patients as observational units.
- Inferring design from timing alone. Prospective or retrospective data collection describes when data are gathered relative to the study question; it does not establish whether investigators assigned a treatment.
- Reporting more certainty than the methods allow. If the allocation procedure is not described well enough to identify a chance mechanism, say randomization cannot be determined from the stated methods.
Check Your Understanding
For each scenario, identify the units, the treatment or exposure, the comparison group, and what can be said about assignment.
- A researcher assigns 40 classrooms to one of two lesson plans by drawing classroom labels from a well-mixed bag, then records quiz scores for students. What are the experimental units and observational units? Is treatment assignment randomized?
- A clinic compares recovery times among patients who independently chose to attend a new exercise class and patients who did not attend. Did the study investigators assign the exposure?
- A methods section says a computer-generated allocation list placed participants into two treatment groups but does not explain how the list was made. What is known, and what is not established?
- Researchers assign a training program to each school using a fixed rule based on school size, then measure each student’s attendance. Is this observational or experimental? Was assignment randomized?
- A study uses records collected before its investigators began the analysis to compare two existing exposure groups. Does the retrospective timing by itself make the study an experiment?