Why the Randomization Unit Matters
A price test needs a clear rule for who can receive the new price and who remains under the comparison plan. The randomization unit is the smallest entity independently assigned to a test group. It may be a policy, household, agency, or geographic area. The unit receiving a price quote is not automatically the right randomization unit: if one assignment can affect another quote, assigning those quotes separately risks contamination.
For example, two policies in one household may be discussed together, or an agent may apply a test procedure to every eligible customer they handle. If related observations receive different assignments, the control experience may be influenced by the treatment. Assignment at a larger unit can limit that risk, but it reduces the number of independently assigned units and may leave groups less balanced. Choosing a unit is therefore a design decision, not merely a data-processing choice.
This tutorial is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus. It focuses on selecting assignment units and blocking variables; it does not address how to calculate the power of the resulting test or estimate its treatment effect.
Selecting an Assignment Unit
Begin by asking where the treatment can spill over. Consider whether a shared decision-maker, communication channel, or operational process could expose a nominal control unit to the test. Then choose the smallest unit that keeps such interference acceptably limited. Larger units can reduce cross-group contamination but can also create unequal group sizes or leave only a few independent assignments.
| Possible unit | When it may fit | Design issue to check |
|---|---|---|
| Policy | Each policy receives a distinct quote and decisions are substantially independent. | Related policies in one household might receive conflicting offers. |
| Household | Household members discuss prices or share a purchasing decision. | Households can have different numbers of policies, so equal household counts need not mean equal policy counts. |
| Agency | Agents or agency processes could carry the test approach from one customer to another. | Agencies can differ in size, location, or customer mix; a small number of agencies limits independent assignments. |
| Geographic area | Local implementation, communication, or market behavior may cause spillover across nearby policies. | Areas may differ substantially in size and composition, and broad assignment can sacrifice balance. |
A larger unit is not automatically safer or better. If there is no plausible route for a treatment to affect other policies in the household, assigning at household level may unnecessarily make all policies move together. Conversely, a policy-level assignment is a poor choice when an agent can readily apply the treatment process to both treatment and control policies.
Count the independent randomization units, not just the policies represented in the data. Assigning 500 policies in four agencies still means only four independently assigned agencies if agency is the assignment unit. A design with few large units is especially vulnerable to imbalance in agency-level characteristics. Where feasible, pairing similar agencies and randomizing one agency in each pair to treatment can improve balance; it does not make the paired agencies identical.
Worked Example: Keeping Household Quotes Together
A hypothetical insurer has 120 households eligible for a renewal-price test. Each household has exactly two policies, for 240 policies in total. The test changes the quote presentation on renewal. Assume household members are likely to compare their two quotes and the shared discussion could affect renewal decisions. Compare policy-level assignment with household-level assignment.
Method. Choose the assignment unit by tracing the possible contamination pathway. Under policy-level assignment, each of the 240 policies is independently assigned with probability \(1/2\) to treatment. For any one two-policy household, there are four equally likely assignment patterns: both treatment, first treatment only, second treatment only, and both control. Two of the four patterns are mixed.
Across 120 households, the expected number of mixed households is \(120(1/2)=60\). Thus policy-level assignment is likely to produce many households exposed to both conditions, which is undesirable under the stated contamination concern.
Under household-level assignment, there are 120 randomization units. Assigning 60 households to each arm gives 60 treatment households and 60 control households. Because each has two policies, each arm contains \(60(2)=120\) policies. No household has policies assigned to different arms.
Check. The total is \(120+120=240\) policies, matching the portfolio. Household assignment avoids the identified within-household mixing. The common mistake is to count 120 treatment and 120 control policies as 240 independent assignments; there are only 120 independent household assignments.
Blocking to Balance Important Characteristics
After choosing the unit, identify pre-treatment characteristics that are both important and available for every unit. Examples include region, renewal month, prior premium band, risk tier, distribution channel, or an agency-size category. A useful blocking variable is related to expected test outcomes, treatment response, or a material operational difference. Block on a small set of characteristics that matter; using many cross-classified variables can create numerous tiny blocks and make assignment awkward.
Within each block, assign units to treatment and control. With an even number of units, equal allocation within that block gives exact balance on the block membership. With an odd number, one arm must receive one more unit unless allocation is otherwise deliberately unequal. Across several odd-sized blocks, the extra units can be allocated to opposite arms to keep overall counts balanced.
Here \(n_b\) is the number of randomization units in block \(b\), and \(n_{Tb}\) and \(n_{Cb}\) are its treatment and control counts. For an odd block, \(n_{Tb}\) and \(n_{Cb}\) differ by one under the closest possible equal allocation.
Blocking balances the variables used to form the blocks; it does not ensure balance on every other characteristic. It also does not justify using post-treatment information to form blocks. A characteristic measured after the price assignment may itself be affected by treatment, so conditioning assignment on it can distort the comparison.
Worked Example: Blocking on Risk Tier and Region
A hypothetical renewal test assigns 84 policies to either a revised price presentation or the existing presentation. The assignment unit is the policy because quotes are handled independently. Before assignment, policies fall into four blocks defined by risk tier and region. The block counts are: low-risk rural 31, low-risk urban 21, high-risk rural 19, and high-risk urban 13. Allocate as evenly as possible, while balancing the extra assignments from odd-sized blocks.
Method. Split each block as evenly as possible. The four block sizes are odd, so each contributes one extra unit to one arm. Assign the extra units to treatment in the first and third blocks and to control in the second and fourth blocks.
Low-risk rural: \(31\) units gives \(16\) treatment and \(15\) control. Low-risk urban: \(21\) gives \(10\) treatment and \(11\) control. High-risk rural: \(19\) gives \(10\) treatment and \(9\) control. High-risk urban: \(13\) gives \(6\) treatment and \(7\) control.
Treatment total: \(16+10+10+6=42\). Control total: \(15+11+9+7=42\). The overall allocation is therefore balanced. By risk tier, low-risk counts are \(16+10=26\) treatment and \(15+11=26\) control; high-risk counts are \(10+6=16\) treatment and \(9+7=16\) control. Each arm also has \(42\) policies in total.
Check. The block totals are \(31+21+19+13=84\), and treatment plus control totals are \(42+42=84\). The risk-tier totals balance exactly, while the rural/urban totals need not: rural is \(26\) treatment versus \(24\) control, and urban is \(16\) versus \(18\). Blocking on the four joint categories does not guarantee equal margins for every factor when odd blocks are split. The common mistake is to say both region and risk tier are exactly balanced without checking their marginal counts.
Why Blocking Works, and Where It Stops
The basic justification is that assignment within a block makes treatment and control draw from the same block composition. For a block with \(n_b\) units and an equal-probability assignment to treatment, each unit has treatment probability \(1/2\). Let \(I_j\) equal 1 if unit \(j\) in that block is assigned to treatment and 0 otherwise. Then the expected treatment count is the sum of the assignment probabilities:
This establishes expected balance within a block. If the design fixes equal counts when \(n_b\) is even, it establishes exact balance in block membership instead. Either way, blocking controls the treatment and control allocation across the chosen categories; it does not ensure equality in realized outcomes or every unblocked characteristic.
Blocking is most useful when the selected variable captures a meaningful source of heterogeneity or portfolio mix. But fine-grained blocking can create many blocks containing one or two units. Such blocks cannot support a balanced split, and may force awkward assignment or make implementation fragile. Combine categories when necessary, and prefer variables with reliable pre-assignment values.
Worked Example: Pairing Agencies Before Assignment
A hypothetical insurer wants to test a new renewal workflow, but agents within an agency may share instructions. Agency is therefore the randomization unit. The eight eligible agencies are paired in advance by territory and approximate size. Their policy counts are: Pair A, 20 and 22; Pair B, 35 and 33; Pair C, 48 and 50; Pair D, 27 and 25. Randomly assign one agency in each pair to treatment and the other to control. To verify the allocation arithmetic, suppose the first-listed agency is selected in Pairs A, B, and C, and the second-listed agency is selected in Pair D.
Method. Treat each agency pair as a block, then assign one of its two agencies to each arm. The treatment policy count is \(20+35+48+25=128\). The control count is \(22+33+50+27=132\). The total is \(128+132=260\), which also equals \(20+22+35+33+48+50+27+25=260\).
There are four agencies in each arm, and every pair contributes one agency to each arm. Policy counts differ by \(132-128=4\), despite exact balance in the number of agencies. This illustrates why agency-count balance and policy-count balance are distinct checks. The pairing controls the matched territory-and-size groups, but it does not guarantee equal exposure counts or balance on every agency characteristic.
Check. The allocation uses all eight agencies and all 260 policies. If the goal also requires close balance in policy exposure, the analyst should consider the counts when constructing pairs or use a prespecified constrained assignment procedure. The common mistake is to conclude that equal numbers of agencies guarantee equal numbers of policies.
Common Design Traps
- Randomizing below the contamination level. Assigning policies separately is not appropriate merely because policy records are the rows in the data. Consider household, agent, agency, or geographic spillover first.
- Counting records instead of independent units. A cluster assignment does not turn every policy in that cluster into an independent randomization.
- Choosing blocks after seeing outcomes. Define blocks using pre-treatment information and a documented rule, not observed responses to the test.
- Blocking on too many variables. A large number of small cells can make balanced assignment impossible or difficult to operate. Prioritize a few important characteristics.
- Assuming blocking solves all imbalance. It controls the variables used in blocking, subject to the actual allocation. It does not guarantee balance in unblocked variables, realized outcomes, or every marginal of a cross-classification.
A practical design sequence is to identify likely interference, select the smallest safe assignment unit, choose a limited set of useful pre-treatment blocking variables, define the blocks before assignment, and verify the resulting counts at both the randomization-unit and policy levels. Record the assignment rule so that the operational team can apply the intended design consistently.
Check Your Understanding
For each problem, identify the design choice and check the relevant counts.
- A hypothetical test includes 72 households with two policies each. If policies are independently assigned to treatment or control with probability \(1/2\), what is the expected number of households with policies in both arms? (Answer: 36 households.)
- A test assigns agencies in three matched pairs. The agency policy counts are \(18,20\), \(25,27\), and \(31,29\). If the first agency in each pair is assigned to treatment, how many policies are in treatment and control? (Answer: 74 treatment policies and 76 control policies.)
- A policy-level test has two even-sized blocks: 24 urban and 16 rural policies. Within each block, assign half to treatment. How many policies are in each arm, and how many urban policies are in each arm? (Answer: 20 policies per arm; 12 urban policies per arm.)
- A hypothetical carrier expects agents in each branch to share test instructions, but branch-level assignment would leave only six assignment units. What is the main reason not to use policy-level assignment, and what is one design concern with branch-level assignment? (Answer: Policy assignment risks contamination through shared instructions; branch assignment has few independent units and may produce imbalance across branches.)
- A portfolio has 40 policies divided into four blocks of 11, 9, 12, and 8. Split each block as evenly as possible, assigning the extra unit in each odd block to treatment for the first block and control for the second. What are the total treatment and control counts? (Answer: 20 treatment and 20 control.)