Why Deidentification Is a Design Choice
Ratemaking analysis may need policy-level exposures, claim histories, coverage details, and geographic or demographic variables. Those fields can be useful for estimating rates, but combinations of them may also make a person or business recognizable. Removing names is not necessarily enough: a rare location, unusual risk characteristic, and distinctive loss history may identify a record when combined with information available elsewhere.
Deidentification is therefore not a single operation that makes data safe for every use. It is a choice among controls that change different things: whether identifiers remain visible, whether records can be linked, how much detail is retained, and who can access the data. A suitable design starts with the analysis requirement and evaluates the residual risk after each control.
Four Controls and Their Trade-Offs
The controls considered here address different parts of the problem. They can be combined, but they are not interchangeable.
| Control | What it does | Utility retained | Important limitation |
|---|---|---|---|
| Masking or generalization | Removes, obscures, or coarsens selected values | Depends on which fields and precision are retained | Remaining quasi-identifiers can still distinguish a record |
| Tokenization | Replaces an identifier with a substitute token, with a separate mechanism for mapping back when needed | Can preserve record linkage if the same token is used consistently | A stable token remains linkable; access to the mapping can expose identity |
| Aggregation | Reports summaries for groups rather than individual records | Group totals, averages, and rates may remain available | Within-group variation and rare subgroups may be lost or exposed |
| Restricted-access environment | Allows approved users to analyze detailed data in a controlled setting | Can retain row-level detail for authorized analysis | Access controls do not remove the risk of misuse, inference, or unsafe outputs |
Masking can mean deleting a field, replacing its values with blanks, or reducing precision. For example, replacing an exact date with a month may lower the chance of matching an external event record, while also making fine-grained timing analysis impossible. Simply covering a value in a display is not effective if the underlying file still contains it.
Tokenization is useful when an analysis needs to connect multiple records for the same policyholder or risk. A token is not automatically anonymous: a stable token permits linkage, and a party with access to the token-to-identity mapping may be able to reverse the substitution. The mapping should be protected separately and made available only when a defined operational purpose requires it.
Aggregation replaces record-level observations with group summaries. It can preserve a group’s total exposure and losses while hiding individual values, but a small or distinctive group can still disclose information. A restricted-access environment takes another approach: detailed data stays in a controlled workspace, and users receive only approved outputs. This can retain analytical detail, but exported tables, charts, and model results still require review.
Assessing Reidentification Risk
A practical assessment begins by identifying the likely user and the information that user could reasonably possess. It then considers which fields could serve as quasi-identifiers, how distinctive their combinations are, and what a successful match would reveal. Risk depends on the release context: a table shared broadly poses a different exposure from a row-level file available to a small, authorized analysis team under controls.
One simple diagnostic is the empirical uniqueness proportion for a chosen set of quasi-identifiers. For a dataset of \(n\) records, group records that have the same quasi-identifier pattern. Let \(U\) be the number of records whose pattern occurs exactly once. Then \(U/n\) is the proportion of records unique on those fields in that dataset. It is a screening measure, not a probability that an attacker will identify a person: it does not account for all external information, attacker capabilities, or safeguards.
A related diagnostic is the group size \(k\): the number of records sharing a given pattern. Small \(k\) means a pattern distinguishes a record from few others in the data. Larger groups generally reduce singling-out risk, but do not establish anonymity. The values might be linked to outside information, and records in the same group could still reveal a sensitive attribute about a subject.
Worked Example: Testing Masked Fields for Uniqueness
A hypothetical insurer removes names and policy numbers from eight records but retains a coarse postal area and age band. The combinations are: (214, 20–29), (214, 20–29), (214, 30–39), (214, 40–49), (215, 30–39), (215, 30–39), (215, 50–59), and (216, 60–69). Calculate the empirical uniqueness proportion for those two fields and explain what it does and does not show.
Method. Group records by the pair (postal area, age band), then count records in groups of size one. The first pattern occurs twice, the fifth pattern occurs twice, and the remaining four patterns each occur once. Thus \(U=4\) and \(n=8\).
Half the records are unique on these two retained fields within this small dataset. Removing the direct identifiers did not make the remaining combinations non-distinctive. If an outside source supplied the same fields for a known person, four records would be candidates for a unique match within this dataset. That observation is a warning to review the fields and release context, not a claim that an actual identification has occurred.
Check. There are four singleton patterns among eight records, so the calculated proportion is 0.50. The common mistake is to count only the number of distinct patterns (six) and divide by eight. The definition counts records in singleton groups, not all distinct groups.
Aggregation: Retained Totals and Lost Detail
Aggregation is often attractive because it can preserve quantities needed for broad rate indications while reducing direct exposure of individual records. Its effect should be measured against the intended use. For example, if a group’s exposure and loss totals are retained, its aggregate loss cost is total loss divided by total exposure. That group result does not reveal how losses vary among the risks inside the group.
The reason a pooled rate is exposure-weighted follows directly from the definitions. Suppose group \(j\) has exposure \(e_j\), loss \(l_j\), and loss cost \(r_j=l_j/e_j\), for positive exposure. The combined loss cost is total losses divided by total exposure. Substituting \(l_j=e_jr_j\) gives:
Thus the combined rate is an exposure-weighted average of group rates. This identity explains what aggregation preserves when the relevant group totals are retained. It does not mean that each group’s rate, distribution of losses, or relationship between variables can be recovered from the combined figure.
Worked Example: Measuring the Effect of Rolling Up Small Areas
A hypothetical portfolio has three small rating areas. Area A has 100 exposures and $80,000 in losses; area B has 50 exposures and $50,000 in losses; area C has 50 exposures and $20,000 in losses. For disclosure review, B and C are combined. Calculate each original loss cost, the combined B–C loss cost, and the portfolio loss cost. State the analytical detail lost by the roll-up.
Method. Divide each area’s losses by its exposure. Area A is \(80{,}000/100=\$800\) per exposure. Area B is \(50{,}000/50=\$1{,}000\) per exposure. Area C is \(20{,}000/50=\$400\) per exposure.
For the combined B–C group, exposures are \(50+50=100\), and losses are \(\$50{,}000+\$20{,}000=\$70{,}000\). Its loss cost is \(\$70{,}000/100=\$700\) per exposure.
The full portfolio has \(100+50+50=200\) exposures and \(\$80{,}000+\$50{,}000+\$20{,}000=\$150{,}000\) in losses. Its loss cost is \(\$150{,}000/200=\$750\) per exposure. The check is the weighted calculation: \((100\times800+100\times700)/200=\$750\).
The roll-up preserves B and C’s combined totals and their combined loss cost, but it no longer supports separate estimates of \(\$1{,}000\) for B and \(\$400\) for C. The method avoids the common mistake of taking the unweighted average \((1{,}000+400)/2=\$700\) as the overall portfolio loss cost; that happens to equal the B–C pooled rate here only because B and C have equal exposures. Unequal exposures require exposure weighting.
Stable Tokens and Controlled Analysis
A stable token can preserve longitudinal linkage without displaying a direct identifier in the analysis file. This is useful when the task requires connecting a risk’s records across terms, such as examining changes in exposure or claims. By contrast, a new unrelated token for every row prevents straightforward linkage. Token design should therefore match the task: use linkage only where it is needed, restrict access to any mapping, and avoid treating the token itself as proof that records cannot be reidentified.
A restricted-access environment can add protection when row-level data is necessary. Typical controls include named-user authorization, least-privilege access, logging, limits on copying or downloading data, and review of outputs before release. These controls reduce opportunities for disclosure; they do not eliminate them. A user may infer sensitive information from repeated queries, or an output may reveal a very small cell even when it contains no names.
Worked Example: Preserving Renewal Linkage Without Exporting Names
In a hypothetical controlled analysis, an authorized analyst receives four rows with stable tokens rather than names. Token R17 has exposure 0.5 and losses of $200 in Year 1, then exposure 0.5 and losses of $500 in Year 2. Token R42 has exposure 1.0 and losses of $900 in Year 1, then exposure 1.0 and losses of $1,100 in Year 2. Calculate total exposure and losses by year, then explain what stable tokens make possible and why the output still needs review.
Method. Sum the exposure and losses for each year. Year 1 exposure is \(0.5+1.0=1.5\), and Year 1 losses are \(\$200+\$900=\$1{,}100\). Its aggregate loss cost is \(\$1{,}100/1.5=\$733.33\) per exposure, rounded to cents.
Year 2 exposure is \(0.5+1.0=1.5\), and Year 2 losses are \(\$500+\$1{,}100=\$1{,}600\). Its aggregate loss cost is \(\$1{,}600/1.5=\$1{,}066.67\) per exposure, rounded to cents. Across both years, total exposure is \(3.0\), total losses are \(\$2{,}700\), and the combined loss cost is \(\$2{,}700/3.0=\$900.00\) per exposure.
Because each token is stable, the analyst can also associate R17’s two annual records and R42’s two annual records, supporting a renewal-level comparison. If tokens were independently replaced on every row, those links would be unavailable without another matching method. The calculated totals check: \(\$1{,}100+\$1{,}600=\$2{,}700\), and \(1.5+1.5=3.0\). The common mistake is to regard a stable token as anonymous by itself. The analysis still has linkable histories, and a released small-cell result or outside information may create disclosure risk.
Selecting a Control Set
The strongest practical design is not necessarily the one that removes the most detail. Excessive coarsening can make a dataset unsuitable for the intended analysis, while retaining every field in a broadly shared file can create avoidable exposure. Choose controls in relation to the purpose, users, data sensitivity, and release route.
Decide whether the work needs record-level linkage, exact dates, fine geographic detail, or only group totals. Do not retain detail without a defined analytical need.
List fields that identify directly and combinations that could distinguish a record when matched with plausible external information.
Consider deleting or generalizing fields, tokenizing identifiers for necessary linkage, aggregating small groups, or keeping detailed data in a restricted environment.
Review uniqueness and small groups, then check that the transformed data still supports the planned calculation. Document what detail was lost and what residual risk remains.
Limit users and permissions, protect any token mapping, log use where appropriate, and review outputs before they leave the controlled setting.
The central trade-off is not simply privacy versus accuracy. A well-designed combination can retain the information needed for a task while reducing exposure, but every transformation changes the analysis in some way. Removing exact dates may obstruct timing studies; aggregation may hide variation relevant to classification; unstable tokens may prevent longitudinal analysis; and stable tokens may permit unwanted linkage. The decision and its limitations should be documented so reviewers can assess whether the data remain appropriate for the intended ratemaking use.
Check Your Understanding
For each hypothetical ratemaking situation, select or assess an appropriate control and check the calculation.
- A 12-record extract removes names but retains exact incident dates and small-town locations. If five records are unique on that pair of fields, what is the empirical uniqueness proportion, and what does it indicate? (Answer: \(5/12=0.4167\), or 41.67% of records are unique on those fields within the extract; it is a screening indicator, not an actual identification probability.)
- Two groups have exposures 40 and 60, with loss costs of $500 and $800 per exposure. What is their combined loss cost? (Answer: \((40\times500+60\times800)/100=\$680\) per exposure.)
- An analysis needs to follow the same insured across three policy years, but the analyst should not see names. Which approach best retains linkage, and what risk remains? (Answer: Use stable tokens with the identity mapping separately protected; records remain linkable, and mapping access or external information may permit identification.)
- A public summary would report one claim in a distinctive small coverage group. Why may aggregation or suppression be appropriate even if names are absent? (Answer: The small, distinctive cell may allow inference about the claimant or sensitive claim information; aggregate or suppress the output and assess the remaining disclosure risk.)
- A detailed claim-level analysis is necessary, but analysts should not download the source file. Which control best fits, and what additional review is still needed? (Answer: Use a restricted-access environment with authorized users and controlled exports; review outputs because small cells or inferential disclosures can still escape.)