Why Model Data Can Disclose More Than It Appears To
A model audit may use data without names or account numbers and still expose people. A combination such as age band, location, occupation, and policy tenure can be distinctive. If the same combination appears in an external source that identifies people, linking the records may reveal who is in the audit data. Even when identities are not exposed, detailed group results can reveal a sensitive outcome about a person or a small group.
These risks apply to both data access and reporting. A row-level file can be linked to an external dataset, while a table of aggregate results can disclose information through small cells or arithmetic across multiple releases. The appropriate safeguard depends on what an authorized or unauthorized recipient can reasonably combine with the data.
Removing direct identifiers is not the same as eliminating disclosure risk. Quasi-identifiers can be combined across sources, and supposedly anonymous data can remain linkable. Risk also depends on the release: a fine-grained table can identify a group member or reveal an outcome even if the source data were access-controlled.
How Linkage Narrows the Candidate Set
A practical way to assess linkage is to ask how many people match the released characteristics, then how many remain when those characteristics are compared with each external source. Each source can narrow the candidate set. If only one person remains, the combination supports a unique identification, assuming the matching information is accurate.
Here, each \(C\) is a set of candidate identities matching the available characteristics. The justification is set intersection: an identity consistent with all the evidence must belong to every candidate set. If the intersection contains one identity, the evidence has reduced the candidate set to a singleton. If it contains several, identification is not unique from that evidence alone; other information may still narrow it further.
Worked Example: Linking an Audit Record to a Named Source
A hypothetical insurer prepares an audit file for 600 policyholders. Names are removed, but the file includes age band, postal district, occupation group, and vehicle class. An analyst is checking whether a particular record can be linked to a public, named event directory. Work through the candidate sets and assess the risk.
Step 1: Use the audit-file characteristics. The target combination of age band, postal district, and occupation group matches four audit records. Write this candidate set as \(C_{\text{audit}}=\{p_1,p_2,p_3,p_4\}\). At this point the combination does not single out one row within the audit file.
Step 2: Compare with the named directory. The same combination matches two named people in the directory, so \(C_{\text{directory}}=\{p_2,p_3\}\), using the corresponding identities. The intersection is \(\{p_2,p_3\}\), leaving two candidates.
Step 3: Use vehicle class as additional information. Of the four audit records, the target vehicle class occurs for \(p_1\) and \(p_3\). In the named directory, only \(p_3\) among the two candidates has that vehicle class. The combined candidate set is therefore \(\{p_2,p_3\}\cap\{p_1,p_3\}=\{p_3\}\).
Conclusion and check. The final set contains one identity, so the record is uniquely linkable using these sources. The calculation does not establish that every record in the file is identifiable; it establishes a disclosure risk for this record and this combination of sources. The method is candidate-set intersection. It avoids the common mistake of treating each quasi-identifier as harmless because none is a name.
Disclosure Through Small Cells and Related Totals
A table can disclose a sensitive attribute even when no person is named. If a group is small and its membership is known, an aggregate outcome can reveal the outcome of a member. More subtly, a suppressed cell may be calculated from other published figures. Suppressing a number in one place is not sufficient if the same number can be recovered by subtraction elsewhere.
Worked Example: Recovering a Suppressed Audit Cell
A hypothetical audit reports positive outcomes by worksite. The underlying counts are: worksite A has 18 people and 10 positive outcomes; worksite B has 4 people and 1 positive outcome; and worksite C has 18 people and 0 positive outcomes. The released summary includes overall totals, worksite A, worksite C, and marks worksite B’s outcome count as suppressed. Determine whether the suppression protects B’s count.
Step 1: Check the published positive-outcome total. The overall number of positive outcomes is \(10+1+0=11\).
Step 2: Apply subtraction using the released cells. Worksite A contributes 10 and worksite C contributes 0. Therefore the suppressed worksite B count is \(11-10-0=1\).
Step 3: Check the group size. The overall group size is \(18+4+18=40\). Since A and C are both reported, B’s size is also recoverable as \(40-18-18=4\).
Conclusion and correction. The primary suppression fails: both B’s group size and its positive-outcome count can be derived. A safer release would avoid publishing the combination of totals and cells that isolates B. For example, it could report A separately and combine B and C into one group, showing \(22\) people and \(1\) positive outcome, while not separately publishing C’s outcome count. This reduces the direct subtraction route, although it does not guarantee that no outside information can reveal an individual fact. The method is a disclosure check across the whole release, not just a review of the visibly suppressed cell. The common mistake is to assume that a suppression marker makes the underlying value unavailable.
A small-cell threshold is a useful screening rule, not proof of privacy. For example, a release might require at least five records in each reported cell. A cell meeting that threshold can still be distinctive in combination with other released fields, or its outcome can be inferred if all members share that outcome. Conversely, a small group may be reportable in a controlled internal analysis when access and use are appropriately restricted. Assess the table, its companion releases, and the recipient’s likely information together.
Choosing Aggregation and Access Controls
For an audit, the goal is to retain information needed to assess model performance while reducing avoidable exposure. Aggregation can combine compatible categories, broaden time periods, or remove a cross-tabulation that creates a distinctive combination. Suppression may be needed when a cell remains too small, but related totals must also be reviewed for subtraction or other inference. Access controls address a different part of the problem: they limit who can view row-level data and what they can do with it.
Worked Example: Reshaping a Small-Group Audit Table
A hypothetical renewal model audit reports denial rates by region and age band. A release policy sets a minimum of five records per reported cell. The initial table is: rural-younger, 2 records and 2 denials; rural-older, 4 records and 3 denials; urban-younger, 7 records and 4 denials; urban-older, 11 records and 5 denials. Choose a table structure that meets the stated minimum and compute the rates.
Step 1: Identify cells that fail the minimum. The rural-younger cell has 2 records and the rural-older cell has 4. Both are below five. The urban cells have 7 and 11 records, so they meet the minimum.
Step 2: Combine compatible rural age bands. If the audit question permits a rural-only comparison without an age split, the combined rural cell has \(2+4=6\) records and \(2+3=5\) denials. Its denial rate is \(5/6=0.8333\), or 83.33% to two decimal places.
Step 3: Compute the urban rates. The urban-younger rate is \(4/7=0.571428\ldots\), or 57.14% to two decimal places. The urban-older rate is \(5/11=0.454545\ldots\), or 45.45% to two decimal places.
Step 4: Check the proposed release. The three reported cell sizes are 6, 7, and 11, each at least five. The rates follow directly from the counts. The release preserves a regional comparison but no longer provides a rural age-specific comparison, so the report should state that limitation. It should also check whether other published totals could reconstruct the rural age-specific values.
Conclusion and control choice. The method is compatible-category aggregation followed by a release review. The common mistake is to treat meeting the five-record threshold as a guarantee that records cannot be identified or outcomes inferred. For the audit, retain row-level data only for authorized analysts with a defined need; release reviewed aggregate tables to broader audiences; restrict downloads or onward sharing where feasible; and log access and output review. These controls reduce exposure but do not replace consideration of the information already available to a recipient.
Practical Review for an Audit
A responsible disclosure review considers both the purpose of the audit and the route by which information could escape. An audit may need detailed group estimates to detect unequal errors, but unnecessary row-level fields or an overly fine public cross-tabulation can add risk without improving the decision. In an actuarial setting, the audit design should therefore distinguish what analysts need to inspect from what a report needs to communicate.
Specify the fairness or performance question and the minimum detail needed to answer it. Avoid collecting or retaining fields solely because they might be useful later.
List direct identifiers, quasi-identifiers, and sensitive attributes. Consider combinations of fields, not only fields examined separately.
Consider what candidate sets external sources could create, whether a person could be singled out, and whether totals or overlapping tables reveal a suppressed count or outcome.
Aggregate compatible groups, broaden categories or periods where appropriate, and review small cells and related totals. Document which comparisons are lost or weakened by the change.
Limit row-level access to authorized roles with a justified need, use secure storage and transfer, and review outputs before wider release. Reassess if the data, audience, or external information changes.
For an exam-style analysis aligned with the SOA Exam PA (Predictive Analytics) syllabus, separate the risk from the proposed remedy. State what could be learned, from which fields or releases, and by whom; then explain how the proposed aggregation or access control reduces that route while acknowledging the resulting analytical limitation. “Names were removed” and “all cells exceed five” are not complete risk assessments.
Check Your Understanding
For each problem, identify the disclosure route or calculate the requested quantity, then compare your result with the answer.
- A de-identified audit file has five records matching a particular age band and district. A named public list has three people matching those fields. A released occupation category narrows both lists to two common candidates. How many candidates remain, and is the evidence sufficient for unique reidentification? (Answer: Two candidates remain; not sufficient for unique reidentification from these fields alone.)
- A report gives 50 total cases, 13 positive outcomes overall, 20 cases and 8 positive outcomes in group A, and 25 cases and 4 positive outcomes in group C. Group B is suppressed. Recover B’s case count and positive-outcome count. (Answer: 5 cases and 1 positive outcome.)
- A reporting rule requires at least five records per cell. Two compatible cells have 3 records with 1 event and 4 records with 2 events. If they are combined, what are the combined event rate and cell size? (Answer: 7 records; \(3/7=0.4286\), or 42.86%.)
- A release reports a group of 8 people, all with the same sensitive outcome, but no names. A recipient knows the identities of all 8 members. What type of disclosure is possible, and why does the group size not prevent it? (Answer: Attribute disclosure; membership is known and the aggregate establishes the shared outcome, so a minimum-size rule alone does not protect the attribute.)
- An audit requires row-level quasi-identifiers for analysts to investigate model errors, but external readers need only regional error rates. Name one suitable data-access control and one suitable reporting control. (Answer: Restrict row-level access to authorized analysts with a justified need; release reviewed regional aggregates and check for small-cell or subtraction disclosure.)