Why Reproducibility Matters
A ratemaking result is useful only if its path from source data to reported output can be repeated and explained. A rerun may otherwise differ because a data extract changed, a transformation used a different date basis, a model setting was left at a new default, or a simulation used a different random-number sequence. These differences can be mistaken for genuine changes in experience or model behavior.
A reproducible ratemaking workflow specifies the inputs and operations that produced an output, and preserves enough information to rerun those operations under stated conditions. Reproducibility is broader than saving a final table or a script: it includes the data, the sequence of transformations, configuration choices, software dependencies, and any randomness that affects the result.
This definition does not mean every computing environment must produce identical bits. Optimization algorithms can stop at slightly different numerical values, and different hardware or library implementations can change rounding at a very small scale. A workflow should therefore say whether it expects exact equality or agreement within an explicit tolerance, and explain why that standard is suitable for the output.
Specify the Run Before Calculating
A useful run description identifies the data as they were used, not merely the system from which they originally came. It should record the extract date or other stable identifier, relevant period, population, grain of the records, units, and any filters. It should also specify how the data were transformed. For example, “losses were trended” is incomplete unless the trend rate, time basis, and order of operations are known.
The run description should also capture model and execution settings: selected response and predictors, exposure definition, offsets, weights, fitting options, convergence criteria, software and package versions, and relevant configuration values. If a step draws random numbers, record the seed, generator, and how random draws are assigned to tasks or parallel processes. A seed alone may not be enough when software versions or random-number implementations differ.
Record the data extract, covered period, included population, record grain, and units. Preserve the exact input used for the run or a stable way to retrieve it.
Write down filters, joins, exclusions, adjustments, aggregation rules, and the order in which they are applied.
Record model choices, parameters, software environment, and any random-number controls that can affect the output.
Compare key totals and results with expected values, record the tolerance for numerical comparisons, and retain the run description with the output.
This process is a practical control, not paperwork for its own sake. It makes it possible to distinguish a real change in indicated rates from a change caused by a different extract, a revised transformation, or an altered computational setting.
Make Transformations Explicit
Each transformation should define its inputs, rule, and output. A transformation is easier to reproduce when it is broken into small named steps rather than hidden inside one long calculation. For example, specify whether loss amounts are nominal or adjusted, which period’s level is the target, and whether the adjustment happens before or after aggregation. Unit conventions matter as well: dollars and thousands of dollars cannot be combined safely without an explicit conversion.
A simple reconciliation can test the transformation chain. If a step filters records, compare counts and exposure before and after the filter. If a step adjusts losses, compare unadjusted totals with adjusted totals. These checks do not prove that the transformation is appropriate, but they can reveal unintended changes or inconsistencies.
Worked Example: Reproducing a Loss Trend Calculation
A hypothetical rating analysis has three records from a common historical period. Their losses are $600, $900, and $600, and their exposures are 0.50, 0.75, and 0.50 exposure-years. The workflow targets a period two years later and applies a 5% annual compound trend. Assume all three losses use the same basis, all exposures are measured in the same unit, and no other adjustments are made. Calculate the trended aggregate loss and loss per exposure-year.
Method. Apply the same two-year trend factor to each loss, then divide the trended aggregate by total exposure. The assumed annual trend rate is 5%, compounded once for each of the two years.
First reconcile the inputs. Total historical loss is $600 + $900 + $600 = $2,100. Total exposure is 0.50 + 0.75 + 0.50 = 1.75 exposure-years.
Apply the trend factor record by record: $600 × 1.1025 = $661.50; $900 × 1.1025 = $992.25; and $600 × 1.1025 = $661.50. The adjusted total is $661.50 + $992.25 + $661.50 = $2,315.25.
The loss per exposure-year is $2,315.25 ÷ 1.75 = $1,323.00. As a check, the historical loss per exposure-year is $2,100 ÷ 1.75 = $1,200.00, and $1,200.00 × 1.1025 = $1,323.00. The two calculation paths agree, so the aggregate calculation reconciles.
Common mistake avoided. Applying 5% only once because the rate is stated as an annual rate gives $1,200 × 1.05 = $1,260.00. That is not the specified two-year compound adjustment. The workflow must record both the annual rate and the number of years.
Reproduce Model Calculations, Not Just Model Files
A saved model object or a table of coefficients does not, by itself, define a reproducible rating calculation. The run also needs the inputs and settings that determine how those coefficients are applied. For a generalized linear model, relevant details can include the response definition, link, predictors, offset or exposure treatment, estimation settings, and software implementation. For a rating indication, the steps that convert model output to the displayed amount should be explicit too.
The following example uses a simple Poisson frequency model to show why exposure and model settings belong in the run specification. It is an arithmetic illustration, not a claim that every frequency model should use these assumptions.
Worked Example: Reproducing an Expected-Loss Calculation
A hypothetical model run uses \(x=1\), exposure \(e=0.40\) exposure-years, coefficients \(\beta_0=\ln(0.08)\) and \(\beta_1=\ln(1.25)\), and an assumed average severity of $7,500 per claim. Calculate expected claim count and expected loss. Assume the coefficients and severity are fixed inputs for this calculation, with no additional trend, expense, or premium adjustment.
Method. Use the specified Poisson mean, then multiply by average severity. The run settings must identify \(x=1\), exposure in exposure-years, both coefficients, and the severity assumption.
Substitute the given values. Since \(\beta_0=\ln(0.08)\), \(\beta_1=\ln(1.25)\), and \(x=1\), the frequency per exposure-year is \(\exp(\ln(0.08)+\ln(1.25))=0.08\times1.25=0.10\).
Expected claim count is \(0.40\times0.10=0.04\) claims. Expected loss is \(0.04\times$7,500=$300.00\).
Check. The model’s frequency for this risk is 0.10 per exposure-year. At 0.40 exposure-years, the expected count is 40% of 0.10, or 0.04. Multiplying that count by the stated severity gives $300.00.
Common mistake avoided. Treating 0.40 exposure-years as 40 exposure-years would give an expected count of 4.00 and expected loss of $30,000.00, a factor of 100 too high. Units and exposure conventions must be recorded alongside the model settings.
Control Randomness and the Software Environment
Some ratemaking analyses use random draws, for example in a bootstrap, simulation, or resampling procedure. A random seed initializes a pseudorandom-number generator, allowing it to produce a repeatable sequence under specified conditions. But the same numerical seed does not necessarily give the same results if the generator, software version, draw order, or parallel execution differs.
For a reproducible stochastic run, document the seed and the generator implementation, as well as the sampling method, number of draws, and any parallelization settings that affect draw assignment. If the workflow must be independently checked, retaining the sampled indices or another compact record of the draws can make the arithmetic verifiable. Do not present a seed by itself as proof that a simulation can be reproduced across all environments.
Worked Example: Checking a Bootstrap Run
A hypothetical bootstrap estimates average claim severity from three observed losses: $4,000, $6,000, and $10,000. Each bootstrap replicate samples three observations with replacement. The run description records seed 314 and the generator and software configuration. For this example, the recorded sampled indices are replicate 1: (1, 3, 2), and replicate 2: (2, 2, 3). Calculate both replicate means and their average. Assume these recorded indices are the draws to check; the seed is not being used here to infer the indices.
Method. Map each index to the corresponding observed loss, calculate the mean within each replicate, and then average the two replicate means. This checks the recorded calculation while keeping the random-number details needed for a full rerun explicit.
Replicate 1 selects $4,000, $10,000, and $6,000. Its total is $4,000 + $10,000 + $6,000 = $20,000, so its mean is $20,000 ÷ 3 = $6,666.6667.
Replicate 2 selects $6,000, $6,000, and $10,000. Its total is $22,000, so its mean is $22,000 ÷ 3 = $7,333.3333.
The average of the two replicate means is \(($6,666.6667 + $7,333.3333) ÷ 2 = $7,000.00\). Rounded only at the final step, the replicate means are $6,666.67 and $7,333.33, and their average is $7,000.00. The calculation is consistent with the two recorded index lists.
Common mistake avoided. Recording seed 314 but not the generator or draw procedure may fail to reproduce these indices in another environment. The sampled losses also cannot be reconstructed from the final average alone; the replicate-level details are needed to check the result.
Set Practical Reproduction Checks
A workflow should define what “same result” means for its outputs. Counts of records and category labels may be expected to match exactly. Dollar outputs may need to match to the cent if the underlying process is deterministic and uses fixed rounding rules. Iterative model estimates may instead be compared within a stated tolerance, provided the tolerance is small enough for the intended use and the comparison covers relevant outputs.
Use checks at several points rather than relying only on the final indicated rate. Useful checks include input record counts, exposure and loss totals, transformation totals, model convergence status, coefficient or prediction comparisons, and final rounded outputs. If a check fails, retain enough run information to identify whether the difference arose from inputs, processing, settings, or execution.
A common but weak practice is to save a final spreadsheet and assume that the result can be recreated from it. The spreadsheet may show the answer without specifying the source extract, filtering, units, model configuration, or software dependencies. A reproducible run documents the chain that produced the answer and defines how the rerun will be assessed.
Check Your Understanding
For each problem, identify the calculation or run details needed to reproduce the stated result.
- A hypothetical loss total of $500 from 0.50 exposure-years is trended for three years at 4% per year, compounded annually. Calculate the trended loss per exposure-year. (Answer: trend factor \(1.04^3=1.124864\); trended loss $562.43; loss per exposure-year $1,124.86.)
- A Poisson frequency model has base frequency 0.12 per exposure-year, a rating relativity of 0.90, exposure of 0.25 exposure-years, and assumed severity of $6,000. Find expected claim count and expected loss, and state one run setting that must be recorded. (Answer: expected count \(0.12\times0.90\times0.25=0.027\); expected loss $162.00; record, for example, the exposure unit or model coefficients.)
- Two records have losses of $900 and $700 and exposures of 0.60 and 0.40. Both losses are trended by 3% annually for two years. Find the adjusted aggregate loss per exposure-year. (Answer: factor \(1.03^2=1.0609\); aggregate loss $1,600; adjusted loss $1,697.44; total exposure 1.00; result $1,697.44 per exposure-year.)
- Observed losses are $50, $100, and $200. Two bootstrap replicates use index lists (1, 2, 3) and (3, 3, 1). Calculate each replicate mean and the average of the two means. (Answer: $116.67, $150.00, and $133.33, respectively.)
- A rerun has the same seed as the original but a different random-number generator version and different parallel settings. Is the seed alone sufficient to establish that the simulation should match? (Answer: No; record the generator and software configuration, sampling procedure, and parallel settings as well as the seed.)