Why Grouped Summaries Matter
An overall average can hide useful variation across the records being analyzed. For example, a portfolio-wide average claim payment does not show whether the average differs by product type. A grouped summary answers a specific question: for each value of a chosen grouping variable, what is the requested statistic of the records in that group?
In R, dplyr provides two functions for this task. group_by() identifies the groups, and summarise() calculates a summary within each group. The result is a compact data frame: ordinarily, one row for each observed group and one column for each grouping variable and named summary expression.
What Each Function Does
The usual sequence is to start with a data frame, pass it to group_by(), and then pass the grouped result to summarise(). In a dplyr pipeline, the symbol |> passes the result on to the next function. The following code assumes dplyr is available, for example after running library(dplyr).
R pattern: claims |> group_by(product) |> summarise(mean_paid = mean(paid), .groups = "drop")
Here, product determines which rows belong together. The expression mean(paid) determines what is calculated within each group. The name on the left of the equals sign, mean_paid, becomes the name of the output column. Changing the grouping variable changes the grouping of the rows; changing the summary expression changes the calculated result.
The grouping operation does not itself calculate a mean or reduce the number of rows. It marks the data so that subsequent summary expressions are evaluated separately within each group. Conversely, a summary expression without a grouping variable is calculated over the full input data, producing an overall result rather than one result per category.
Name the column whose values define the groups, such as product type or office.
Specify the calculation, such as mean(paid) or mean(renewed), that should be evaluated within each group.
Give the summary column a meaningful name, then check that the output has the expected group labels and one result per group.
For a mean, the underlying calculation in a group is the sum of its available values divided by the number of values used. If group \(g\) contains \(n_g\) observed values \(x_{g1},\ldots,x_{gn_g}\), its mean is:
This formula explains why the result is group-specific: both the sum and the denominator are calculated using only the rows assigned to that group. A group with fewer records can therefore have a different mean from a larger group, even when the overall mean is similar.
Worked Example: Mean Payment by Product
A hypothetical insurer has seven recorded payments in three product categories. Use dplyr to calculate the mean payment for each product. Payments are in dollars.
Data and method. The records are: Home, 120; Home, 180; Home, 150; Travel, 90; Travel, 210; Marine, 240; Marine, 360. Group by product and calculate the mean of payment.
Home calculation. The sum is \(120+180+150=450\), and there are 3 records. The mean is \(450/3=150\) dollars.
Travel calculation. The sum is \(90+210=300\), and there are 2 records. The mean is \(300/2=150\) dollars.
Marine calculation. The sum is \(240+360=600\), and there are 2 records. The mean is \(600/2=300\) dollars.
R implementation: payments |> group_by(product) |> summarise(mean_payment = mean(payment), .groups = "drop")
The result is one row for each product, with mean_payment equal to 150, 150, and 300 dollars for Home, Travel, and Marine, respectively. As a check, the group record counts are 3, 2, and 2, which add to all 7 input records.
Common mistake and correction. If group_by(product) is omitted, mean(payment) is calculated once over all seven records: the total is \(450+300+600=1{,}350\), so the overall mean is \(1{,}350/7\approx192.86\) dollars. That single pooled result does not answer the question about each product. Add group_by(product) before summarise() to obtain the three group-specific means.
Missing Values Must Be Handled Deliberately
By default, R’s mean() returns a missing result if any value supplied to it is missing. In a grouped summary, a missing payment in one group can therefore make that group’s mean appear as NA. If the intended calculation is the mean of the available recorded payments, specify na.rm = TRUE inside mean(). This removes missing entries from that calculation; it does not replace them with zero.
Whether to exclude missing values is an analytic decision, not merely a coding preference. The mean with na.rm = TRUE describes the nonmissing values. If missingness is related to payment size or group membership, that mean may not represent the values that were not recorded. State what was calculated and check how the missing values affect interpretation.
Worked Example: Grouped Means with Missing Payments
A hypothetical claims file records payment amounts for two service areas. The East records are 40, missing, and 80 dollars. The West records are 50, 70, and 90 dollars. Calculate the mean of the nonmissing payments in each area.
East calculation. Exclude the missing entry for this calculation. The observed sum is \(40+80=120\), and 2 payments are observed, so the mean is \(120/2=60\) dollars.
West calculation. The sum is \(50+70+90=210\), and all 3 payments are observed, so the mean is \(210/3=70\) dollars.
R implementation: claims |> group_by(area) |> summarise(mean_payment = mean(payment, na.rm = TRUE), .groups = "drop")
The output has mean_payment values of 60 dollars for East and 70 dollars for West. The calculation uses 2 observed payments for East and 3 for West, so the denominators are not the same.
Common mistake and correction. Writing mean(payment) without na.rm = TRUE gives NA for East because that group contains a missing value; it gives 70 for West. It is incorrect to treat East’s missing value as zero, which would produce \((40+0+80)/3=40\) dollars. The requested mean of the available East payments is 60 dollars, obtained with na.rm = TRUE.
A Logical Mean as a Grouped Proportion
A useful summary expression for a logical variable is its mean. In R, TRUE is treated as 1 and FALSE as 0 in arithmetic. If a group contains \(r_g\) TRUE values among \(n_g\) nonmissing logical values, its mean is \(r_g/n_g\), the proportion that are TRUE. This connection allows a grouped summary to calculate a rate or proportion without first creating a separate numeric variable.
The interpretation depends on what the logical column represents. For example, mean(renewed) is the proportion of records marked TRUE for renewal, not a mean of dollar amounts. Missing logical values still require a deliberate missing-value decision; use na.rm = TRUE only if the intended proportion is among the nonmissing records.
Worked Example: Renewal Proportion by Plan
A hypothetical book of policies records whether each policy renewed. Basic has the values TRUE, FALSE, TRUE, TRUE, FALSE. Plus has the values FALSE, FALSE, TRUE, FALSE. Calculate the renewal proportion for each plan.
Basic calculation. TRUE contributes 1 and FALSE contributes 0. The sum is \(1+0+1+1+0=3\), across 5 policies, so the mean is \(3/5=0.60\). The renewal proportion is 0.60, or 60%.
Plus calculation. The sum is \(0+0+1+0=1\), across 4 policies, so the mean is \(1/4=0.25\). The renewal proportion is 0.25, or 25%.
R implementation: policies |> group_by(plan) |> summarise(renewal_proportion = mean(renewed), .groups = "drop")
The output has one row per plan and reports 0.60 for Basic and 0.25 for Plus. These values are proportions, so multiply by 100 to express them as percentages. The sum of the logical values and the denominator in each calculation verify both results.
Common mistake and correction. Counting only TRUE values without dividing by the number of policies would report counts of 3 and 1, not proportions of 0.60 and 0.25. The mean performs the division by the number of logical values in each group.
Interpreting the Output Carefully
A grouped summary is only as clear as its grouping variable and summary expression. Before interpreting the numbers, read the output column names and ask what each row represents. If the grouping variable is product, a row is a product group; if the expression is mean(payment), the summary column contains mean payments. A column named average or result may be technically valid, but a descriptive name makes the output easier to interpret.
For one grouping variable and a summary that returns one value per group, each group contributes one output row. Using .groups = "drop" explicitly asks dplyr to return an ungrouped result after summarising. That makes the output’s status clear if it is passed to later calculations. The group labels remain as ordinary columns; dropping grouping does not remove them.
The grouping variable also determines which distinctions the result preserves. If records are grouped by plan, the summary is organized by plan. The summary does not automatically identify other subgroups or calculate other statistics. The expression you supply is the request: a mean, median, or logical mean gives a different kind of result even if the groups are unchanged.
Check Your Understanding
For each problem, identify the group-specific calculation and check that the final output has one result for each stated group.
- A hypothetical file has Red office payments of 10 and 20 dollars, and Blue office payments of 15, 25, and 35 dollars. What would group_by(office) followed by summarise(mean_payment = mean(payment)) return? (Answer: Red = 15 dollars; Blue = 25 dollars.)
- A service file has A values of 12, NA, and 18, and B values of 10 and 14. If the requested statistic is the mean of observed values, what expression and results are appropriate? (Answer: Use mean(value, na.rm = TRUE); A = 15 and B = 12.)
- For a logical outcome, Basic has TRUE, FALSE, FALSE and Plus has TRUE, TRUE, FALSE. What proportions result from grouping by plan and summarising with mean(outcome)? (Answer: Basic = 1/3; Plus = 2/3.)
- A data set has East values 4, 8, 12 and West values 3, 9, 15, 21. If grouped by region, what does summarise(median_value = median(value)) return? (Answer: East = 8; West = 12.)
- A hypothetical renewal file has Silver outcomes TRUE, FALSE, TRUE, TRUE and Gold outcomes FALSE, TRUE. Calculate the renewal proportion in each tier, and state why counting TRUE values alone would not give the requested result. (Answer: Silver = 3/4 = 0.75; Gold = 1/2 = 0.50. Counts alone would be 3 and 1, not proportions.)