Capacity Constraints and Prioritizing Cases

When Capacity Changes the Prediction Task

A team may be able to review only a fraction of the cases that arrive. In that situation, it is not enough to ask, “What outcome should we predict?” The problem also requires a clear statement of which cases can be reviewed, how many can be reviewed, and what the team wants to prioritize. These are related questions, but they are not the same task.

An outcome-prediction task assigns a prediction to each eligible case. A capacity-limited review task uses information about those cases to choose a subset for human attention. If the team can review 12 cases from a queue of 100, for example, the model may produce predictions for all 100, while the operational decision is which 12 to send for review. The cases not selected are not thereby predicted to have no relevant outcome.

Definition: A capacity constraint is a limit on the amount of review or other operational work that can be performed in a specified period. Prioritization is the process of ordering or selecting eligible cases under that limit, using a stated objective and information available when the selection is made.

A well-framed task therefore identifies the prediction unit, the outcome to be predicted, the time at which the prediction is made, the eligible queue, the capacity, and the selection objective. The earlier material on defining the unit of analysis, prediction timing, forecast horizon, and decision-time information is useful here: the cases must be comparable in these respects for their scores to support a meaningful queue.

Scores, Selection, and Expected Yield

Suppose each eligible case $i$ has a binary outcome $Y_i$, where $Y_i=1$ denotes the specified outcome occurring within the defined forecast horizon and $Y_i=0$ denotes that it does not. Let $p_i$ be the predicted probability of that outcome, based only on information available at the prediction time. The outcome must be defined consistently for all cases; a vague target such as “needs attention” is not a usable substitute for a stated event and outcome window.

If review effort is the same for every case and the objective is to find as many cases with $Y_i=1$ as possible, selecting the cases with the largest predicted probabilities is appropriate. The capacity determines how far down the ranked list the team can go. A probability cutoff such as 0.50 is not automatically the right rule: the cutoff that results from the capacity may be higher or lower.

Result: For a selected set $S$, if $p_i=P(Y_i=1\mid\text{information available at prediction time})$, then the expected number of positive outcomes in the set, conditional on that information, is $$ E\left[\sum_{i\in S}Y_i\mid\text{information}\right]=\sum_{i\in S}p_i. $$ This result does not require the outcomes for different cases to be independent.

The equation describes the expected count for a chosen set; it does not guarantee how many positive outcomes will actually occur. Its use also depends on the probabilities referring to the same event and horizon. If the model supplies only an uncalibrated score, the scores may still be useful for ranking, but summing them as if they were probabilities need not estimate the expected count correctly.

Define the queue.
Specify which cases are eligible for selection and when they enter the queue.

State the outcome and horizon.
Make clear what event is predicted and the period in which it is assessed.

Measure capacity.
State whether the constraint is a number of cases, staff hours, or another resource, and specify the period it covers.

Choose the selection objective.
For example, the objective might be to maximize the expected number of cases with the specified outcome within the available capacity.

Apply a selection rule.
Use the rule that fits both the objective and the operational constraint; do not assume that a single probability cutoff always fits.

Why Ranking by Probability Works When Effort Is Equal

The justification for choosing the largest probabilities is a direct exchange argument. Suppose the team has $K$ review slots and two cases, $a$ and $b$, have $p_a>p_b$. If a proposed set includes $b$ but not $a$, replacing $b$ with $a$ keeps the set size at $K$. Its expected number of positive outcomes changes by $p_a-p_b>0$, so the replacement improves the objective.

\sum_{i\in (S\setminus\{b\})\cup\{a\}}p_i-\sum_{i\in S}p_i=p_a-p_b>0.

Repeating this replacement eliminates any selected case with a lower probability than an unselected case. Therefore, selecting the $K$ highest probabilities maximizes the expected count when each case uses one equal slot and the objective is expected positive cases. If two cases tie at the selection boundary, either can be selected without changing that expected count; an additional, stated operational rule can resolve the tie.

Worked Example: Choosing Two Equal-Effort Cases

A hypothetical service team can review exactly two of five referrals today. Each review takes the same amount of time. The model predicts, for each referral, the probability that the referral will meet a defined specialist-escalation criterion within 30 days: A, 0.42; B, 0.35; C, 0.28; D, 0.17; and E, 0.08. The stated objective is to maximize the expected number of reviewed referrals that meet the criterion. Which referrals should be selected, and what is their expected count?

Method. Because the team has two equal-effort slots and the objective is expected positive count, rank the probabilities from largest to smallest: A, 0.42; B, 0.35; C, 0.28; D, 0.17; E, 0.08.

The two selected referrals are A and B. Their expected positive count is the sum of their predicted probabilities: $0.42+0.35=0.77$.

As a check, the next-highest available probability is C at 0.28, so replacing B with C would give $0.42+0.28=0.70$, which is lower than 0.77. Thus the selected pair is consistent with the stated objective. The count 0.77 is an expectation, not a claim that a fraction of a referral will meet the criterion. No independence assumption between referral outcomes is needed to calculate this expected count.

Common mistake avoided: using 0.50 as a mandatory probability threshold and concluding that only A can be reviewed. The task is to fill two available slots by priority, not to classify cases using a fixed 0.50 cutoff.

When Cases Require Different Amounts of Review

Equal-slot ranking does not solve every capacity problem. Some cases may require different review times. If case $i$ takes $t_i$ hours and has predicted probability $p_i$, then selecting cases with the largest $p_i$ can produce an infeasible set, while ranking only by $p_i/t_i$ is not guaranteed to find the best feasible set. For indivisible cases, the team must choose a combination that fits the total time limit and optimizes the stated objective.

Formulation: If $z_i=1$ means case $i$ is selected and $z_i=0$ means it is not, then one possible objective is to maximize expected positive cases subject to a review-time capacity $C$: $$ \text{maximize }\sum_i p_i z_i \quad\text{subject to}\quad \sum_i t_i z_i\le C,\qquad z_i\in\{0,1\}. $$ This formulation assumes the objective is the expected number of positive outcomes, cases are selected as whole units, and $t_i$ and $C$ are expressed in the same time units.

This is a discrete selection problem: a case cannot usually be reviewed for half of its required time. Dividing probability by time may be a useful descriptive comparison, but choosing the highest ratio one by one is not a generally valid optimization rule. A combination of cases with lower individual ratios can sometimes fit the capacity better and produce a larger total expected count.

Worked Example: Selecting a Feasible Set Under a Time Limit

A hypothetical review team has four hours available. It must choose whole cases, and its objective is to maximize the expected number of cases meeting a specified audit criterion. The predicted probabilities and review times are: A, 3 hours and 0.75; B, 2 hours and 0.60; C, 2 hours and 0.59; D, 1 hour and 0.20. Find the best feasible set.

Method. For each feasible combination, add the review times and add the predicted probabilities. The probability sum is the expected number meeting the criterion under the stated objective.

A and B together require $3+2=5$ hours, so they are infeasible. A and C also require $3+2=5$ hours and are infeasible. A and D require $3+1=4$ hours, with expected count $0.75+0.20=0.95$.

B and C require $2+2=4$ hours, with expected count $0.60+0.59=1.19$. B and D require $2+1=3$ hours, with expected count $0.60+0.20=0.80$. C and D require $2+1=3$ hours, with expected count $0.59+0.20=0.79$. The single-case possibilities have expected counts 0.75 for A, 0.60 for B, 0.59 for C, and 0.20 for D; selecting no cases gives 0.

Among the feasible combinations, B and C have the largest expected count, 1.19. They use all four available hours, so the selected set is feasible and optimal among the listed possibilities.

Common mistake and correction: selecting A first because it has the largest individual probability and then adding D gives a feasible expected count of 0.95. That is not the best set: comparing feasible combinations shows B and C yield 1.19. A high score for one case does not by itself establish that a set is optimal.

Frame the Prediction Separately From the Review Queue

A capacity limit on review does not necessarily imply a limit on which cases receive predictions. If the goal is to rank the whole incoming queue, the model’s predictions must be available for all eligible cases before selection. The team can then select the cases its capacity permits. It is important to distinguish that scoring operation from the review itself: an unselected case still has a prediction, even though it receives no review in the current period.

The distinction also matters when describing the target. “Which cases will be selected for review?” describes a selection outcome, not the substantive event the team may want to predict. If the model is intended to prioritize cases that meet a defined criterion, that criterion should be the target. Otherwise, a model may simply reproduce an existing queue policy without answering the operational question the team intended to address.

Capacity should be stated in operational terms. “Review 10 cases per day” is different from “use no more than 20 staff-hours per day” if cases take different amounts of time. The period matters too: a weekly capacity cannot be applied to a daily queue without specifying how cases are carried forward or re-ranked. Likewise, the prediction time must precede selection, and the predictors used to rank the cases must be available then.

When capacity changes, the selected boundary can change even if every case’s predicted probability stays the same. For equal-effort cases, a capacity of $K$ selects the $K$ highest probabilities; changing $K$ changes the number selected and potentially the lowest score in the selected set. This is another reason to describe the capacity and time period as part of the problem definition rather than treating them as an afterthought.

State the objective. “Prioritize high-risk cases” is incomplete unless “risk” and the purpose of prioritization are made precise.
Match the rule to effort. Highest-probability ranking follows from equal review effort and an expected-count objective; variable effort requires considering feasible combinations.
Do not confuse selection with prediction. A case outside the review capacity is not automatically a predicted negative.
Describe the boundary carefully. A capacity-based selection rule does not imply a fixed probability threshold across days or teams.

A clear capacity-limited task can be summarized as follows: define the eligible cases and prediction time; predict a consistent outcome over a stated horizon; describe the operational capacity and its time unit; and choose a selection objective that matches the intended use. This framing is aligned with the SOA Exam PA (Predictive Analytics) syllabus because it connects a predictive task to the actual constraints under which its scores will be used.

Worked Example: Scoring a Queue Before Selecting Cases

A hypothetical claims unit receives six cases in one morning and can review two before the end of the day. Each review takes the same amount of time. The intended target is whether a case will meet a stated documentation-deficiency criterion within the next 14 days. The predicted probabilities, available before the queue is assigned, are A, 0.18; B, 0.63; C, 0.47; D, 0.71; E, 0.39; and F, 0.55. The objective is to maximize the expected number of selected cases that meet the criterion. Specify the predictions and the review selection.

Method. Keep the prediction task and selection task distinct. The outcome prediction is defined for all six cases, so each receives its stated probability. Then rank the probabilities: D, 0.71; B, 0.63; F, 0.55; C, 0.47; E, 0.39; A, 0.18.

The capacity is two equal-effort reviews, so select D and B. The expected number of selected cases meeting the criterion is $0.71+0.63=1.34$. To check the boundary, replacing B with the next-ranked case F gives $0.71+0.55=1.26$, which is smaller.

All six cases have predictions, but only D and B enter the review queue today. The other four are not assigned predicted outcomes of zero; their predicted probabilities remain 0.55, 0.47, 0.39, and 0.18. The calculation assumes that the probabilities correspond to the same 14-day target and that review effort is equal. It does not require independence among the six outcomes.

Common mistake avoided: reporting only two model outputs because only two cases can be reviewed. That would leave the unit unable to rank the full eligible queue. The prediction coverage and the review capacity are separate parts of the framing.

Check Your Understanding

For each problem, identify the selection rule implied by the stated objective and capacity. Treat the scenarios and values as hypothetical.

A team has two equal-effort slots. Four cases have predicted probabilities 0.44, 0.41, 0.39, and 0.20 for the same binary outcome and horizon. It seeks to maximize the expected number of positive cases. Which probabilities are selected, and what is the expected count? (Answer: Select 0.44 and 0.41; expected count $0.44+0.41=0.85$.)
A three-hour team can review whole cases. A takes 1 hour with probability 0.30, B takes 2 hours with probability 0.72, C takes 1 hour with probability 0.40, and D takes 1 hour with probability 0.35. Which feasible set maximizes expected positive count? (Answer: Select B and C; they use $2+1=3$ hours and have expected count $0.72+0.40=1.12$.)
A team has two equal-effort slots and probabilities 0.50, 0.50, 0.49, and 0.12. The objective is expected positive count. Does the tie at the selection boundary change the expected count? (Answer: Select both 0.50 cases; the expected count is 1.00, and there is no tie at the boundary between selected and unselected cases.)
A four-hour team chooses among X, which takes 3 hours and has probability 0.70; Y, which takes 2 hours and has probability 0.60; and Z, which takes 2 hours and has probability 0.58. Which set maximizes expected positive count? (Answer: Select Y and Z; they use 4 hours and have expected count 1.18, exceeding X alone at 0.70.)
A hypothetical unit has 100 eligible cases, can review eight today, and seeks to prioritize cases by their probability of meeting one consistently defined outcome within one horizon. Must the unit assign predictions only to the eight selected cases? (Answer: No. To rank the full eligible queue, assign predictions to all 100 cases, then select the eight highest-ranked cases if review effort is equal.)

What You'll Learn