Judge Leniency as an Instrument

Why Judge Assignment Might Provide Leverage

People with similar cases may receive different decisions because their cases are assigned to judges with different tendencies. If assignment to a more or less lenient judge is unrelated to the case’s potential outcomes and treatment responses, that assignment can provide variation in treatment receipt that is not driven by the case’s underlying circumstances. The central challenge is that a judge’s tendency is not automatically an instrument: the assignment process and the judge’s other effects on the case both matter.

Consider a hypothetical study of whether pretrial release affects a defendant’s probability of employment six months later. Let \(Z=1\) indicate assignment to a judge classified as more lenient and \(Z=0\) assignment to a less lenient judge. Let \(D=1\) mean the defendant is released before trial and \(D=0\) mean detention. Let \(Y\) be an employment outcome, such as an indicator equal to one if the defendant is employed at six months. The proposed instrument is judge assignment, not release itself.

For this coding, greater leniency is hypothesized to increase release. The potential treatment values are \(D(1)\) and \(D(0)\), the release status the same defendant would have under lenient and less-lenient assignment. Potential outcomes are \(Y(1)\) and \(Y(0)\), the outcome under release and detention. The usual LATE interpretation also uses consistency and no interference: the observed treatment is \(D=D(Z)\), the observed outcome is \(Y=Y(D)\), and one case’s assignment does not change another case’s outcome. These assumptions must be plausible in the particular setting, not merely stated in notation.

What the Instrument Must Establish

Relevance is an observable first-stage claim: release rates differ by assigned judge group. With \(Z=1\) coded as the more lenient group, the expected direction is \(P(D=1\mid Z=1)>P(D=1\mid Z=0)\). A small difference may mean the instrument carries little information about treatment, even if the difference has the expected sign. Relevance does not show that assignment is independent, nor that judges affect outcomes only through release.

Independence is a claim about the assignment process. In an observational setting, a plausible version is that, within a defined court, location, and time stratum, assignment is as if random conditional on pre-assignment covariates \(X\). For example, a rotation or genuinely random case-allocation system might support that argument. It is not enough that judges have different release rates. If court staff direct more serious cases to stricter judges, the assignment may be associated with case characteristics that also predict outcomes.

Exclusion requires judge assignment to influence \(Y\) only by changing whether the defendant is released. In this example, a judge may also set release conditions, affect case timing, influence plea negotiations, or impose a sentence. If any such pathway affects six-month employment apart from the binary release treatment as defined, exclusion is in doubt. Defining treatment broadly enough to include meaningful versions of release can clarify the question, but does not guarantee that every judge effect is captured.

Monotonicity for this coding is \(D(1)\geq D(0)\) for every defendant: no defendant who would be released by a less-lenient judge would instead be detained by a more-lenient judge. This leaves four compliance types. Compliers have \(D(1)=1,D(0)=0\); always-released defendants have \(D(1)=D(0)=1\); never-released defendants have \(D(1)=D(0)=0\); and defiers have \(D(1)=0,D(0)=1\). The LATE is the average causal effect of release for compliers, not for all defendants.

Why the Wald Ratio Targets Compliers

In a binary-instrument setting, suppose assignment is independent of potential outcomes and treatment responses, exclusion holds, consistency and no interference hold, and monotonicity holds. The difference in average treatment receipt across assignment groups is the proportion of compliers, while the outcome difference is that proportion multiplied by the average treatment effect among compliers. Their ratio therefore identifies the complier average causal effect.

Here \(C\) denotes the complier group. The first equality in the outcome contrast uses independence to compare the potential values under the two assignment states; exclusion lets us write the outcome as \(Y(D(z))\). Under monotonicity, always-released and never-released defendants have \(D(1)-D(0)=0\), while every complier has difference one. Thus only compliers contribute to either contrast. If \(Y\) is a binary employment indicator, the effect is measured as a difference in employment probabilities, not as an effect for every defendant.

Judge-Specific Risks and Failure Modes

The substantive case for each assumption must fit the court’s actual procedures. A judge label such as “lenient” may summarize average behavior, but an average tendency does not guarantee that every defendant would be at least as likely to receive the treatment from that judge. Judges may respond differently to different kinds of cases: one may be more lenient for low-risk defendants and less lenient for another group. Such individual reversals threaten monotonicity. If judges vary across several decisions at once, a single ranking may not represent a one-directional change in treatment.

Instrument construction also matters. A judge’s observed release rate can be noisy, especially when based on few cases. If the assigned defendant is included in the rate used to classify that defendant’s judge, the measure partly incorporates the case’s own treatment decision. Researchers often consider judge tendency measured from other cases, but that choice does not itself solve case sorting or establish independence. The assignment unit, time period, and relevant comparison group need to be clearly specified.

The key threats and their implications can be summarized as follows.

Balance checks can reveal warning signs of nonrandom assignment: for instance, large differences in measured case severity across judge groups are difficult to reconcile with an unqualified as-if-random claim. But similar measured covariates do not prove independence, because unmeasured characteristics may still determine assignment. Likewise, a strong first stage cannot test exclusion. A diagnostic can expose a problem; passing a diagnostic does not establish an assumption that concerns unobserved potential outcomes.

When This Design Is Useful—and What It Cannot Promise

Judge assignment can be attractive when defendants are allocated to judges by a credible rotation, calendar, or other process that is plausibly unrelated to their potential outcomes within an appropriate comparison group. Its advantage is that it may generate treatment variation among cases that would otherwise differ in reasons for receiving treatment. Its limitation is that the result is local to defendants whose treatment is changed by the particular judge contrast. It need not describe the effect for always-treated, never-treated, or all defendants.

A randomized assignment system, when feasible, can strengthen the independence argument, but random assignment alone does not establish exclusion or monotonicity. Covariate adjustment or propensity score methods instead address measured differences between treatment groups under their own assumptions; they do not automatically remove unmeasured confounding. Conversely, using judge assignment as an instrument relies on a defensible assignment process and the restriction that judges do not affect outcomes through other channels. These approaches answer different identification problems, and none is validated merely by producing a numerical estimate.

A careful judge-leniency argument therefore specifies the case population and comparison strata, defines the treatment and outcome precisely, documents how cases reach judges, shows that assignment predicts treatment, and explains why other judge actions do not affect the outcome. It also makes the direction of the treatment coding explicit and considers whether that direction is plausible for every relevant case. If these substantive conditions are not credible, a calculated Wald ratio should not be presented as a causal LATE.