When Adversarial Risk Equals Distributionally Robust Risk

Why the Constraint on Transport Matters

Pointwise adversarial risk asks how much the loss could increase if each observation were changed within a prescribed perturbation set. Wasserstein distributional robustness instead asks for the largest expected loss over distributions reachable by transporting probability mass from the observed distribution. These formulations are related, but the word “reachable” depends on the transport constraint: a bound on every transported observation is different from a bound on the average transport cost.

The exact equivalence is most direct when the ambiguity set consists of distributions that can be coupled to the source so that every transported point stays within its local perturbation set. Under suitable measurability and compactness conditions, the worst-case distribution can make the same pointwise choices as the adversary. A Wasserstein-one ball generally imposes only an average-cost constraint, so it need not give the same risk.

Two Risk Definitions

Let \(Z\) be a compact metric space with metric \(d\), let \(P\) be a Borel probability distribution on \(Z\), and let \(\ell:Z\to\mathbb{R}\) be a bounded continuous loss. For a radius \(\varepsilon\geq 0\), the pointwise perturbation set at \(z\) is the closed ball \(A_\varepsilon(z)=\{z'\in Z:d(z,z')\leq\varepsilon\}\). The adversarial risk is the expectation of the largest loss available in each such ball.

A coupling \(\pi\) of \(P\) and a candidate distribution \(Q\) is a joint distribution on \(Z\times Z\) whose first marginal is \(P\) and whose second marginal is \(Q\). The coupling records how source points \(z\) are transported to destination points \(z'\). Define the per-observation transport ambiguity set by requiring that at least one such coupling puts all its mass on pairs no farther apart than \(\varepsilon\).

The first marginal requirement is essential: transported mass originates from \(P\), rather than being chosen independently at each destination. But there is no shared capacity restriction on destinations. Each source point may be coupled to any feasible destination, including through a randomized conditional distribution, as long as the per-observation distance bound holds.

Equality Under a Per-Observation Constraint

The closed-ball correspondence \(z\mapsto A_\varepsilon(z)\) has nonempty compact values and a closed graph. The measurable maximum theorem therefore ensures that \(g(z)=\max_{z'\in A_\varepsilon(z)}\ell(z')\) is measurable and that a measurable maximizer can be selected. In particular, there is a measurable map \(T\) such that \(T(z)\in A_\varepsilon(z)\) and \(\ell(T(z))=g(z)\). These are the regularity conditions that allow pointwise maximization to define a valid distribution.

The upper bound in Step 1 holds for every feasible \(Q\); Step 2 attains that bound. Together they prove the equality. The proof also shows the mechanism: a measurable pointwise choice induces a single joint coupling, whose second marginal is the adversarial distribution. \(\square\)

Why a Wasserstein-One Ball Is Different

For cost \(d\), the Wasserstein-one distance \(W_1(P,Q)\) is the infimum, over couplings \(\pi\) of \(P\) and \(Q\), of the average distance \(\int d(z,z')\,\pi(dz,dz')\). A constraint \(W_1(P,Q)\leq\rho\) limits the total average transport cost. It does not require each source point to move by at most \(\rho\), or even by at most a common pointwise radius. Some mass can move farther if the rest moves less or stays still.

Label Preservation and Support Restrictions

In prediction problems, the transported object is often a pair \(z=(x,y)\), where \(x\) is the input and \(y\) is the label. If the intended adversary changes only the input, its feasible set is \(A_\varepsilon((x,y))=\{(x',y):d_X(x,x')\leq\varepsilon\}\). The coupling formulation must enforce the same condition. A cost that measures only input displacement but permits the label to change does not encode this adversary: a zero-cost change of label can change the loss.

A separate issue is whether destination points must remain in a restricted support. If pointwise sets are defined in an ambient space but the distributional ambiguity set additionally requires \(Q\) to be supported on a smaller set, the two feasible sets may differ. For example, take ambient space \([0,1]\), \(P=\delta_0\), loss \(\ell(x)=x\), and radius \(1\). The ambient pointwise adversarial risk is \(1\), since the point \(1\) is reachable from \(0\). If the distributional ambiguity set additionally requires \(Q(\{0\})=1\), its only feasible distribution is \(\delta_0\), and its risk is \(0\). Equality is restored only if the pointwise feasible destinations and the allowed destination support agree.

Common Mistake and Practical Check

A common mistake is to argue, “Both formulations use the same radius, so their worst-case risks are equal.” The radius alone does not identify the constraint. A Wasserstein-one radius bounds average transport cost; a pointwise perturbation radius bounds each observation’s displacement. A second mistake is to use an input-only cost on labelled observations while allowing couplings to alter labels. In both cases the proposed ambiguity set admits moves that the intended pointwise adversary does not.

Before claiming equivalence, check that the coupling has the source distribution as its first marginal, that its allowed moves match the pointwise sets, and that its constraint is imposed per observation rather than only on average. Also check whether labels must be preserved and whether the distributional ambiguity set has extra support restrictions. Under the compactness and continuity conditions stated above, agreement on these details gives the proved equality; changing them requires a new argument.