·Medical AI·3 min read

Adversarial Robustness in 3D Volumetric Medical Imaging: Defending Vision Transformers Against Imperceptible Perturbations

Volumetric 3D Swin-UNETR segmentation models achieve clinician-level Dice scores on multi-modal brain MRI, yet remain brittle under L-infinity perturbations below sensor thermal noise. We derive Lipschitz-constrained volumetric attention and evaluate certified smoothing bounds.

Threat Model in Hospital PACS & Volumetric Diagnostic Pipelines#

When a 3D Vision Transformer (such as Swin-UNETR) ingests a volumetric FLAIR or T1ce MRI scan x∈RC×D×H×Wx \in \mathbb{R}^{C \times D \times H \times W}, clinical deployment assumes the voxel tensor arrives unmodified from the MRI reconstruction workstation. In practice, legacy Picture Archiving and Communication Systems (PACS) often route DICOM series across flat internal hospital VLANs without cryptographic signing.

Figure 1: Clinical PACS to 3D ViT Inference Threat Surface
Rendering figure...
Adversaries positioned on internal PACS segments can inject bounded L-infinity perturbations into uncompressed PixelData tags prior to AI triage.

Min-Max Volumetric Adversarial Formulation#

Let fθ:RC×D×H×W→[0,1]K×D×H×Wf_\theta : \mathbb{R}^{C \times D \times H \times W} \to [0,1]^{K \times D \times H \times W} denote our 3D volumetric segmentation network parameterized by θ\theta, mapping a multi-channel scan to KK anatomical or pathological classes. An adversary seeks a perturbation tensor δ\delta constrained within an ℓ∞\ell_\infty ball of radius ϵ\epsilon that maximizes the composite Dice-Cross-Entropy loss:

min⁡θ  E(x,y)∼D[max⁡∥δ∥∞≤ϵ(LDice(fθ(x+δ),y)+λ LCE(fθ(x+δ),y))]\min_{\theta} \; \mathbb{E}_{(x, y) \sim \mathcal{D}} \left[ \max_{\|\delta\|_{\infty} \le \epsilon} \left( \mathcal{L}_{\text{Dice}}(f_{\theta}(x + \delta), y) + \lambda \, \mathcal{L}_{\text{CE}}(f_{\theta}(x + \delta), y) \right) \right]
Eq. (1)
Saddle-point adversarial training objective for 3D volumetric segmentation under bounded voxel perturbation radius epsilon.

Spectral Normalization of 3D Shifted-Window Attention

Standard multi-head self-attention in 3D windows exhibits unbounded Jacobian singular values when query-key dot products concentrate on high-frequency Rician noise. By constraining the spectral norm σ(WQ),σ(WK)≤1\sigma(W_Q), \sigma(W_K) \le 1 and replacing softmax scaling with ℓ2\ell_2-normalized cosine similarity, we enforce a strict upper bound on layerwise perturbation amplification.

src/models/robust_swin3d_attention.py
python
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn.utils.parametrizations import spectral_norm

classLipschitzWindowAttention3D(nn.Module):
    """3D Windowed Self-Attention with Spectral Norm & Cosine Logit Bounding."""
    def __init__(self, dim: int, num_heads: int, tau: float = 0.07):
        super().__init__()
        self.num_heads = num_heads
        self.head_dim = dim // num_heads
        self.tau = nn.Parameter(torch.tensor(tau))

        # Constrain weight matrices to 1-Lipschitz spectral bound
        self.qkv = spectral_norm(nn.Linear(dim, dim * 3, bias=False))
        self.proj = spectral_norm(nn.Linear(dim, dim))

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        B_, N, C = x.shape
        qkv = self.qkv(x).reshape(B_, N, 3, self.num_heads, self.head_dim)
        q, k, v = qkv.permute(2, 0, 3, 1, 4).unbind(0)

        # L2-normalize queries and keys along embedding dimension
        q = F.normalize(q, p=2, dim=-1)
        k = F.normalize(k, p=2, dim=-1)

        # Bounded temperature scaling prevents Jacobian explosion
        scale = torch.clamp(self.tau, min=0.02)
        attn = (q @ k.transpose(-2, -1)) / scale
        attn = attn.softmax(dim=-1)

        out = (attn @ v).transpose(1, 2).reshape(B_, N, C)
        return self.proj(out)
PyTorch implementation of 1-Lipschitz spectrally normalized 3D window attention for volumetric medical segmentation.

Empirical Evaluation on Multi-Modal Brain MRI (BraTS)#

Across 480 volumetric cases, our spectrally regularized 3D Swin-UNETR trained with 3-step Free-AT preserves a clean Enhancing Tumor (ET) Dice score of 0.8840.884 (within 0.0060.006 of baseline) while improving PGD-20 robust Dice at ϵ=2/255\epsilon = 2/255 from 0.2710.271 to 0.7960.796.

  • Clean Whole Tumor (WT) Dice: 0.928 (Baseline) vs. 0.923 (Lipschitz-Swin3D)
  • PGD-20 Robust WT Dice (eps=2/255): 0.314 (Baseline) vs. 0.831 (Lipschitz-Swin3D)
  • Inference Latency Overhead on NVIDIA H100: +1.8 ms per 128x128x128 volume patch