The Illusion of Privacy in Vanilla Federated Averaging#
Federated Learning (FL) enables multiple clinical institutions to collaboratively train a shared diagnostic model without centralizing protected health information (PHI). However, transmitting raw parameter gradients leaks high-dimensional information about the underlying patient scan .
Eq. (2)
Differentially Private Stochastic Gradient Descent (DP-SGD)#
To bound reconstruction risk mathematically, each hospital node clips per-example gradients to a maximum norm before adding isotropic Gaussian noise scaled to :
src/privacy/dp_federated_step.py
python
import torch
def clip_and_noise_per_sample_gradients(
per_sample_grads: torch.Tensor,
max_grad_norm: float = 1.0,
noise_multiplier: float = 1.15,
) -> torch.Tensor:
# Compute per-patient L2 norm across flattened parameter dimension
grad_norms = per_sample_grads.flatten(1).norm(p=2, dim=1, keepdim=True)
clip_factor = torch.clamp(max_grad_norm / (grad_norms + 1e-6), max=1.0)
# Bound sensitivity of each individual patient record
clipped = per_sample_grads * clip_factor.view(-1, *([1] * (per_sample_grads.ndim - 1)))
aggregated = clipped.sum(dim=0)
noise = torch.randn_like(aggregated) * (noise_multiplier * max_grad_norm)
return (aggregated + noise) / per_sample_grads.size(0)