The Holographic Gradient: On the Privacy Limits of Split Federated Learning

Split Federated Learning (SFL) is built on a reassuring premise: keep the labels on the client, and the server learns nothing sensitive. We show this premise is wrong. The gradient the server receives to train its model is not an opaque update; it encodes the private labels in a way that is mathematically unavoidable. We formalise this vulnerability, prove information-theoretic lower bounds on the leakage, and show why existing defences fail: they cannot distinguish the gradient's optimization signal from its label leakage. We prove these two components occupy different subspaces near convergence, a consequence of Neural Collapse, and build on this to propose Rate-Limited SFL (RL-SFL), which suppresses the leakage subspace by transmitting only the most informative gradient coordinates. RL-SFL comes with convergence guarantees and works well under near-homogeneous data distributions, but degrades under high client heterogeneity and provides no privacy at all under extreme skew; it also carries no formal differential privacy certificate. On CIFAR-10/100 and BloodMNIST, RL-SFL achieves substantially lower attack accuracy than standard DP-SGD and variational baselines at comparable utility, with reduced communication cost on bandwidth-constrained links.