Auditing the Gap Between Theoretical Equivariance Guarantees and Empirical Performance in Finite, Symmetry-Broken Architectures

The problem

Current theoretical guarantees for equivariant neural networks rely on assumptions that practical pipelines routinely violate: infinite network width, exact group representations, strict absence of symmetry breaking, and fully equivariant backbones. Practitioners regularly deploy approximate, learned, or symmetry-broken equivariant models alongside non-equivariant backbones (e.g., ResNet, Swin), where theoretical equivariance guarantees formally collapse. Because no unified evaluation has tested these isolated scope limitations across common benchmarks, it remains unknown whether reported empirical gains stem from geometric inductive biases or trivial capacity increases, and why learned equivariance fails to extrapolate under distribution shift.

Why it matters

Provides practitioners with clear empirical boundaries showing when strict equivariance constraints actually provide generalization advantages over data augmentation or symmetry-broken variants. Enables researchers to ground future symmetry theory in the finite-width, non-exact regimes that are actually used in practice.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

1. 1

   Cross-architecture equivariance stress-testing: Evaluate exact, learned, and symmetry-broken models across standardized vision and graph benchmarks with controlled parameter counts, measuring empirical equivariance error $\epsilon$ and task performance under in-distribution and out-of-distribution group transformations.

   Check for prior work
2. 2

   Finite-width versus infinite-width augmentation audit: Compare data-augmented networks directly against exact equivariant and invariant layers across a sweep of network widths (from narrow MLPs to wide ResNets), measuring feature map rank, empirical NTK alignment, and accuracy on CIFAR-10 and rotated datasets.

   Check for prior work
3. 3

   Backbone integration ablation: Benchmark hybrid architectures (standard backbones paired with equivariant heads or sampling layers) against parameter-matched non-equivariant baselines, measuring whether the equivariance gap on test and OOD (e.g., medical imaging) data correlates with downstream generalization.

   Check for prior work

Why it might fail

The project would be undermined if community interest has already shifted away from exact equivariance toward unconstrained foundation models with heavy data augmentation, rendering the exact-versus-approximate distinction practically obsolete. It could also fail if the divergence across settings is entirely explained by standard optimization difficulties rather than architectural inductive biases.

Evidence

Each paper's own statement of the limitation, verbatim.

• ReDiffDet: Rotation-equivariant Diffusion Model for Oriented Object DetectionCVPR 2025

  With ResNet50 or Swin-T backbones (non-equivariant features), the theoretical equivariance guarantee does not hold, and gains are attributed to larger parameter counts rather than the framework

• Equivariant Neural Tangent KernelsICML 2025

  The equivalence between augmented and equivariant training is exact only in the infinite-width limit with quadratic loss, up to quadratic corrections in learning rate; finite-width agreement is only approximate and demonstrated on small datasets (MNIST, CIFAR10)

  Theorems 5.2 and 5.3 hold only for invariant networks (group pooling); extension to the equivariant (non-invariant) case requires new layers because the infinite-width MLP NTK becomes proportional to the identity in output channels, trivializing the feature map

• Adaptive Sampling for Continuous Group Equivariant Neural NetworksICML 2024

  Exact equivariance requires the sampling matrix to be generated by equivariant layers and relies on approximate orthogonality of A for the pseudoinverse approximation; learned matrices only empirically approach orthogonality (ϵ2 converges to ~0.7 due to normalization)

• The Lie Derivative for Measuring Learned EquivarianceICLR 2023

  Learned equivariance does not extrapolate: the equivariance gap grows on test and OOD (medical) data regardless of architecture, so learned equivariance is only local and approximate.

• Approximately Equivariant Graph NetworksNeurIPS 2023

  The theory analyzes strictly equivariant models without symmetry breaking, while the best empirical results use symmetry-broken variants (graph convolution, learnable edge weights) that are not covered theoretically.

Nearest existing work

• Approximation-Generalization Trade-offs under (Approximate) Group EquivarianceNeurIPS 2023
• Probing Equivariance and Symmetry Breaking in Convolutional NetworksNeurIPS 2025
• To Augment or Not to Augment? Diagnosing Distributional Symmetry BreakingICLR 2026
• On Fairly Comparing Group Equivariant NetworksICML 2024
• Approximately Equivariant Networks for Imperfectly Symmetric DynamicsICML 2022
• The Lie Derivative for Measuring Learned EquivarianceICLR 2023
• Investigating how ReLU-networks encode symmetriesNeurIPS 2023
• Any-Subgroup Equivariant Networks via Symmetry BreakingICLR 2026
• Learning Invariances in Neural Networks from Training DataNeurIPS 2020
• Emergent Equivariance in Deep EnsemblesICML 2024
• Learning Layer-wise Equivariances Automatically using GradientsNeurIPS 2023
• Equivariant Adaptation of Large Pretrained ModelsNeurIPS 2023
• Equivariance versus Augmentation for Spherical ImagesICML 2022
• A General Theory of Correct, Incorrect, and Extrinsic EquivarianceNeurIPS 2023
• LieGG: Studying Learned Lie Group GeneratorsNeurIPS 2022

Related open problems