Open Problems

Molecular Modeling & Drug Design

Benchmarks and Training Protocols for Size-and-Structure Transfer in Molecular ML

Scope to testOpen
Possible candidate · 3/5 runs14 papers report this75% from 2025+

Generated automatically from the limitations stated in 14 papers (ICML, NeurIPS, ICLR), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Molecular machine-learning models are trained and evaluated within narrow slices of chemical space — small, drug-like, scaffold-homogeneous molecules averaging ~44 atoms — so any deployment beyond that slice degrades sharply or requires full retraining per system. This blocks the field's central promise: screening large protein-ligand complexes, extrapolating to novel chemistries, and reusing models across temperatures, states, or unseen systems. Without a measurement standard for out-of-distribution molecular generalization, papers cannot even quantify how far their models actually transfer.

Why it matters

Models that a lab can apply to a new molecular system, size regime, or thermodynamic condition without retraining, plus an honest yardstick for claiming transferability.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Build a standardized size/chemistry-shift benchmark suite: split existing datasets (GEOM, MD22, PubChem) by atom count, scaffold, and functional-group novelty, with held-out regimes far from training support. Measure zero-shot accuracy curves as a function of distance from training distribution, for at least five published architectures.

  2. 2

    Test whether unlabeled structures from the target regime help: pretrain on large/chemically diverse molecules without labels, then fine-tune on the small labeled sets, measuring how much the zero-shot error (e.g., ε_Δ on held-out sizes) drops relative to training from scratch.

  3. 3

    Probe architectural capacity limits directly: vary the number of cross-attention tokens and replace fully-connected message passing with local neighborhoods, measuring reconstruction and prediction error as a function of molecule size to isolate whether bottleneck (information) or scaling (compute) dominates degradation.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

Degradation may turn out to be irreducibly data-limited — no benchmark or protocol fixes it, and all architectures fail similarly, yielding a purely negative result. Or the benchmark splits may be dominated by confounds (e.g., conformer quality) rather than true distribution shift.

Sub-problems

  • Systematic Cross-Regime Generalization and Robustness Benchmarking in Molecular Machine Learning

    Current molecular machine learning models are predominantly validated on standard scaffold or assay splits, obscuring severe performance degradation when applied out-of-distribution. Across diverse tasks, existing models fail to generalize across underrepresented functional groups, varying thermodynamic variables (temperature and pressure), temporal bioassay splits, and unseen meta-stable state transitions, frequently reverting to performance no better than trivial frequent-hitter baselines. Because no single study has systematically evaluated architectures across these separate operating regimes, practitioners cannot distinguish whether a model learns transferable chemical representations or merely memorizes dataset-specific artifacts. This forces downstream discovery pipelines to retrain models per system without clear failure criteria.

Evidence

Each paper's own statement of the limitation, verbatim.

Show all 14 papers

Nearest existing work

Related open problems

Molecular Modeling & Drug Design

Effect to explainPartly addressed

Lossless Generalization Across Molecular Scaffold Diversity: Closing the Accuracy Gap on Unseen Compounds

Molecular generative and predictive models currently trade off reconstruction fidelity, chemical validity, and generalization: models that reconstruct accurately lose validity on novel chemistry, and models that stay valid degrade sharply on scaffolds, functional groups, and complex molecules absent or rare in training. This means any model's utility on genuinely new chemistry — the cases that matter most in drug discovery — is unpredictable and unmeasured per scaffold. Structurally, the field cannot yet certify that a model generalizes to uncorrelated compounds rather than interpolating within its training distribution.

Possible candidate · 1/2 runs6 papers report this100% from 2025+

Molecular Modeling & Drug Design

Scope to testPartly addressed

Benchmarking Cross-System and Cross-Ensemble Generalization in Molecular Generative and Transition Models

Current deep learning models for molecular transitions, ensemble generation, and structure-based drug design are routinely trained and evaluated on isolated systems, fixed thermodynamic conditions, or homogeneous configuration spaces. When deployed on unseen molecular systems, new temperature/pressure regimes, novel pairs of meta-stable states, or shifted configuration spaces (such as moving from isolated ligands to pocket environments), these models require costly de novo retraining. Because prior studies only evaluate models within their narrow training distribution, the extent to which existing architectures can transfer across these varying physical and structural conditions remains unknown.

Weak candidate · 1/3 runs3 papers report this33% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.