Open Problems

Model Safety, Alignment & Jailbreaks

Evaluating the Generalisation of Small-Model Safety and Jailbreak Techniques to 70B+ and Frontier Models

Scope to testOpen
Possible candidate · 3/5 runs6 papers report this100% from 2025+

Generated automatically from the limitations stated in 6 papers (ICML, ACL, NeurIPS), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Current safety, alignment, and jailbreak methodologies are overwhelmingly developed and empirically evaluated on small models under 14B parameters. Safety defenses and attack surfaces developed on smaller models are assumed to transfer to production scale, but whether their efficacy degrades, persists, or exhibits non-linear scaling transitions at 70B+ parameters remains unverified. As a result, the literature cannot determine whether safety interventions tested on 7B–13B models provide meaningful security guarantees for deployed frontier systems.

Why it matters

Provides empirical verification of whether alignment and jailbreak findings on lightweight models reliably predict behavior at frontier scale, establishing empirical bounds on small-model safety research as a proxy for frontier safety.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Re-evaluate a standardized suite of published jailbreak attacks and safety defenses (originally validated only on $\le$14B models) across open-weights scaling ladders (e.g., 8B, 32B, 70B, and 405B from Llama and Qwen families), measuring attack success rate (ASR) and defense retention across scale.

  2. 2

    Adapt small-model representation-level or optimization-based safety methods (e.g., representation steering, prompt-based safety guards) to black-box and open-weights frontier-scale models (70B+ and frontier APIs), measuring computational feasibility and defense effectiveness.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

Rapid architectural shifts (such as specialized reasoning models and inference-time safety monitors) could render existing prompt-based attack and defense formulations obsolete before scaling trends across standard base models can be systematically documented.

Evidence

Each paper's own statement of the limitation, verbatim.

Nearest existing work

Related open problems

Model Safety, Alignment & Jailbreaks

Barrier to removeOpen

A Framework for Deploying Activation- and Logit-Based Methods Against Closed-Source LLM APIs via Instrumented Proxy Models

A very large body of techniques — safety defenses, interpretability probes, evaluation metrics, decoding controls — requires hidden states, attention maps, gradients, or token-level probabilities, and is therefore silently restricted to open-weight models. As a result, published methods are never validated on, and cannot protect or audit, the models most people actually use (GPT-4, Claude, Gemini). This is the single most common self-reported scope limitation in the literature, yet each paper treats it as an isolated footnote rather than a solvable engineering and inference problem.

Strong candidate · 5/5 runs177 papers report this78% from 2025+

Model Safety, Alignment & Jailbreaks

Barrier to removePartly addressed

Decision-Only and Logit-Free Safety Auditing and Alignment Evaluation for Black-Box Models

Current safety auditing, jailbreak discovery, and representation-based alignment diagnostics overwhelmingly rely on white-box preconditions such as gradient access, intermediate activations, or output token logit distributions. When models are deployed behind production APIs that return only generated text, practitioners cannot run these diagnostic and red-teaming pipelines directly. Relying on surrogate transfer from open-source models produces high false-negative rates due to incomplete transferability across distinct architectures and tokenizers. Consequently, safety assessments for proprietary, API-governed models remain structurally disconnected from the methods developed in the literature.

Strong candidate · 4/5 runs17 papers report this59% from 2025+

Model Safety, Alignment & Jailbreaks

Barrier to removePartly addressed

Recovering Internal Attention and Activation Signals from API-Visible Model Behavior

The shared wall is a dependence on inference-time extraction of layer-wise internal states — attention weights, hidden activations, per-layer execution — which are unavailable outside open-weight deployments and costly and per-model-fragile even where weights are open. Every technique built on those signals stops working the moment the model is served through an API; the sanctioned fallbacks, transferring from an open proxy or degrading to logit-only features, demonstrably forfeit most of the signal (one published detector drops from its white-box ceiling to 0.66 AUROC). Even with open weights, each release re-breaks the tooling: layer indices must be re-selected per model and multi-pass decoding adds latency. The result is a method family whose reach shrinks exactly as the most capable models become less open, and whose findings nobody without weight access can verify or use.

Strong candidate · 5/5 runs15 papers report this100% from 2025+

Model Safety, Alignment & Jailbreaks

Effect to explainPartly addressed

Robust Evaluation of Non-Binary and Phrasing-Agnostic Refusal in Safety-Aligned Language Models

Current safety benchmarks and verifiable reward mechanisms evaluate model refusals primarily through binary keyword matching, hand-crafted refusal templates, or uncalibrated model-based autoraters (which show inter-annotator agreement as low as $\alpha = 0.378$). This creates a blind spot where models that partially comply, use non-standard refusal vocabulary, or alter their stance under multi-turn pushback are misclassified as either fully compliant or safely refusing. Consequently, alignment interventions optimized against these brittle metrics suffer from artificial over-refusal on benign queries while leaking safety risks through stylistic variation and conversational pressure. Without a rigorous, non-binary evaluation standard across diverse phrasing distributions, researchers cannot reliably assess whether safety alignment generalizes beyond rigid heuristic templates.

Strong candidate · 4/5 runs14 papers report this71% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.