Model Safety, Alignment & Jailbreaks
Safety Monitoring and Alignment Interventions Under Black-Box and API-Only Constraints
Generated automatically from the limitations stated in 7 papers (ICML, ICLR, CVPR), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.
The problem
Current state-of-the-art alignment monitoring, jailbreak detection, and activation-steering interventions explicitly require full white-box access to hidden state activations, KV caches, attention logits, or gradient flows. In practice, third-party safety auditors, downstream system builders, and end users interact with models exclusively via black-box query APIs or restricted endpoints. Because existing methods are structurally coupled to internal state inspection and modification, they cannot be deployed or evaluated on proprietary commercial models or in privacy-preserving environments.
Why it matters
Enables independent third-party safety auditing, jailbreak defense, and policy monitoring on closed-weight models and API-restricted deployments.
Ways to approach it
Prior-work checks are free with an account. Results someone already ran are shown to everyone.
- 1
Surrogate-model transferability audit: Train open-source proxy models to mirror black-box target models, measuring how much detection AUROC drops when activation-based jailbreak detectors are transferred across model boundaries via black-box input-output pairs.
- 2
Query-based behavioral probing: Implement input perturbation and sequential sampling protocols to estimate Jacobian sensitivity and safety boundaries strictly from output token distributions, measuring defense success rates on standard red-teaming benchmarks.
- 3
Access-hierarchy ablation study: Systematically evaluate five prominent white-box safety mechanisms under degraded access tiers (full white-box, logit-only, top-$k$ logprobs, and pure text I/O) to quantify the exact minimum access threshold required for effective defense.
Have a different approach?
Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.
Why it might fail
If theoretical information barriers make black-box behavioral queries fundamentally insufficient for detecting stealthy safety failures, or if model providers standardize secure white-box enclave auditing before black-box workarounds become practical.
Evidence
Each paper's own statement of the limitation, verbatim.
- SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained FromICLR 2026
Requires white-box access to logits or internal hidden state representations.
- SARSteer: Safeguarding Large Audio Language Models via Safe-Ablated Refusal SteeringICML 2026
Requires white-box access to intermediate layer hidden states and activation interventions in the language backbone during inference.
- Dynamic Token Reweighting for Robust Vision-Language ModelsCVPR 2026
Requires white-box access to the model's activation space and KV cache during inference
- CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News GenerationICML 2026
Requires full white-box access to model parameters, internal hidden states, and attention logits to compute Jacobian spectral properties.
- Adaptive Probe-based Steering for Robust LLM JailbreakingICML 2026
Requires full white-box inference access to read and modify hidden state activations across all layers and sequence positions.
- Attention! Your Vision Language Model Could Be Maliciously ManipulatedNeurIPS 2025
White-box only: requires full access to architecture, weights, and per-token gradients during generation
- A Simple Yet Effective Method for Non-Refusing Context Relevant Fine-grained Safety Steering in LLMsEMNLP 2025
Requires white-box access to model activations and hidden states
Nearest existing work
- Calibrating Inference Time Alignment with Sequence-level Risk AccumulationACL 2026
- JULI: Jailbreak Large Language Models by Self-IntrospectionICLR 2026
- Weak-to-Strong Jailbreaking on Large Language ModelsICML 2025
- Jailbreaking the Matrix: Nullspace Steering for Controlled Model SubversionICLR 2026
- SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual ConnectionEMNLP 2025
- Alignment-Enhanced Decoding: Defending Jailbreaks via Token-Level Adaptive Refining of Probability DistributionsEMNLP 2024
- Enhancing the Transferability of Jailbreak Attacks on Large Language Models via Exploiting Reparameterization InvarianceACL 2026
- Lifelong Safety Alignment for Language ModelsNeurIPS 2025
- Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time AlignmentCVPR 2025
- Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution StrategyCVPR 2025
- SafeSpec: Fast and Safe LLM via Dynamic Reflective SamplingICML 2026
- Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language ModelsACL 2025
- Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language ModelsICLR 2025
- Ghost in the Cloud: Your Geo-Distributed Large Language Models Training is Easily ManipulatedICLR 2026
- Jailbreaking as a Reward Misspecification ProblemICLR 2025
Related open problems
Model Safety, Alignment & Jailbreaks
A Framework for Deploying Activation- and Logit-Based Methods Against Closed-Source LLM APIs via Instrumented Proxy Models
A very large body of techniques — safety defenses, interpretability probes, evaluation metrics, decoding controls — requires hidden states, attention maps, gradients, or token-level probabilities, and is therefore silently restricted to open-weight models. As a result, published methods are never validated on, and cannot protect or audit, the models most people actually use (GPT-4, Claude, Gemini). This is the single most common self-reported scope limitation in the literature, yet each paper treats it as an isolated footnote rather than a solvable engineering and inference problem.
Model Safety, Alignment & Jailbreaks
Decision-Only and Logit-Free Safety Auditing and Alignment Evaluation for Black-Box Models
Current safety auditing, jailbreak discovery, and representation-based alignment diagnostics overwhelmingly rely on white-box preconditions such as gradient access, intermediate activations, or output token logit distributions. When models are deployed behind production APIs that return only generated text, practitioners cannot run these diagnostic and red-teaming pipelines directly. Relying on surrogate transfer from open-source models produces high false-negative rates due to incomplete transferability across distinct architectures and tokenizers. Consequently, safety assessments for proprietary, API-governed models remain structurally disconnected from the methods developed in the literature.
Model Safety, Alignment & Jailbreaks
Recovering Internal Attention and Activation Signals from API-Visible Model Behavior
The shared wall is a dependence on inference-time extraction of layer-wise internal states — attention weights, hidden activations, per-layer execution — which are unavailable outside open-weight deployments and costly and per-model-fragile even where weights are open. Every technique built on those signals stops working the moment the model is served through an API; the sanctioned fallbacks, transferring from an open proxy or degrading to logit-only features, demonstrably forfeit most of the signal (one published detector drops from its white-box ceiling to 0.66 AUROC). Even with open weights, each release re-breaks the tooling: layer indices must be re-selected per model and multi-pass decoding adds latency. The result is a method family whose reach shrinks exactly as the most capable models become less open, and whose findings nobody without weight access can verify or use.
Model Safety, Alignment & Jailbreaks
Robust Evaluation of Non-Binary and Phrasing-Agnostic Refusal in Safety-Aligned Language Models
Current safety benchmarks and verifiable reward mechanisms evaluate model refusals primarily through binary keyword matching, hand-crafted refusal templates, or uncalibrated model-based autoraters (which show inter-annotator agreement as low as $\alpha = 0.378$). This creates a blind spot where models that partially comply, use non-standard refusal vocabulary, or alter their stance under multi-turn pushback are misclassified as either fully compliant or safely refusing. Consequently, alignment interventions optimized against these brittle metrics suffer from artificial over-refusal on benign queries while leaking safety risks through stylistic variation and conversational pressure. Without a rigorous, non-binary evaluation standard across diverse phrasing distributions, researchers cannot reliably assess whether safety alignment generalizes beyond rigid heuristic templates.