Open Problems

Model Safety, Alignment & Jailbreaks

A Framework for Deploying Activation- and Logit-Based Methods Against Closed-Source LLM APIs via Instrumented Proxy Models

Barrier to removeOpen
Strong candidate · 5/5 runs177 papers report this78% from 2025+

Generated automatically from the limitations stated in 177 papers (ACL, EMNLP, ICLR), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

A very large body of techniques — safety defenses, interpretability probes, evaluation metrics, decoding controls — requires hidden states, attention maps, gradients, or token-level probabilities, and is therefore silently restricted to open-weight models. As a result, published methods are never validated on, and cannot protect or audit, the models most people actually use (GPT-4, Claude, Gemini). This is the single most common self-reported scope limitation in the literature, yet each paper treats it as an isolated footnote rather than a solvable engineering and inference problem.

Why it matters

It converts hundreds of open-weights-only methods into candidate deployable tools for closed APIs, and gives providers an evidence-based menu of which minimal exposures would enable which safety/evaluation capabilities.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Systematic transfer study: take 10–20 representative white-box methods (refusal probing, hallucination detection, jailbreak scoring, evaluation metrics), fit them on open proxies (Llama, Qwen, Mistral families), and deploy them against closed APIs using only black-box signals (text, top-k logprobs where offered, embeddings endpoints). Measure transfer of detection/calibration quality (AUROC, F1) relative to native white-box performance, and identify which internal quantities have cheap black-box surrogates.

  2. 2

    Black-box reconstruction: train lightweight surrogate probes on API outputs (logprobs, responses to contrastive prompt sets) that predict the latent quantities the method needs (refusal probability, uncertainty, hidden-state-derived scores), and measure how much of the white-box method's discriminative power is recovered per API budget.

  3. 3

    Provider-side instrumentation protocol: specify the minimal state exposure (e.g., top-k logprobs, one chosen hidden layer) a provider would need to release, and empirically measure, method by method, how performance degrades across exposure tiers — turning "white-box required" into a quantified requirement rather than a binary.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

Internal states may simply not transfer across model families — refusal/uncertainty geometry differs enough between the open proxy and the closed target that surrogate probes recover little signal, leaving only the (unactionable) recommendation that providers expose more.

Sub-problems

  • Black-Box Safety Auditing and Alignment Verification for API-Only Foundation Models

    Current safety auditing methods, alignment interventions, and vulnerability evaluations rely directly on internal model representations, attention maps, or backward gradients. Because commercial frontier models are accessible only via black-box inference APIs, third-party auditors and downstream developers cannot inspect internal activations, calculate attributions, or fine-tune weights to enforce safety constraints. This leaves the most widely deployed foundation models structurally unverifiable by standard white-box safety pipelines.

  • Black-Box Behavioral Attribution and Interpretability for Closed-Source Language Models

    Current interpretability and in-context learning diagnostic tools overwhelmingly require white-box access to attention matrices, intermediate activations, or raw token logits. Because commercial frontier models are accessible solely through text-generation APIs, these diagnostic and auditing methods cannot be deployed on the systems that handle most real-world user traffic. This leaves safety auditors and practitioners unable to verify reasoning mechanisms, explain unexpected failures, or trace factual hallucinations in closed-source models.

  • Behavioral and Query-Based Surrogates for White-Box LLM Activation Probing

    State-of-the-art evaluation techniques—including truthfulness probes, sparse autoencoder feature analysis, and internal anomaly detectors—depend on access to internal hidden states, layer activations, or full-vocabulary log probabilities. Because leading commercial models are accessible exclusively through restricted text-generation APIs, independent researchers and regulators cannot run these diagnostic evaluations on the systems deployed at scale. Consequently, external auditing of proprietary LLMs is limited to surface-level output metrics, unable to leverage the finer-grained diagnostic power of representation-level evaluations.

  • Black-Box Vision-Language Model Attribution and Verification Without Internal State Access

    Leading multimodal foundation models are predominantly deployed behind commercial APIs that expose only prompt inputs and output text or token probabilities. Existing techniques for vision-language tasks—such as visual grounding, hallucination detection, feature attribution, and model steering—overwhelmingly require white-box access to internal attention maps, layer activations, or gradients. Consequently, practitioners cannot apply standard interpretability, auditing, or safety-verification tools to the most capable proprietary vision-language models. This creates an operational barrier between open-weight academic methods and closed-source production deployments.

  • Black-Box Reasoning Steering and Diagnosis Without Hidden-State Access

    Current interpretability, steering, and verification techniques for LLM reasoning rely strictly on white-box access to intermediate activations, residual streams, or unembedding matrices. Because leading commercial models (such as GPT-4 and Claude) are served exclusively via text-only or restricted-logit APIs, these internal-state methods cannot be deployed or audited on frontier reasoning engines. Structurally, developers and safety evaluators cannot monitor, steer, or verify latent reasoning trajectories without internal tensor access.

  • Black-Box Compression and Steering of Chain-of-Thought Reasoning

    Current chain-of-thought optimization, compression, and steering techniques require internal access to hidden activations, attention weights, or gradient updates. Consequently, these methods cannot be deployed on proprietary, API-hosted frontier models where a substantial fraction of production reasoning workloads take place. Users of closed-source models are forced to pay full inference latency and token costs without access to the efficiency or calibration mechanisms developed for open-weight models. This creates a hard practical divide between white-box reasoning research and black-box deployment.

  • Black-Box Behavioral Steering and State Tracking for API-Based Language Agents

    Current methods for auditing, steering, and editing language agents depend on white-box access to internal states, including activation vectors, attention maps, gradients, and model weights. Because the most capable frontier models are accessible only via closed-source inference APIs, these white-box techniques cannot be applied to deployed commercial agents. Practitioners are consequently unable to monitor internal reasoning failures, edit agent capabilities, or apply activation-level safety steering to proprietary models. Developing black-box equivalents is necessary to inspect and control agents operating behind API boundaries.

  • Black-Box Interpretability and Mechanism Auditing for API-Restricted Large Language Models

    The vast majority of mechanistic interpretability and in-context learning diagnostics rely on inspecting internal activations, attention maps, and layer-wise representations. As a result, these diagnostic and hallucination-detection tools are entirely unusable on frontier commercial models accessible only via query APIs (e.g., GPT-4o, Gemini). Practitioners deploying or evaluating closed-source systems are blocked from verifying whether a model's in-context reasoning relies on genuine task induction or superficial heuristic retrieval. Developing black-box proxies that infer internal mechanism dynamics from behavioral traces and output distribution perturbations bridges this critical evaluation divide.

  • Black-Box Interpretability and Diagnostic Auditing for Closed-API Language Models

    Current interpretability and mechanistic diagnostic methods strictly depend on internal representations—gradients, hidden states, attention maps, and predictive logits—that are unavailable for closed-source, API-served models. Consequently, third-party researchers and auditors cannot inspect, explain, or verify the reasoning and factual reliability of the most capable deployed frontier models. This leaves external safety and compliance evaluations structurally blocked unless model providers voluntarily expose internal access.

  • Cross-Model Generalization and Failure Boundaries of Chain-of-Thought Reasoning Frameworks

    Current prompt-based and search-augmented reasoning methods are overwhelmingly validated on a single proprietary model (often GPT-3.5 or GPT-4), yet independent reports show their benefits frequently vanish or reverse when applied to different model families or capability scales. Without systematic cross-model evaluation, researchers cannot distinguish whether a proposed reasoning technique provides general algorithmic utility or merely exploits idiosyncratic behaviors of specific backbones. This blocks the reliable deployment of reasoning frameworks across open-source and alternative foundation models.

  • Black-Box Hallucination Detection and Auditing for Vision-Language Models Without Proprietary Judges

    Current techniques for assessing and mitigating hallucinations in large vision-language models (LVLMs) depend heavily on either white-box access to internal activations or validation by proprietary API judges such as GPT-4V. This creates a structural barrier: white-box methods cannot audit commercial, closed-source models where weights and activations are inaccessible. Simultaneously, relying on proprietary APIs for evaluation introduces severe reproducibility issues caused by model deprecation, opaque backend updates, and high query costs. As a result, there is currently no standardized, reproducible way to verify multimodal generation quality on arbitrary black-box endpoints.

  • Black-Box Model Safety Auditing and Defense Without Internal Hidden State Access

    Current activation-based safety mechanisms—including representation engineering, hidden-state linear probes, and internal steering vectors—depend strictly on white-box access to intermediate activations. Because major commercial models operate strictly behind text- or top-logprob APIs, these safety defenses and latent auditing tools cannot be deployed by downstream developers or third-party auditors. Consequently, closed-weight models remain vulnerable to safety blind spots that internal representation monitors are designed to detect, leaving a major discrepancy between academic safety tooling and commercial deployment.

  • Black-Box Reasoning Evaluation and Confidence Estimation Without Token Log-Probabilities

    Current evaluation, verification, and uncertainty estimation frameworks for chain-of-thought reasoning depend on access to token-level log-probabilities or output distributions. Because major proprietary foundation models (e.g., GPT-4, Claude, Gemini) only expose generated text via black-box APIs, these evaluation and selection methods cannot be applied to the most capable deployed systems. This creates an empirical blind spot where open-source and proprietary models cannot be benchmarked or guided using the same evaluation machinery.

  • Black-Box Context Pruning and Efficiency Without Model Internals

    Modern inference efficiency and long-context techniques rely heavily on white-box signals such as internal hidden states, embedding modifications, and full-vocabulary entropy or perplexity scores. Consequently, these techniques cannot be deployed on commercial proprietary LLMs (e.g., GPT-4, Claude) that only expose string-in, string-out interfaces. Developers building on commercial APIs are forced to transmit full, uncompressed context windows at maximum token cost and latency.

  • Strict Black-Box Safety and Alignment Auditing for Text-Only APIs

    The majority of formal alignment diagnostics, safety verification methods, and red-teaming frameworks rely on access to internal activations, full-vocabulary logits, or top-k log probabilities. As commercial frontier LLM providers increasingly restrict API outputs to raw text to defend against model extraction and distillation, these safety assessment methods become unusable on deployed systems. Independent auditors, researchers, and regulators are structurally blocked from validating the safety boundaries and alignment guarantees of closed-source models using internal-state diagnostics.

  • Black-Box Cross-Lingual Transfer and Steering for API-Only Large Language Models

    Current multilingual transfer and cross-lingual steering techniques rely on inspecting or modifying internal hidden states, forward passes, and FFN activations. Because frontier models (such as GPT-4o and Gemini) are accessible strictly via text-in/text-out APIs, these internal intervention methods cannot be deployed on top-tier commercial LLMs. Practitioners working with low-resource languages in production environments are consequently blocked from using representation-level alignment techniques without hosting their own white-box weights.

  • Black-Box Safety and Alignment Auditing Without Internal Representation Access

    Current representation-based safety audits, latent guardrails, and jailbreak vulnerability assessments strictly require white-box access to internal hidden states, embedding lookup tables, or intermediate activations. As a direct consequence, safety evaluators cannot apply these internal auditing methods to commercial, closed-source API-only models where only text input-output interfaces are exposed. This creates a critical blind spot where safety guarantees evaluated on open-weight models cannot be verified on the frontier models actually deployed to end users.

  • Black-Box Jailbreak Detection and Refusal Control Without Logit or Representation Access

    Existing state-of-the-art defenses against jailbreaks, over-refusal mitigations, and refusal steering methods depend fundamentally on white-box signals such as internal layer activations, latent refusal directions, or next-token log-probabilities. Commercial and proprietary large language models are increasingly served via strictly black-box text-in/text-out APIs where token probabilities and internal embeddings are completely hidden. As a result, third-party auditors, downstream application developers, and API consumers cannot deploy modern representation- or logit-guided safety interventions on closed-source systems.

  • Black-Box Surrogates and Behavioral Probing for Internal State LM Evaluation

    A prominent class of language model evaluation and diagnostic benchmarks relies on inspecting intermediate-layer activations, residual stream dynamics, and unembedding projections. Because leading frontier models (e.g., GPT-4, Claude) are accessible exclusively via black-box text or logprob APIs, these diagnostic evaluation techniques cannot be executed on the very models most widely deployed in practice. As a result, comparative evaluations between open-weight and proprietary models remain bifurcated, unable to audit internal failure modes or calibration on closed systems.

  • Black-Box Retrieval-Augmented Generation Without Internal Model States

    Existing RAG methods for adaptive retrieval, attribution tracking, and generation steering rely on internal model representations such as attention weights, input embedding layers, intermediate gradients, or full vocabulary logit distributions. Because commercial LLM providers expose only text outputs (or at most top-$k$ output token probabilities), these advanced retrieval-augmented techniques cannot be deployed on state-of-the-art closed-source models. Practitioners using proprietary APIs are consequently restricted to naive text-in text-out prompting without confidence-guided retrieval or internal attribution. Developing black-box proxies for these mechanisms is necessary to extend advanced RAG capabilities to closed APIs.

  • Systematic Evaluation and Scaffolding of Multi-Step Tool-Use Agents Across Open-Source Model Scales

    Current language agent and tool-use frameworks are predominantly evaluated on proprietary, closed-source LLM APIs, leaving their generalizability to open-source models across parameter scales systematically untested. When open-source models are used, multi-step execution frequently destabilizes due to brittle formatted outputs, API syntax errors, and compounding trajectory drift. Without empirical characterization across model sizes and open architectures, research in language agents remains economically gated by commercial APIs and structurally non-reproducible.

  • Black-Box Watermarking and Controlled Decoding for Closed-Source Language Model APIs

    The majority of production large language models are hosted behind closed-source APIs that return only raw text completions, offering no access to hidden states, logits, or decoding configurations. Existing watermarking, safety steering, and provenance-tracking techniques rely on modifying the token sampling distribution or monitoring internal representations during inference. Because of this architectural precondition, third-party developers and downstream auditors cannot apply current provenance or control mechanisms to proprietary model endpoints.

  • Preference Optimization and Alignment for Strictly Black-Box Language Model APIs

    Current alignment and preference optimization methods fundamentally depend on white-box or gray-box access, such as updating model weights, reading internal activations, modifying vocabulary tokens, or querying full output logit distributions. Consequently, practitioners and downstream developers interacting with closed-source, API-only models cannot apply standard preference optimization techniques to customize or align model behavior to domain-specific human preferences. This locks proprietary systems into fixed alignment profiles unless providers explicitly retrain them, preventing users from executing post-hoc preference steering beyond basic context prompting.

  • Black-Box Adaptation and Calibration for API-Constrained Question Answering

    High-performance question answering and reading comprehension techniques increasingly rely on parameter-level fine-tuning, internal activation steering, or hidden-state uncertainty estimation. These methods cannot be deployed on state-of-the-art commercial LLMs that are only accessible via text-in/text-out APIs. Consequently, practitioners using closed-source models cannot apply established internal-state interventions to improve faithfulness, calibrate answer confidence, or correct reasoning errors. Bridging this interface barrier is necessary to transfer white-box reliability enhancements to proprietary frontier models.

Evidence

Each paper's own statement of the limitation, verbatim.

Show all 177 papers

Nearest existing work

Related open problems

Model Safety, Alignment & Jailbreaks

Barrier to removePartly addressed

Decision-Only and Logit-Free Safety Auditing and Alignment Evaluation for Black-Box Models

Current safety auditing, jailbreak discovery, and representation-based alignment diagnostics overwhelmingly rely on white-box preconditions such as gradient access, intermediate activations, or output token logit distributions. When models are deployed behind production APIs that return only generated text, practitioners cannot run these diagnostic and red-teaming pipelines directly. Relying on surrogate transfer from open-source models produces high false-negative rates due to incomplete transferability across distinct architectures and tokenizers. Consequently, safety assessments for proprietary, API-governed models remain structurally disconnected from the methods developed in the literature.

Strong candidate · 4/5 runs17 papers report this59% from 2025+

Model Safety, Alignment & Jailbreaks

Barrier to removePartly addressed

Recovering Internal Attention and Activation Signals from API-Visible Model Behavior

The shared wall is a dependence on inference-time extraction of layer-wise internal states — attention weights, hidden activations, per-layer execution — which are unavailable outside open-weight deployments and costly and per-model-fragile even where weights are open. Every technique built on those signals stops working the moment the model is served through an API; the sanctioned fallbacks, transferring from an open proxy or degrading to logit-only features, demonstrably forfeit most of the signal (one published detector drops from its white-box ceiling to 0.66 AUROC). Even with open weights, each release re-breaks the tooling: layer indices must be re-selected per model and multi-pass decoding adds latency. The result is a method family whose reach shrinks exactly as the most capable models become less open, and whose findings nobody without weight access can verify or use.

Strong candidate · 5/5 runs15 papers report this100% from 2025+

Model Safety, Alignment & Jailbreaks

Effect to explainPartly addressed

Robust Evaluation of Non-Binary and Phrasing-Agnostic Refusal in Safety-Aligned Language Models

Current safety benchmarks and verifiable reward mechanisms evaluate model refusals primarily through binary keyword matching, hand-crafted refusal templates, or uncalibrated model-based autoraters (which show inter-annotator agreement as low as $\alpha = 0.378$). This creates a blind spot where models that partially comply, use non-standard refusal vocabulary, or alter their stance under multi-turn pushback are misclassified as either fully compliant or safely refusing. Consequently, alignment interventions optimized against these brittle metrics suffer from artificial over-refusal on benign queries while leaking safety risks through stylistic variation and conversational pressure. Without a rigorous, non-binary evaluation standard across diverse phrasing distributions, researchers cannot reliably assess whether safety alignment generalizes beyond rigid heuristic templates.

Strong candidate · 4/5 runs14 papers report this71% from 2025+

Model Safety, Alignment & Jailbreaks

Barrier to removeOpen

Safety Monitoring and Alignment Interventions Under Black-Box and API-Only Constraints

Current state-of-the-art alignment monitoring, jailbreak detection, and activation-steering interventions explicitly require full white-box access to hidden state activations, KV caches, attention logits, or gradient flows. In practice, third-party safety auditors, downstream system builders, and end users interact with models exclusively via black-box query APIs or restricted endpoints. Because existing methods are structurally coupled to internal state inspection and modification, they cannot be deployed or evaluated on proprietary commercial models or in privacy-preserving environments.

Strong candidate · 5/5 runs7 papers report this100% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.