Open Problems

Language Model Interpretability & In-Context Learning

Benchmarking Whether LLM-Based Methods Survive a Change of Model: A Transfer Study Across Access Levels and Model Families

Barrier to removeOpen
Strong candidate · 5/5 runs7 papers report this50% from 2025+

Generated automatically from the limitations stated in 7 papers (EMNLP, ICML, SIGIR), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Methods built on one LLM — whether they depend on its logits, its instruction-following behavior, its pretraining data, or its annotation quality — are currently reported on one or two models and simply assumed to generalize. Because the assumptions are implicit, a follow-up researcher cannot tell whether a failure on a new model is a flaw of the method or an unexamined dependency on the original model's scale, data, or API surface. This blocks cumulative progress: every new model release forces methods to be silently re-validated or abandoned, and conclusions attributed to "LLMs" are really conclusions about a single backbone.

Why it matters

Researchers could claim results about a class of models rather than one model, and new model releases could be triaged automatically against existing methods instead of rediscovering each incompatibility by hand.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Assemble a testbed of 5–8 LLMs spanning open-weight and API-only models, and port a fixed set of representative methods (a surrogate-training pipeline, a data-generation pipeline, an annotation pipeline) across them. Measure performance retention on transfer versus the original model, and decompose the loss into attributable causes (missing logit access, weak instruction-following, data mismatch) using controlled ablations.

  2. 2

    Build a dependency audit protocol: a checklist plus automated probes that, given a method, determine which model properties it actually requires (internal access, zero-shot generalization, unlabeled in-domain data). Validate by showing the audit's predictions of transfer failure on held-out model/method pairs are accurate (e.g., >80% rank correlation with measured degradation).

  3. 3

    Derive and test adaptation rules: for each dependency type, a documented fix (distillation for logit access, fallback prompting strategies for weak instruction-followers). Measure how much of the transfer gap each rule closes.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

The transfers fail too idiosyncratically — each model×method pair breaks for a unique reason — so no audit or taxonomy of dependencies generalizes, and the deliverable collapses into a spreadsheet of one-off anecdotes.

Sub-problems

  • Robustness and Generalization Limits of LLM-Assisted Information Extraction Across Model Families and Low-Resource Regimes

    Current LLM-assisted and semi-supervised information extraction (IE) and named entity recognition (NER) methods report empirical gains under narrow evaluation conditions, primarily using top-tier closed models (e.g., GPT-4o, DeepSeek-V3) and domains with abundant unlabeled text. Practitioners cannot know whether reported improvements transfer to open-weight models, compute-constrained setups, or low-resource domains lacking unlabeled data. Without systematic cross-model and cross-resource evaluation, extraction pipelines designed under favorable assumptions fail unpredictably when deployed in real-world resource-constrained settings. Consequently, teams cannot reliably choose an IE architecture without conducting redundant, expensive validation experiments for every specific target model and domain.

Evidence

Each paper's own statement of the limitation, verbatim.

Nearest existing work

Related open problems

Language Model Interpretability & In-Context Learning

Scope to testOpen

Cross-Linguistic Robustness of Interpretability and Cognitive Alignment in Language Models

Mechanistic interpretability claims and brain-to-LM alignment results currently rest almost entirely on English-only stimuli, English-trained models, and English-speaking participant data. Because cross-linguistic evaluation has not been conducted across these methods, it remains unknown whether identified circuits, induction heads, and representational alignments reflect general linguistic processing or are artifacts of English syntax and data abundance. Consequently, researchers cannot rely on existing interpretability mechanisms when analyzing multilingual models or studying cognitive language processing across typologically diverse languages.

Strong candidate · 5/5 runs9 papers report this33% from 2025+

Language Model Interpretability & In-Context Learning

UnclassifiedOpen

A benchmark of naturally occurring hallucinations with model-generated ground truth, replacing synthetic entity-swap surrogates for detection evaluation

Today, hallucination detection methods are trained and evaluated against synthetic artifacts — entity-swapped summaries, outputs from a different model than the one under test — rather than hallucinations that actually arise during generation. This means reported detection numbers do not measure whether a method works on the failure mode that matters, and the modest predictive results (e.g., R² ≈ 0.27 on hallucination scores) may partly reflect this distribution mismatch. Detection validated only on 7B–13B open-weight models with synthetic data cannot be trusted as evidence about frontier-model behavior. Without a natural-hallucination benchmark, every detection paper inherits the same unquantified gap between "detects injected errors" and "detects real hallucinations."

Weak candidate · 1/3 runs6 papers report this33% from 2025+

Language Model Interpretability & In-Context Learning

Scope to testPartly addressed

Benchmarking Interpretability and In-Context Learning Mechanisms Beyond Toy Synthetic PCFGs

Mechanistic interpretability and in-context learning findings in formal linguistic settings currently rely on tiny, unambiguous PCFGs with vocabularies of fewer than 100 words and corpora of only ~20K sentences evaluated on toy models like nanoGPT. It remains completely unknown whether the specific internal circuits, parsing behaviors, and induction mechanisms documented in these toy studies survive when evaluated on richer formal grammars, ambiguous grammars, larger vocabularies, or natural syntax. Without systematic robustness evaluations across these broader settings, interpretability claims cannot be reliably extrapolated beyond the micro-benchmarks on which they were discovered.

Possible candidate · 3/5 runs5 papers report this60% from 2025+

Language Model Interpretability & In-Context Learning

Scope to testOpen

Testing the Transferability of Toy-Model Theoretical Guarantees to Deep Transformer Architectures

Theoretical analyses of in-context learning and mechanistic interpretability rely almost entirely on heavily idealized architectures, such as single-layer or two-layer transformers with linear attention, orthogonal embeddings, and no residual connections or normalization. Because these proofs make structural assumptions that diverge substantially from production models, it is currently unknown whether their theoretical guarantees hold in real large language models or function purely as loose analogies. Without empirical validation across the gap between toy settings and deep models, downstream interpretability methods risk relying on mechanistic claims that fail in standard architectures.

Possible candidate · 3/5 runs5 papers report this60% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.