Open Problems

Visual Reasoning & Embodied Navigation

Visual Reasoning Without Dense Scene-Graph and Rationale Preconditions

Barrier to removePartly addressed
Possible candidate · 2/5 runs6 papers report this67% from 2025+

Generated automatically from the limitations stated in 6 papers (ACL, ICLR, CVPR), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Current visual reasoning methods depend on restrictive preconditions, including ground-truth answer hints during rationale generation, curated contrastive image pairs, and dense GQA-style scene-graph annotations. Because these annotations do not exist in most real-world datasets or uncurated domains, existing techniques cannot be transferred to novel environments or extended to multi-image tasks. Furthermore, evaluations have remained confined to narrow, single-image benchmarks, leaving model robustness in unannotated settings unknown. Consequently, visual reasoning systems cannot be deployed outside the handful of bespoke benchmarks engineered to support them.

Why it matters

Enables visual reasoning models to be trained and deployed on arbitrary natural images and multi-image collections without requiring expensive scene-graph or rationale annotations. It also establishes standardized robustness baselines for visual reasoning under sparse supervision.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Systematic annotation ablation study: Evaluate existing rationale- and scene-graph-dependent models on standard VQA benchmarks while progressively removing access to ground-truth hints, contrastive pairs, and graph annotations to quantify exact performance degradation across question types.

  2. 2

    Self-supervised rationale extraction: Train visual reasoning pipelines to derive intermediate reasoning steps from unsupervised object proposals and end-task question-answer supervision alone, measuring VQA accuracy on datasets lacking scene graphs.

  3. 3

    Multi-image generalization evaluation: Test single-image reasoning models on multi-image benchmarks (such as ISVQA) without curated contrastive pairs to establish baseline reasoning failure rates in multi-view contexts.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

Frontier multimodal large language models trained on massive internet-scale data may implicitly solve visual reasoning and multi-image grounding zero-shot, dissolving the need for explicit annotation-efficient reasoning architectures.

Evidence

Each paper's own statement of the limitation, verbatim.

Nearest existing work

Related open problems

Visual Reasoning & Embodied Navigation

Scope to testOpen

Systematic Generalisation and Robustness Evaluation of Visual Reasoning Methods Across Distinct Perceptual Regimes

Current visual reasoning techniques are predominantly evaluated on static images containing localized, explicitly present visual evidence where language reasoning dominates. Consequently, it is unknown whether their performance improvements transfer to vision-dominant perceptual tasks, temporal video sequences, global visual synthesis problems, or queries involving implicit visual evidence. Practitioners cannot determine whether a method’s gains represent generalizable visual reasoning or narrow optimizations for localized static VQA. A systematic robustness evaluation across these untested dimensions is necessary to chart where existing methods hold and where they fail.

Possible candidate · 3/5 runs4 papers report this100% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.