Open Problems

Text-to-Image & Video Generation/Editing

Dense Motion Supervision Without Optical-Flow Ground Truth

Barrier to removePartly addressed
Strong candidate · 5/5 runs8 papers report this100% from 2025+

Generated automatically from the limitations stated in 8 papers (CVPR, ICCV, ICML), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Nearly every pipeline that needs dense correspondence—video generation control, camera pose estimation, motion attribution—currently anchors its supervision or its inference to optical flow, either ground-truth annotations or outputs of a pretrained model (RAFT, AllTracker). This couples training data to static-camera, high-quality, curated footage, caps dataset size (22k–630k pairs), injects flow errors directly into downstream quality (cycle-consistency errors >1000 px), and bakes in simplifying assumptions like first-frame conditioning or pinhole intrinsics. Every consumer of flow inherits the same ceiling set by flow estimators and the data they were trained on.

Why it matters

Video generation, pose/SLAM, and motion-attribution systems trained on vastly larger, uncurated video without inheriting flow-estimator errors; flow stops being a single point of failure.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Build a correspondence benchmark with per-domain reliability labels: run existing flow estimators (RAFT, AllTracker, video-depth-induced correspondence) on diverse domains (egocentric, cartoon, camera-only motion, dynamic scenes) and measure agreement against multi-teacher consensus plus held-out hand annotations. Deliverable: a dataset and a per-pixel confidence map showing where flow supervision is trustworthy. Measured: consensus error vs. domain, correlation of confidence with downstream task error.

  2. 2

    Train downstream models (e.g., video generation control or pose estimation) with confidence-weighted flow supervision, where unreliable flow pixels are down-weighted or replaced by self-supervised consistency losses (cycle consistency, photometric warping). Measured: downstream quality as a function of how much filtered/confident flow is retained, compared to hard-thresholded filtering pipelines.

  3. 3

    Distill flow supervision from self-supervised objectives only—train a correspondence model using cycle-consistency over unlabeled video and evaluate whether it substitutes for RAFT/AllTracker outputs in the pipelines above. Measured: downstream task parity against flow-anchored baselines with no flow model in the loop.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

Self-supervised or confidence-weighted correspondence may simply be too weak where optical flow is genuinely needed—precise, dense, large-displacement matching—and downstream tasks may end up needing full curated flow supervision anyway.

Sub-problems

  • Self-Supervised Training and Calibration for Flow-Driven Camera Pose Estimation

    Current learning-based camera pose and Structure-from-Motion architectures rely strictly on optical flow backbones trained with synthetic ground-truth annotations or fixed off-the-shelf flow estimators. When deployed on in-the-wild capture domains such as egocentric video, these models fail because real-world optical characteristics, lens distortions, and dynamic artifacts diverge sharply from synthetic training distributions. Without self-supervised training formulations, practitioners cannot fine-tune flow-driven pose models directly on unlabelled target domain videos where ground-truth optical flow is physically impossible to obtain.

Evidence

Each paper's own statement of the limitation, verbatim.

Nearest existing work

Related open problems

Text-to-Image & Video Generation/Editing

Barrier to removePartly addressed

Systematically Benchmarking and Calibrating VLM-as-a-Judge Failure Modes in Image and Video Generation

Current text-to-image and video generation benchmarks overwhelmingly rely on Vision-Language Models (VLMs) like GPT-4o, Gemini, and Qwen-VL as automated judges for win rates and task success metrics. However, these VLM judges systematically exhibit low correlation with human raters on fine-grained visual details, suffer from position and same-family evaluator biases, and under-penalize visual artifacts such as boundary glitches or text corruption. Consequently, benchmark progress across generative models is structurally uninterpretable, as claimed state-of-the-art gains may reflect evaluator artifacts rather than genuine improvements in visual quality and prompt fidelity.

Possible candidate · 2/5 runs15 papers report this100% from 2025+

Text-to-Image & Video Generation/Editing

Barrier to removePartly addressed

Benchmarking and Mitigating Upstream LLM Prompting Dependencies in Text-to-Visual Generation

Modern text-to-image and text-to-video generation and editing methods rely heavily on external large language models (LLMs) for prompt rewriting, sub-prompt decomposition, and attribute filtering. Consequently, visual generation quality is bottlenecked by upstream LLM biases, template artifacts, and hallucinated attributes, frequently failing on ambiguous or in-the-wild user instructions unless manually tuned. Because no systematic study has evaluated visual generation frameworks across diverse upstream prompt generators and decomposition strategies, the sensitivity and failure modes of these pipelines remain unquantified.

Possible candidate · 2/5 runs12 papers report this58% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.