Open Problems

Diffusion Models

Extending Class-Conditional ImageNet Generation Methods to Text-Conditioned, Open-Domain Synthesis

Scope to testOpen
Possible candidate · 3/5 runs53 papers report this83% from 2025+

Generated automatically from the limitations stated in 53 papers (CVPR, ICLR, ICML), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Nearly every method validated on class-conditional ImageNet 256×256 stalls at that benchmark, and it is unknown whether any of them survive contact with natural language conditioning and open-caption distributions. Class labels provide weak, discrete conditioning with 1,000 modes; text requires cross-attention or caption-token conditioning, resolution diversity, and open-vocabulary semantics. Until the class-to-text transition is demonstrated, the large literature of ImageNet-benchmarked generative advances cannot be assumed to transfer to practical text-to-image systems.

Why it matters

A bridge from the ImageNet benchmark literature to open-domain text-to-image, letting dozens of validated techniques (efficient tokens, flow objectives, AR tokenizers) be ranked on realistic conditioning and making cross-method comparisons on captions possible for the first time.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Take 2–3 representative ImageNet-256 methods (e.g., a DiT/SiT-based flow or AR model plus a token-reduction variant), replace the class embedding with T5/CLIP caption conditioning on a text-to-image dataset (e.g., a CC12M/JourneyDB fine-tune from ImageNet-pretrained checkpoints), and measure FID-30k, CLIP score, and GenEval against the class-conditional baselines and against a text-conditioned DiT reference (PixArt-alpha style).

  2. 2

    Study what specifically breaks under text conditioning: measure conditioning-signal utilization (caption-adherence metrics vs. guidance scale), tokenizer/patch-size sensitivity at 512×512, and compute cost at open resolutions—identifying which ImageNet-era design choices (patch sizes, token budgets, guidance schedules) are the actual barriers.

  3. 3

    Release a standardized text-conditioned benchmark suite and re-evaluation protocol so class-conditional ImageNet papers can be compared directly on text-to-image, reporting CLIP score, GenEval, and FID under matched compute.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

Text-to-image fine-tuning at scale requires compute and data access far beyond an academic budget, so adapted models may be too weak for the comparison to be informative. If every method transfers or every method fails uniformly, the study yields no discriminative findings.

Sub-problems

  • Benchmarking Flow Matching Across Conditioning Regimes, Architectures, and Resolutions

    Recent advances in flow matching and continuous normalizing flows are almost exclusively evaluated on class-conditional ImageNet-256 with latent DiT backbones. It remains unknown whether key algorithmic properties—such as path straightness, optimal transport formulation advantages, and ODE integration efficiency—transfer to open-domain text conditioning, higher resolutions, pixel-space training, or unconditional settings. Generative modeling practitioners cannot determine whether theoretical improvements reported on class-conditional ImageNet reflect intrinsic algorithmic superiority or dataset-specific artifacts.

  • Evaluating the Transferability of ImageNet Generative Architecture Innovations to Text-to-Image Generation

    Recent architectural and training innovations in diffusion transformers and autoregressive image models are developed and tuned exclusively on closed-world, class-conditional benchmarks like ImageNet-1K. Because class conditioning provides fixed, low-dimensional categorical signals, it remains unknown whether reported gains in sampling efficiency, tokenization schemes, and attention backbones hold under open-vocabulary text conditioning. Researchers cannot currently determine which modeling advances reflect fundamental generative improvements versus artifacts of 1,000-class label embeddings.

  • Systematic Generalization Benchmark of Generative Vision Transformer Token Reduction Across Text-to-Image Conditioning

    Existing token reduction and efficiency mechanisms for generative vision transformers have been developed and evaluated almost exclusively on class-conditional ImageNet. It is currently unknown whether these pruning, masking, and token-budgeting strategies retain visual fidelity and cross-modal alignment when moving from 1,000 discrete class embeddings to dense, open-vocabulary text conditioning. Consequently, researchers lack empirical evidence on whether current efficiency gains transfer to real-world text-to-image pipelines or collapse under open-domain cross-attention.

Evidence

Each paper's own statement of the limitation, verbatim.

Show all 53 papers

Nearest existing work

Related open problems

Diffusion Models

Barrier to removeOpen

Black-Box and Few-Step Control of Diffusion Models Without Intermediate State Access

Existing diffusion manipulation, guidance, and acceleration methods almost universally require white-box model access, intermediate noise predictions, or dozens of trajectory steps. Consequently, these techniques fail when interacting with commercial API endpoints that only return final image outputs, or with modern few-step distilled models (e.g., 1-to-4-step generators) where temporal redundancy across timesteps does not exist. Practitioners cannot apply state-of-the-art guidance, watermarking, or editing pipelines to closed-source foundation models or ultra-fast distilled backends. Resolving how to steer or audit generation under sample-only access is required to bridge the gap between white-box research methods and deployed diffusion infrastructure.

Strong candidate · 4/5 runs6 papers report this50% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.