Open Problems

3D Shape Modeling & Surface Reconstruction

Reducing the Computational and Fidelity Cost of Distilling 2D Diffusion Priors into Explicit 3D Representations

Barrier to removeOpen
Possible candidate · 3/5 runs9 papers report this40% from 2025+

Generated automatically from the limitations stated in 9 papers (CVPR, NeurIPS, ICCV), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Every pipeline that extracts 3D from pretrained 2D diffusion models currently pays a massive per-object tax: hundreds of rendered views, hundreds of diffusion evaluations, per-scene hyperparameter hand-tuning, and per-identity training — and still inherits the 2D prior's view-inconsistency and bias toward common categories. Nobody can train a single shared 3D generative model end-to-end on a shape dataset at acceptable cost, so 3D diffusion remains per-instance rather than dataset-scale. This blocks both generalization guarantees and practical deployment.

Why it matters

Dataset-scale 3D diffusion with predictable (rather than empirical) generalization, and 3D generation at costs competitive with direct 3D descriptors.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Cache-and-distill: run the expensive multi-view diffusion sampling once per training shape, store the (view, noise, output) triples, and train a direct 3D generator (e.g., a feedforward point-cloud/Gaussian head) on them offline. Measure: reconstruction quality (Chamfer/PSNR) vs. the iterative baseline, and inference cost at generation time.

  2. 2

    Consistency-style few-step sampling: fine-tune the 2D diffusion backbone so multi-view generation works in 1–2 steps with cross-view attention enforcing 3D consistency, evaluated on view-consistency metrics (epipolar error) and detail fidelity vs. the 30-step baseline.

  3. 3

    Shared end-to-end training: extend SSDNeRF-style autodecoding to a single model over a multi-category dataset, measuring whether one shared model matches per-identity trained models and whether category generalization improves.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

The distillation step may simply reproduce the 2D prior's biases and multi-view inconsistencies in the student model, so cheap inference buys nothing in fidelity. If consistency training degrades high-frequency detail, the 1–2-step model fails precisely on the cases that matter.

Sub-problems

  • Resolving Multi-View Inconsistencies and High-Frequency Detail Degradation in Diffusion-Guided 3D Gaussian Splatting

    Diffusion-based novel view synthesis and 3D Gaussian Splatting pipelines rely on 2D or video diffusion models to generate novel perspectives or supervise unobserved regions. Because these diffusion priors operate in 2D image or frame space without strict 3D geometric inductive biases, generated views suffer from cross-view appearance drift, pose misalignment, and high-frequency inconsistencies. When fused into a 3D Gaussian representation, these errors accumulate into structural floaters, blurred textures, and severe geometric degradation under occlusion or uncommon viewpoints.

Evidence

Each paper's own statement of the limitation, verbatim.

Show all 9 papers

Nearest existing work

Related open problems

3D Shape Modeling & Surface Reconstruction

UnclassifiedOpen

Cross-Category and Vocabulary Generalization Benchmark for Text-to-3D Shape Models

Current text-driven 3D shape generation and modeling methods are predominantly trained and benchmarked strictly on Text2Shape, which is limited to chairs and tables (~11.5k shapes and a ~3.6k-word vocabulary). Consequently, it is unknown whether existing architectural designs, text encodings, and shape priors maintain generative fidelity when exposed to out-of-domain object categories or open-vocabulary text prompts. Without evaluating existing methods across broader category and vocabulary distributions, the field cannot determine if current performance gains reflect generalizable shape modeling or dataset-specific memorization.

Weak candidate · 1/4 runs3 papers report this0% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.