Open Problems

3D Point Clouds & LiDAR Perception

Benchmarking and Adapting Indoor-Trained 3D Perception and Generation Models to Outdoor and LiDAR Point Cloud Scenes

Scope to testOpen
Possible candidate · 3/5 runs20 papers report this53% from 2025+

Generated automatically from the limitations stated in 20 papers (CVPR, NeurIPS, ICML), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Nearly every 3D foundation-ish model — for detection, segmentation, registration, scene generation, and novel view synthesis — is trained and validated exclusively on indoor RGB-D or synthetic benchmarks (ScanNet, SUN RGB-D, 3D-FRONT, Habitat simulators), with explicit author acknowledgments that outdoor and LiDAR data is untested. This means the field's core 3D representations are calibrated to room-scale geometry (bounded depth, dense views, flat priors like indoor depth predictors and category vocabularies), and nobody can currently deploy them on autonomous-driving or aerial scans without unquantified degradation. The consequence is a silent incompatibility: outdoor researchers must either rebuild models from scratch or trust unmeasured transfer.

Why it matters

Quantified, reusable evidence of what breaks and why when indoor-native 3D models meet outdoor scans — plus adapted checkpoints and a benchmark suite that future model papers must report on, displacing the ScanNet-only evaluation norm.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Build an "indoor-to-outdoor transfer suite": take 5–8 representative released models spanning detection, segmentation, and reconstruction, and evaluate them, off the shelf, on outdoor benchmarks (SemanticKITTI, nuScenes, ONCE, Waymo) under matched class mappings. Measure class-wise mIoU/AP and geometry-driven failure modes (density, extent, sensor noise) versus indoor baselines, isolating which failure comes from domain shift versus sensor modality.

  2. 2

    Interpolate a mid-domain probe: collect or reconstruct paired indoor-like/outdoor-like scans (e.g., campus-scale multi-view LiDAR, docked indoor-outdoor sequences like Aria/Tartan datasets) and measure performance as a function of scene extent, point density, and view coverage — producing transfer curves rather than a binary indoor/outdoor result.

  3. 3

    Domain adaptation minimalism: apply test-time adaptation (e.g., entropy minimization, pseudo-label self-training) to the worst-performing models on outdoor scans and measure how much of the transfer gap can be closed without outdoor training data; report whether priors learned indoors (depth scales, room-size layouts) are repairable at inference time or baked in.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

The degradation turns out to be trivially explained by point-density and sensor-type mismatch alone (so the interesting "indoor prior" story dissolves into standard domain adaptation), or existing robust-model papers have already quietly included outdoor splits that were simply never cited in the limitation statements.

Sub-problems

  • Cross-Domain Robustness and Out-of-Domain Generalization in Feed-Forward 3D Gaussian Scene Priors

    Current feed-forward 3D Gaussian Splatting and novel view synthesis pipelines rely on priors and models trained exclusively on curated indoor datasets such as 3D-FRONT, ScanNet++, and HyperSim. Because these systems have not been evaluated outside room-scale synthetic or scanned indoor environments, their actual degradation on outdoor, unbounded, or unstructured real-world scenes remains unquantified. Consequently, researchers cannot determine whether architectural limitations, depth-prior biases, or dataset artifacts are the primary bottleneck when deploying these models beyond indoor domains. A rigorous multi-domain robustness evaluation is necessary to establish the operational boundaries and transfer limits of existing indoor-trained 3D scene synthesis methods.

Evidence

Each paper's own statement of the limitation, verbatim.

Show all 20 papers

Nearest existing work

Related open problems

3D Point Clouds & LiDAR Perception

Scope to testOpen

An Outdoor-Scale Benchmark for 3D Scene Understanding Methods Currently Validated Only on Indoor ScanNet-Derived Data

Nearly the entire 3D perception, scene reconstruction, grounding, and question-answering stack is trained and evaluated on ScanNet-family indoor scans, so claims of scene understanding hold only within bounded, static, room-scale environments with known camera poses. As a result, no one can tell whether reported architectural advances—reasoning modules, open-vocabulary grounding, generalization arguments—transfer to outdoor, unbounded, or unsegmented scenes at all, or whether they simply exploit ScanNet's statistics. Progress reports that look cumulative within this benchmark family may be incommensurable with outdoor work that cannot reuse the same evaluation. Any paper today claiming 3D scene understanding is doing so on a single environmental regime.

Possible candidate · 3/5 runs39 papers report this30% from 2025+

3D Point Clouds & LiDAR Perception

Barrier to removeOpen

Robust Multimodal 3D Perception Under Sensor Extrinsic and Intrinsic Miscalibration

State-of-the-art multimodal 3D perception architectures strictly require accurate extrinsic and intrinsic calibration between LiDAR and camera frames to construct point-to-pixel projections, frustum cost volumes, and cross-attention neighbourhoods. In real-world deployments, mechanical vibration, thermal expansion, and mounting shifts cause calibration parameters to drift, leading to severe geometric misalignment. Because current multimodal models hardcode these spatial transformations into their feature lifting layers, minor calibration errors cause catastrophic performance degradation or complete pipeline failure.

Possible candidate · 2/5 runs13 papers report this38% from 2025+

3D Point Clouds & LiDAR Perception

Barrier to removePartly addressed

Benchmark and Training Protocols for 3D Perception from Unpaired or Calibration-Free 2D-3D Data

Nearly every current 2D-3D fusion method assumes registered multi-view imagery, known intrinsics/extrinsics, and per-frame paired data — assumptions that hold only in curated lab captures. The consequence is that point clouds collected standalone (LiDAR scans, crowdsourced reconstructions, legacy datasets) cannot benefit from image features at all, and methods cannot be deployed where a fixed calibrated rig with synchronized cameras is impractical. Progress is blocked because no established protocol or dataset exists for learning 2D-3D correspondences from unpaired, uncalibrated, or single-view inputs.

Possible candidate · 2/5 runs11 papers report this17% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.