Camera Pose, SLAM & Structure-from-Motion

Robust Pose and 3D Structure Estimation Under Imperfect and Open-World Instance Masks

Barrier to removeOpen

Strong candidate · 5/5 runs6 papers report this33% from 2025+

Generated automatically from the limitations stated in 6 papers (CVPR, ICLR, ICCV), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Current object-centric camera pose estimation, SLAM, and structure-from-motion pipelines treat 2D or 3D instance segmentation as a frozen upstream precondition (predominantly off-the-shelf Mask R-CNN or Mask3D). When masks contain boundary leakage, missed detections, or category misclassifications, geometric estimation degrades sharply because segmentation noise directly injects outlier correspondences that standard robust estimators cannot fully filter out. Consequently, existing methods cannot operate reliably outside closed object benchmarks with clean instance masks.

Why it matters

Object-level SLAM and pose estimation can run in unconstrained scenes without requiring curated, ground-truth, or category-restricted offline segmentations.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

1
Systematic Robustness and Noise-Sensitivity Benchmark: Evaluate leading object-level pose and SLAM pipelines across controlled synthetic and empirical segmentation perturbations (dilation, erosion, false positives, boundary jitter, and off-the-shelf zero-shot segmenters). Measure the pose error and tracking failure rate as a function of mask IoU and boundary precision.
2
Uncertainty-Aware Joint Feature and Geometry Optimization: Formulate pose and correspondence optimization to weight geometric residuals by upstream mask boundary confidence and feature consistency, soft-weighting uncertain mask boundaries rather than performing hard masking. Measure pose recall and translation/rotation errors on REAL275 and 3RScan with noisy detector outputs.
3
Iterative Feedback from Geometric Fit to Mask Refinement: Build an iterative loop where 3D geometric alignment and reprojection residuals feed back into updating the 2D instance assignment, discarding mis-segmented pixels during bundle adjustment. Measure camera localization and 3D object detection accuracy under noisy off-the-shelf detections.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Why it might fail

If upstream foundation models (e.g., SAM) produce near-perfect zero-shot instance masks that effectively dissolve the mask noise problem, or if end-to-end neural scene representations make explicit object instance masking obsolete for camera localization.

Evidence

Each paper's own statement of the limitation, verbatim.

Rethinking Correspondence-based Category-Level Object Pose EstimationCVPR 2025
Still relies on the fixed Mask R-CNN instance segmentation; segmentation noise directly injects outlier points, which the paper only mitigates rather than eliminates
Learning Shape-Independent Transformation via Spherical Representations for Category-Level Object Pose EstimationICLR 2025
Relies on offline Mask R-CNN instance segmentation masks and frozen DINOv2 features, inheriting their failure modes
Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D EnvironmentsCVPR 2024
Requires ground-truth or predicted instance segmentation masks as input; performance degrades notably with noisy Mask3D masks (e.g., matching recall 71.77→45.76 on 3RScan)
Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose EstimationCVPR 2024
Depends on off-the-shelf MaskRCNN segmentation; errors in segmentation masks must be handled by keypoint outlier filtering, and performance under poor segmentation is only indirectly addressed
IST-Net: Prior-Free Category-Level Pose Estimation with Implicit Space TransformationICCV 2023
Evaluation is limited to the 6 rigid REAL275 categories with MaskRCNN-provided masks; performance depends on high-quality instance segmentation
HS-Pose: Hybrid Scope Feature Extraction for Category-Level Object Pose EstimationCVPR 2023
Requires pre-computed instance segmentation masks (via MaskRCNN) and category labels; end-to-end performance depends on detector quality