Open Problems

Hand-Object Interaction & Affordance

Benchmarking Hand-Object Interaction Models Across Diverse Hand Morphologies and Embodiments

Scope to testPartly addressed
Possible candidate · 2/5 runs7 papers report this29% from 2025+

Generated automatically from the limitations stated in 7 papers (CVPR, NeurIPS, ECCV), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Existing hand representation and hand-object interaction models are predominantly evaluated on small demographic cohorts or fixed kinematic setups (e.g., standard anthropomorphic models like MANO or fixed robotic hands like ShadowHand). Because existing pipelines implicitly bind kinematics, palm-to-finger ratios, or subject identities into their representations, their performance on unseen human hand proportions and non-standard robotic end-effectors remains untested. As a result, it is unknown whether current interaction and affordance estimators degrade gracefully or fail catastrophically when applied outside their narrow training morphologies.

Why it matters

Establishes the first rigorous empirical baseline for how hand pose, mesh reconstruction, and affordance estimators generalize across diverse human demographics and robotic hand morphologies, providing clear evaluation standards for cross-embodiment dexterity.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Conduct a cross-dataset and cross-demographic evaluation of leading 3D hand and affordance models across diverse anthropometric benchmarks (varying palm-to-finger ratios, hand sizes, and ethnic/age diversity), measuring 3D joint error, mesh surface distance, and contact affordance transfer.

  2. 2

    Build a standardized cross-embodiment benchmark spanning simulated robotic hands (e.g., 3-finger, 4-finger, and non-anthropomorphic grippers) and human hands, measuring zero-shot and few-shot grasp/interaction transfer success rates.

  3. 3

    Systematically evaluate morphology-conditioned fine-tuning versus modular kinematic decoupling across current VAE and regression baselines to quantify the minimum adaptation data required for unseen hand structures.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

The study would fail if existing single-embodiment architectures are fundamentally too rigid to adapt without full architectural redesigns, turning an empirical evaluation into disparate engineering patches without clear comparative takeaways.

Evidence

Each paper's own statement of the limitation, verbatim.

Nearest existing work

Related open problems

Hand-Object Interaction & Affordance

Barrier to removeOpen

Robust Hand-Object Interaction Learning Beyond Cascaded Parametric Pose Estimators

Downstream hand-object interaction (HOI) and affordance models are universally bottlenecked by a strict reliance on upstream off-the-shelf 3D hand pose estimators (e.g., FrankMocap, HaMeR, MANO parameterizations) and bounding-box detectors. When these upstream tools encounter severe object occlusions, fast manipulation dynamics, or egocentric perspective distortion, their estimation errors cascade directly into downstream HOI training, causing catastrophic mis-projections and synthetic contact artifacts. Because downstream methods require clean parametric 3D meshes as preconditions, massive repositories of in-the-wild manipulation videos cannot be utilized without fragile filtering heuristics that discard over half the data.

Possible candidate · 2/5 runs42 papers report this45% from 2025+

Hand-Object Interaction & Affordance

Effect to explainOpen

Robust 3D Hand-Object Pose and Interaction Estimation Under Upstream 2D Segmentation and Localization Noise

Current hand-object interaction and 3D hand pose estimation models rely heavily on upstream 2D bounding box crops, hand segmentation masks, or pre-extracted interaction regions. In real-world egocentric scenes, upstream 2D segmenters and detectors degrade severely due to object occlusions, motion blur, and cluttered backgrounds, causing downstream 3D estimation to fail or hallucinate. Because models are trained and benchmarked assuming near-ideal 2D crops or masks, existing pipelines cannot operate reliably in autonomous, in-the-wild video streams.

Possible candidate · 3/4 runs9 papers report this56% from 2025+

Hand-Object Interaction & Affordance

Barrier to removeOpen

Fine-Grained 3D Hand Affordance Grounding and Generation for Small-Part Articulated Interactions

Current affordance learning pipelines rely on self-training loops, video extraction, or upstream generative HOI models that fail when contact targets are small functional components such as bottle caps, zippers, or rotary knobs. Because generated hand poses easily drift to adjacent object surfaces, models exhibit catastrophic failure on precision affordances (e.g., 0% accuracy on pull or sub-50% on twist), propagating noisy pseudo-labels across unobserved geometries. Downstream manipulation systems are consequently blocked from executing precision tasks like uncapping, pulling sliders, or fine bimanual manipulation.

Possible candidate · 3/5 runs4 papers report this75% from 2025+

Hand-Object Interaction & Affordance

Barrier to removeOpen

Hand-Object Affordance Estimation Beyond Clean Pre-Scanned Scenes and Predefined 3D Grids

Current methods for 3D hand-object interaction and affordance estimation depend strictly on clean geometric preconditions: high-fidelity 3D scans, pre-detected object instances, static overhead views, or tightly predefined 3D bounding grids. In practical manipulation and augmented reality scenarios, environments are unsegmented, viewpoints are dynamic, and point clouds are noisy and incomplete. Relying on manually tuned grid volumes and pre-extracted meshes prevents existing models from operating directly on raw, uncurated sensor streams.

Strong candidate · 5/5 runs3 papers report this33% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.