Open Problems

Hand-Object Interaction & Affordance

Dynamic Closed-Loop Execution and Task-Conditioned Hand-Object Manipulation

Barrier to removePartly addressed
Weak candidate · 1/4 runs3 papers report this33% from 2025+

Generated automatically from the limitations stated in 3 papers (NeurIPS, CVPR), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Current hand-object interaction and grasp synthesis frameworks evaluate grasp success almost exclusively as static contact configurations generated from full, pre-captured object point clouds. In dynamic deployment, these methods suffer severe rotational tracking drift and mid-trajectory drops because visual observations during motion are partial, occluded, and streaming rather than static and complete. Consequently, high static grasp success rates fail to translate into functional multi-step manipulation or object placement in the wild.

Why it matters

Enables dexterous grasp synthesis methods to execute continuous pick-carry-place sequences under real-world sensor constraints rather than terminating at static initial contact.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Benchmark representative static grasp synthesis models under streaming single-view partial observations during pick-and-place trajectories, measuring mid-trajectory drop rate and rotation tracking error relative to ground truth.

  2. 2

    Develop a closed-loop tracking and grasp-adjustment policy driven by real-time partial point clouds and contact feedback, measuring trajectory completion rate and orientation drift on standard manipulation benchmarks.

  3. 3

    Formulate a downstream-aware grasp synthesis framework that jointly predicts initial grasp poses and placement-compatible trajectories, measuring downstream placement success rate across varied object categories.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

End-to-end visuomotor diffusion policies trained directly on raw camera streams and multi-step tasks may bypass modular grasp-then-plan pipelines altogether, making standalone grasp-trajectory bridging less relevant if end-to-end learning scales efficiently to dexterous hands.

Evidence

Each paper's own statement of the limitation, verbatim.

Nearest existing work

Related open problems

Hand-Object Interaction & Affordance

Barrier to removeOpen

Robust Hand-Object Interaction Learning Beyond Cascaded Parametric Pose Estimators

Downstream hand-object interaction (HOI) and affordance models are universally bottlenecked by a strict reliance on upstream off-the-shelf 3D hand pose estimators (e.g., FrankMocap, HaMeR, MANO parameterizations) and bounding-box detectors. When these upstream tools encounter severe object occlusions, fast manipulation dynamics, or egocentric perspective distortion, their estimation errors cascade directly into downstream HOI training, causing catastrophic mis-projections and synthetic contact artifacts. Because downstream methods require clean parametric 3D meshes as preconditions, massive repositories of in-the-wild manipulation videos cannot be utilized without fragile filtering heuristics that discard over half the data.

Possible candidate · 2/5 runs42 papers report this45% from 2025+

Hand-Object Interaction & Affordance

Effect to explainOpen

Robust 3D Hand-Object Pose and Interaction Estimation Under Upstream 2D Segmentation and Localization Noise

Current hand-object interaction and 3D hand pose estimation models rely heavily on upstream 2D bounding box crops, hand segmentation masks, or pre-extracted interaction regions. In real-world egocentric scenes, upstream 2D segmenters and detectors degrade severely due to object occlusions, motion blur, and cluttered backgrounds, causing downstream 3D estimation to fail or hallucinate. Because models are trained and benchmarked assuming near-ideal 2D crops or masks, existing pipelines cannot operate reliably in autonomous, in-the-wild video streams.

Possible candidate · 3/4 runs9 papers report this56% from 2025+

Hand-Object Interaction & Affordance

Scope to testPartly addressed

Benchmarking Hand-Object Interaction Models Across Diverse Hand Morphologies and Embodiments

Existing hand representation and hand-object interaction models are predominantly evaluated on small demographic cohorts or fixed kinematic setups (e.g., standard anthropomorphic models like MANO or fixed robotic hands like ShadowHand). Because existing pipelines implicitly bind kinematics, palm-to-finger ratios, or subject identities into their representations, their performance on unseen human hand proportions and non-standard robotic end-effectors remains untested. As a result, it is unknown whether current interaction and affordance estimators degrade gracefully or fail catastrophically when applied outside their narrow training morphologies.

Possible candidate · 2/5 runs7 papers report this29% from 2025+

Hand-Object Interaction & Affordance

Barrier to removeOpen

Fine-Grained 3D Hand Affordance Grounding and Generation for Small-Part Articulated Interactions

Current affordance learning pipelines rely on self-training loops, video extraction, or upstream generative HOI models that fail when contact targets are small functional components such as bottle caps, zippers, or rotary knobs. Because generated hand poses easily drift to adjacent object surfaces, models exhibit catastrophic failure on precision affordances (e.g., 0% accuracy on pull or sub-50% on twist), propagating noisy pseudo-labels across unobserved geometries. Downstream manipulation systems are consequently blocked from executing precision tasks like uncapping, pulling sliders, or fine bimanual manipulation.

Possible candidate · 3/5 runs4 papers report this75% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.