Open Problems

Digital Humans, Avatars & Virtual Try-On

Overcoming Parametric 3DMM Expressiveness and Mouth Interior Bottlenecks in Monocular Facial Avatars

UnclassifiedPartly addressed
Possible candidate · 3/5 runs11 papers report this91% from 2025+

Generated automatically from the limitations stated in 11 papers (CVPR, NeurIPS), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Neural head avatar pipelines universally rely on parametric mesh models like FLAME as geometric proxies, inheriting their topological limitations. Because FLAME lacks internal oral anatomy (teeth and tongue) and fine-grained geometric detail, avatars suffer from severe projection artifacts such as teeth textures baked directly onto lip surfaces and distorted speech articulation. Furthermore, low-dimensional linear expression spaces cap the capture of dynamic micro-expressions, dynamic wrinkles, and non-parametric regions like hair.

Why it matters

Enables realistic conversational avatars capable of natural speech articulation without oral rendering artifacts or expression damping.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Hybrid Parametric-Volumetric Mouth Interior: Augment a FLAME-guided neural radiance or Gaussian avatar with a dedicated articulated oral cavity representation (combining rigid dental structures with a deformable tongue model), measuring lip-mouth rendering fidelity (LPIPS/PSNR) and oral depth/penetration error on speaking video datasets.

  2. 2

    Residual Per-Vertex Dynamic Displacement Learning: Train a lightweight neural deformation field conditioned on audio and high-resolution video frames that predicts non-linear geometric residuals on top of 3DMM coefficients, measuring surface reconstruction error against multi-view/3D scan benchmarks (e.g., capture of micro-wrinkles and non-parametric facial dynamics).

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

End-to-end generative 3D diffusion and video models might bypass explicit 3DMM mesh conditioning altogether if they learn temporal 3D consistency directly from large video corpora. Additionally, monocular oral capture remains inherently under-constrained due to severe self-occlusion during speech.

Evidence

Each paper's own statement of the limitation, verbatim.

Show all 11 papers

Nearest existing work

Related open problems

Digital Humans, Avatars & Virtual Try-On

Barrier to removePartly addressed

Joint Pose Refinement and Error-Tolerant Conditioning for Downstream Human Avatar Synthesis

Current avatar creation and virtual try-on pipelines treat SMPL/SMPL-X estimations, camera calibrations, and segmentation masks as fixed, ground-truth preconditions. When off-the-shelf estimators inevitably produce misaligned or out-of-distribution body parameters on in-the-wild images, these errors cascade into severe geometric distortion, stitching artifacts, and generation failure. Because existing downstream pipelines do not jointly optimize human body parameters or account for parameter uncertainty, high-fidelity human generation remains restricted to controlled studio captures with pristine initializations.

Strong candidate · 5/5 runs8 papers report this25% from 2025+

Digital Humans, Avatars & Virtual Try-On

Barrier to removePartly addressed

Reducing Downstream Dependency of Garment/Avatar Generation on Upstream 2D and 3D Human-Parsing Predictions

Every pipeline in this space chains off-the-shelf estimators — pose keypoint detectors, DensePose, SMPL/SMPL-X fitting, segmentation/parsing models, SAM — and treats their outputs as ground truth. When these upstream predictions are noisy, fail on exaggerated poses, unusual garments, or garment-only imagery, the downstream model inherits or amplifies the errors (garment leaking, misinitialized garment geometry, failed reposing). No one measures or controls how error propagates through this preprocessing chain, so each new pipeline is brittle in its own undocumented way.

Possible candidate · 3/5 runs8 papers report this60% from 2025+

Digital Humans, Avatars & Virtual Try-On

Effect to explainPartly addressed

Benchmarking and Improving 3D Human Body Model Robustness Across Underrepresented Demographics and Non-Standard Morphologies

Existing 3D statistical body models and avatar pipelines are predominantly trained on narrow demographic subsets (such as adult European CAESAR scans) and evaluated solely on standard normative body templates. Consequently, downstream avatars and virtual try-on methods systematically produce distorted geometry, unnatural surface artifacts, and inaccurate anthropometric measurements on children, adolescents, diverse global populations, individuals with disabilities, and non-standard body proportions. Because existing benchmarks do not measure generalization across diverse body shapes, failure modes on these populations remain unquantified and unaddressed.

Possible candidate · 2/5 runs4 papers report this50% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.