DensePose
Graphics research from INRIA, CentraleSupélec and Facebook AI proposes method of calculating 3D model of people from 2D video using neural networks in realtime, and can even apply texture mapping to its subjects:
In this work, we establish dense correspondences between an RGB image and a surface-based representation of the human body, a task we refer to as dense humanpose estimation. We first gather dense correspondences for 50K persons appearing in the COCO dataset by introducing an efficient annotation pipeline. We then use our dataset to train CNN-based systems that deliver dense correspondence ‘in the wild’, namely in the presence of back-ground, occlusions and scale variations. We improve ourtraining set’s effectiveness by training an ‘inpainting’ net-work that can fill in missing ground truth values, and report clear improvements with respect to the best results that would be achievable in the past. We experiment with fully-convolutional networks and region-based models and observe a superiority of the latter; we further improve accuracy through cascading, obtaining a system that delivers highly-accurate results in real time.
More Here