Workflow from Scientific Research

Proposed architecture for egocentric 3D human pose estimation consisting of two modules: a) interchangeable 2D pose detector that predicts heatmaps from the input RGB image; b) multi-branch auto-encoder that finds a representation of poses which includes also a level of uncertainty of predictions per joint. Alongside the main branch, for 3D joint location prediction, two auxiliary branches as used at training-time to improve latent space distribution. Branch ii) estimates local joint rotations, forcing them to be consistent with those rotations extracted by the predicted pose from i); branch iii) forces the latent space to include a level of uncertainty of the 2D joint locations by reconstructing the given predicted heatmaps from the pose embedding. These additional branches have demonstrated considerable improvements with respect to a standard AE architecture, as shown in Section 5.

DOI