3D Hand Pose Estimation Using Convolutional Neural Networks
2533D hand pose is challenging due to complicated variations caused by high Degree of Freedom articulations, multiple viewpoints, self-similar parts, severe self-occlusions, different shapes, and sizes. My Ph.D. work tackles 1) the multiple viewpoints and complex articulations of hand pose estimation by decomposing and transforming the input and output space by spatial transformations following the hand structure; 2) the multi-modality of the locations for occluded hand joints by a hierarchical mixture density deep network which leverages the state-of-the-art hand pose estimators based on Convolutional Neural Networks to facilitate feature learning while models the multiple modes in a two-level hierarchy to reconcile single-valued (for visible joints) and multi-valued (for occluded joints) mapping in its output; 3) the lack of complete labeled real hand datasets by a tracking system with six 6D magnetic sensors and inverse kinematics to automatically obtain 21-joints hand pose annotations of depth maps.
See more at https://www.microsoft.com/en-us/research/video/3d-hand-pose-estimation-using-convolutional-neural-networks/