Human Pose Estimation from Images

Skeletons taken from the nearest matching exemplar using geometric blur and warping for better alignment.   The exemplar based approach relies on modern computational power to store many possible examples of the appearance of humans in various poses, and requires a robust similarity measure (eg geometric blur based descriptors) to compare novel images to stored exemplars. The skeleton associated with the best matching exemplar is used for the novel image. An aligning transformation between the exemplar and the novel image can be applied to the skeleton to improve accuracy. Video or motion information can be used when available. This approach has the advantage that arbitrarily complex priors on pose and appearance can be handled by simply sampling the possibilities. The disadvantage is that the number of samples or exemplars may increase beyond a manageable level as the range of possible appearances increases. Papers Skeleton taken from the nearest matching exemplar using an optical flow based descriptor (middle). Matching to a rendered 3d figure (right).

Finding the correspondence from parts of a model to locations on an image. The locations are selected using a bottom up approach starting with edge detection. Example pose estimations (click for larger version).   Our bottom up approach uses low level detectors based on edges to find parts of an image that might be limbs. These are then assembled into an estimate of the whole body configuration using pairwise constraints and approximate integer quadratic programming as in our work on shape matching and object recognition. The IQP framework allows more general constraints between body parts than can be modeled with traditional tree structured models. The bottom up approach can provide better localization than exemplar based approaches and can handle a wide range of poses with lower computational and storage requirements, but suffers when the low level limb detectors fail. (More from Xiaofeng's page.) Papers

Papers