Interacting two-hand reconstruction is a fundamental task in applications such as virtual reality. This task remains unsolved because of severe mutual occlusions and appearance similarity.

Virtual reality glasses. Image credit: Max Pixel, CC0 Public Domain

A recent paper published on arXiv.org proposes Interacting Attention GraphHand (IntagHand), a novel graph convolutional network (GCN) based single image two-hand reconstruction method.

In order to utilize the interaction context between two hands, traditional GCN is equipped with two novel attention modules. The first produces better alignment between the hand vertices and the image features. The second improves the reconstruction accuracy for closely interacting poses by implicitly modeling the two-hand interaction context.

It is shown that the proposed method outperforms existing methods by a large margin. For the first time, well-aligned two-hand reconstruction results on various in-the-wild images are demonstrated.

Graph convolutional network (GCN) has achieved great success in single hand reconstruction task, while interacting two-hand reconstruction by GCN remains unexplored. In this paper, we present Interacting Attention Graph Hand (IntagHand), the first graph convolution based network that reconstructs two interacting hands from a single RGB image. To solve occlusion and interaction challenges of two-hand reconstruction, we introduce two novel attention based modules in each upsampling step of the original GCN. The first module is the pyramid image feature attention (PIFA) module, which utilizes multiresolution features to implicitly obtain vertex-to-image alignment. The second module is the cross hand attention (CHA) module that encodes the coherence of interacting hands by building dense cross-attention between two hand vertices. As a result, our model outperforms all existing two-hand reconstruction methods by a large margin on InterHand2.6M benchmark. Moreover, ablation studies verify the effectiveness of both PIFA and CHA modules for improving the reconstruction accuracy. Results on in-the-wild images further demonstrate the generalization ability of our network. Our code is available at this https URL.

Research paper: Li, M., “Interacting Attention Graph for Single Image Two-Hand Reconstruction”, 2022. Link: https://arxiv.org/abs/2203.09364

You can offer your link to a page which is relevant to the topic of this post.