View-embedding GCN for skeleton-based cross-view gait recognition

Gait has emerged as a promising biometric modality due to its non-invasive nature and the ability to capture samples from a distance. Model-based gait recognition using skeleton data conveys rich information that remains invariant to carried objects and clothing variations. However, viewing a person from different angles alters their gait posture, resulting in increased intra-subject variability compared to inter-subject variability. Therefore, we propose a novel framework, View-embedding Modified Residual Graph Convolutional Network (VeMResGCN), for cross-view gait recognition (CVGR) by exploiting two modules: Modified Residual Graph Convolutional Network (MResGCN) and View-embedding Feature Extraction (VeFE) for viewinvariant features. A state-of-the-art pose estimation algorithm extracts skeleton key points from raw video input, from which multiple features (e.g., relative joint positions, motion velocities, and bone structures) are computed. The final feature vector for gait recognition is computed by consolidating the features from the MResGCN and VeFE modules. To the best of our knowledge, this work is the first to extract view-invariant features in a unified Graph Convolutional Network (GCN) for skeletonbased CVGR. We evaluate our proposed framework on two of the largest publicly available skeleton datasets, CASIA-B and OUMVLP-Pose, under challenging covariates of clothing variation and carried objects. Results demonstrate that VeMResGCN significantly outperforms state-of-the-art methods with average rank-1 accuracies of 90.3%, 80.7%, and 73.4% for normal, carried object, and clothing variations on CASIA-B, and 71.0% on OU-MVLP in terms of skeleton-based CVGR. These results demonstrate the ability of our proposed framework to maintain superior CVGR performance despite the presence of carried objects and clothing variations. The proposed framework holds strong implications for real-world biometric applications, including robust person re-identification and surveillance systems, where maintaining consistent recognition across varying views and covariates is crucial. The source code will be available on https://github.com/RayAusrukona/VeMResGCN.