Multi-Modality Cross Gait Recognition Combined with Transformer

QIN Yu-zhen; LIU Yi-xin; QING Lin-bo; HE Xiao-hai

doi:10.3969/j.issn.2096-6091.2023.19.001

您当前的位置：

首页 >

文章列表页 >

Multi-Modality Cross Gait Recognition Combined with Transformer

RESEARCH PAPER | 更新时间：2025-03-26

- Multi-Modality Cross Gait Recognition Combined with Transformer
- New Generation of Information Technology Vol. 6, Issue 19, Pages: 1-7(2023)
- 作者机构：
  
  1.四川大学电子信息学院，四川成都 610065
  2.四川大学华西医院国家老年疾病临床医学研究中心，四川成都 610041
- 作者简介：
- 基金信息：
- DOI：10.3969/j.issn.2096-6091.2023.19.001
  CLC： TP391.41
- Published：15 October 2023
- 稿件说明：
移动端阅览
秦予禛, 刘怡欣, 卿粼波, 等. 结合Transformer的多模态交叉步态识别[J]. 新一代信息技术, 2023, 6(19): 01-07

QIN Yu-zhen, LIU Yi-xin, QING Lin-bo, et al. Multi-modality Cross Gait Recognition Combined with Transformer[J]. New Generation of Information Technology, 2023, 6(19): 01-07
秦予禛, 刘怡欣, 卿粼波, 等. 结合Transformer的多模态交叉步态识别[J]. 新一代信息技术, 2023, 6(19): 01-07 DOI： 10.3969/j.issn.2096-6091.2023.19.001.

QIN Yu-zhen, LIU Yi-xin, QING Lin-bo, et al. Multi-modality Cross Gait Recognition Combined with Transformer[J]. New Generation of Information Technology, 2023, 6(19): 01-07 DOI： 10.3969/j.issn.2096-6091.2023.19.001.

摘要

近年来，随着神经网络的不断发展，人类步态识别技术逐渐成熟，被广泛应用于身份识别、健康评估等领域。针对现有的步态识别模型对外观变化的鲁棒性较低、提取特征的特异性较差等问题，在现有步态识别网络GaitPart的基础上，提出了一种结合Transformer的多模态交叉步态识别方法。一方面，引入步态轮廓图和骨骼图，设计多模态交叉融合模块，减少外观变化的影响；另一方面，结合Transformer与卷积神经网络，提取多尺度时间特征，丰富时空特征，提高识别准确率。实验结果表明，改进的方法在CASIA-B数据集的正常行走、背包和穿着外套三种场景下分别达到了98.8%、95.1%和84.2%的准确率，同时在自建步态数据集的不同视角下的结果也均有提升，大量实验充分证明了改进模型的优越性以及有效性。

Abstract

In recent years

with the continuous development of neural networks

human gait recognition technology has gradually matured and been widely used in fields such as identity recognition and health assessment. A multimodal cross gait recognition method combining Transformer is proposed based on the existing gait recognition network GaitPart. It is aiming at the problems of low robustness to appearance changes and poor specificity in feature extraction in existing gait recognition models. On the one hand

it introduces gait contour maps and skeleton maps

designs a multimodal cross fusion module to reduce the impact of appearance changes. On the other hand

it combines Transformer with convolutional neural networks to extract multi-scale temporal features

enrich spatiotemporal features

and improve recognition accuracy. The experimental results show that the improved method achieves accuracy of 98.8%

95.1%

and 84.2% in three scenarios of normal walking

backpacking

and wearing jackets on the CASIA-B dataset

respectively. At the same time

the results have also been improved in different perspectives on the self built gait dataset. A large number of experiments fully demonstrate the superiority and effectiveness of the improved model.

关键词

Keywords

references

WAN C S , WANG L , PHOHA V V . A survey on gait recognition [J ] . ACM Computing Surveys , 51 ( 5 ): 89 .

KUMAR M , SINGH N , KUMAR R , et al . Gait recognition based on vision systems: A systematic survey [J ] . Journal of Visual Communication and Image Representation , 2021 , 75 : 103052 .

HAN J , BHANU B . Individual recognition using gait energy image [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2005 , 28 ( 2 ): 316 - 322 .

WANG C , ZHANG J P , WANG L , et al . Human identification using temporal information preserving gait template [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 34 ( 11 ): 2164 - 2176 .

BASHIR K , XIANG T , GONG S G . Gait recognition without subject cooperation [J ] . Pattern Recognition Letters , 2010 , 31 ( 13 ): 2052 - 2060 .

SAMPATH DAKSHINA MURTHY A , KARTHIKEYAN T , VINOTH KANNA R . Gait-based person fall prediction using deep learning approach [J ] . Soft Computing , 2022 , 26 ( 23 ): 12933 - 12941 .

HE Y W , ZHANG J P , SHAN H M , et al . Multi-task GANs for view-specific feature learning in gait recognition [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 14 ( 1 ): 102 - 113 .

YOSHIYASU Y , SAGAWA R , AYUSAWA K , et al . Skeleton Transformer Networks: 3D Human Pose and Skinned Mesh from Single RGB Image [M ] // Computer Vision – ACCV 2018 . Cham : Springer International Publishing , 2019 : 485 - 500 .

YAN S J , XIONG Y J , LIN D H . Spatial temporal graph convolutional networks for skeleton-based action recognition [J/OL ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2018 , 32 ( 1 ). https://doi.org/10.1609/aaai.v32i1.12328 https://doi.org/10.1609/aaai.v32i1.12328 .

SI C Y , JING Y , WANG W , et al . Skeleton-based action recognition with spatial reasoning and temporal stack learning [C ] // Computer Vision – ECCV 2018: 15th European Conference , Proceedings, Part I . New York : ACM , 2018 : 106 - 121 .

TEEPE T , KHAN A , GILG J , et al . Gaitgraph: Graph convolutional network for skeleton-based gait recognition [C ] // 2021 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2021 : 2314 - 2318 .

SUN K , XIAO B , LIU D , et al . Deep high-resolution representation learning for human pose estimation [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 5693 - 5703 .

LI G D , GUO L J , ZHANG R , et al . TransGait: Multimodal-based gait recognition with set transformer [J ] . Applied Intelligence , 2023 , 53 ( 2 ): 1535 - 1547 .

AMSAPRABHAA M , NANCY JANE Y , KHANNA NEHEMIAH H . Multimodal spatiotemporal skeletal kinematic gait feature fusion for vision-based fall detection [J ] . Expert Systems with Applications , 2023 , 212 : 118681 .

CAO Z , SIMON T , WEI SHIH-EN , et al . Realtime multi-person 2D pose estimation using part affinity fields [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 7291 - 7299 .

CHAO H Q , HE Y W , ZHANG J P , et al . GaitSet: Regarding gait as a set for cross-view gait recognition [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 8126 - 8133 .

FAN C , PENG Y J , CAO C S , et al . GaitPart: Temporal part-based model for gait recognition [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 14225 - 14233 .

LIN B B , ZHANG S L , BAO F . Gait recognition with multiple-temporal-scale 3D convolutional neural network [C ] // Proceedings of the 28th ACM International Conference on Multimedia . New York : ACM , 2020 : 3054 - 3062 .

LIN S C , YANG L J , SALEEMI I , et al . Robust high-resolution video matting with temporal guidance [C ] // 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE , 2022 : 238 - 247 .

HERMANS A , BEYER L , LEIBE B . In defense of the triplet loss for person re-identification [J ] . ( 2017-11-21 )[ 2023-12-01 ] . http: http://arxiv.org/abs/1703.07737 http://arxiv.org/abs/1703.07737 .

SZEGEDY C , VANHOUCKE V , IOFFE S , et al . Rethinking the inception architecture for computer vision [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 2818 - 2826 .

HUANG X H , ZHU D W , WANG H , et al . Context-sensitive temporal feature learning for gait recognition [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 12909 - 12918 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 6000 - 6010 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7132 - 7141 .

YU S , TAN D , TAN T . A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition [C ] // 18th International Conference on Pattern Recognition (ICPR'06) . Piscataway : IEEE , 2006 , 4 : 441 - 444 .

HUANG Z , XUE D , SHEN X , et al . 3D local convolutional neural networks for gait recognition [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . 2021 : 14920 - 14929 .

Views

151

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Bone Age Prediction Based on Lightweight VGG16 and Attention Mechanism

Related Author

QIN Yu-zhen

HE Xiao-hai

GUO Wei

GUO Jin-xing

CHEN Guang-xin

HAN Xue-shan

GUO Jinxing

Chen Guangxin

Related Institution

College of Life Science， Mudanjiang Medical University

Hongqi Hospital Affliated to Mudanjiang Medical University

School of Medical Imaging， Mudanjiang Medical University

⁰