基于改进的多算法融合地铁站内乘客行为识别

章宇翔; 李先旺; 贺德强; 贺岁球; 陈彦君; 李琴

doi:10.19713/j.cnki.43-1423/u.T20230047

您当前的位置：

首页 >

文章列表页 >

基于改进的多算法融合地铁站内乘客行为识别

智慧交通与物流 | 更新时间：2023-12-05

- 基于改进的多算法融合地铁站内乘客行为识别
- Passenger action recognition in subway stations based on improved multi-algorithm fusion
- 铁道科学与工程学报 2023年20卷第11期页码：4096-4106
- 作者机构：
  
  1.广西大学机械工程学院，广西南宁 530004
- 作者简介：
  
  贺德强(1973—)，男，湖南桃江人，教授，博士，从事列车故障诊断与智能维护、列车优化控制研究；E-mail：hdqlqy@gxu.edu.cn
- 基金信息：
  
  国家自然科学基金资助项目(U22A2053);广西创新驱动发展专项(桂科AA20302010);广西研究生教育创新计划资助项目(YCBZ2022043)
- DOI：10.19713/j.cnki.43-1423/u.T20230047
  中图分类号：
扫描看全文
章宇翔,李先旺,贺德强等.基于改进的多算法融合地铁站内乘客行为识别[J].铁道科学与工程学报,2023,20(11):4096-4106.

ZHANG Yuxiang,LI Xianwang,HE Deqiang,et al.Passenger action recognition in subway stations based on improved multi-algorithm fusion[J].Journal of Railway Science and Engineering,2023,20(11):4096-4106.
章宇翔,李先旺,贺德强等.基于改进的多算法融合地铁站内乘客行为识别[J].铁道科学与工程学报,2023,20(11):4096-4106. DOI： 10.19713/j.cnki.43-1423/u.T20230047.

ZHANG Yuxiang,LI Xianwang,HE Deqiang,et al.Passenger action recognition in subway stations based on improved multi-algorithm fusion[J].Journal of Railway Science and Engineering,2023,20(11):4096-4106. DOI： 10.19713/j.cnki.43-1423/u.T20230047.

摘要

乘客的行为识别在保障乘客安全方面发挥着重要作用，它能提高地铁站对乘客安全的管理能力。然而，由于地铁车站内乘客人数众多，在出现照明变化和人员遮挡时会严重影响识别的准确率。为了解决准确率低的问题，在时空图卷积神经网络(STGCN)的基础上结合有效通道注意力网络(ECANet)，加强了不同节点的连接，提出一种STEGCN节点注意力算法。此外，为了进一步提高准确率，采用双流结构，更进一步提出一种2s-STEGCN算法。应用Alphapose框架，结合YOLOv5_m目标检测算法、SPPE单人姿态估计算法和2s-STEGCN算法，提出一种改进的多算法融合行为识别方法用于地铁站内乘客的行为识别。首先，利用YOLOv5_m对乘客进行框定；然后，通过SPPE对框定的乘客进行骨骼关键点提取；最后，将提取到的骨骼关键点以坐标的形式输入2s-STEGCN，完成乘客的行为识别。为了验证2s-STEGCN算法的有效性，使用南宁地铁1号线的客流数据集分别在单人场景与多人场景下开展实验。实验结果表明：在损失值方面，2s-STEGCN具有最低的损失值，它的损失值仅为0.244，比STGCN的损失值低约0.025，这表明了2s-STEGCN具有更强的模型构建能力。在准确率方面，单人场景下的2s-STEGCN拥有最高的准确率，它的准确率达到96.13%，比STGCN高3.69%。此外，2s-STEGCN的准确率在多人场景下也有明显提升。该实验结果可为地铁乘客行为识别提供参考和理论支持。

Abstract

Passenger action recognition plays a vital role in safeguarding passenger safety, which improves the capability of managing passenger safety in subway stations. However, the accuracy of the passenger action recognition is seriously affected by the large number of passengers in the subway stations when there are lighting changes and obstructions. Thus, to solve this problem, a STEGCN algorithm was proposed based on a spatio-temporal graphic convolutional neural network (STGCN) combined with an effective channel attention network (ECANet), with enhanced connectivity of different nodes. In addition, an algorithm of 2s-STEGCN was further proposed by using a dual-stream structure to achieve further improvement in the accuracy of passenger action recognition. Specifically, the Alphapose framework was applied in combining the YOLOv5_m object detection algorithm, the single-person pose estimation (SPPE) algorithm and the 2s-STEGCN algorithm to present an improved multi-algorithm fusion action recognition method for the passenger action recognition in subway stations. Firstly, the passenger was framed using the YOLOv5_m algorithm. Secondly, the skeletal key points of the framed passenger were detected by the SPPE algorithm. Finally, the detected skeletal key points were input into the 2s-STEGCN algorithm as coordinates to realize passenger action recognition. To verify the validity of the 2s-STEGCN algorithm, experiments were conducted using the passenger flow dataset of Nanning Rail Transit Line 1 in single-person scenarios and multi-person scenarios, respectively. The results show that the 2s-STEGCN algorithm has the lowest loss value, which is only 0.244, lower than that of the STGCN algorithm by about 0.025. The results indicate that the 2s-STEGCN algorithm has a stronger model-building ability. Regarding the accuracy rate, the 2s-STEGCN algorithm possesses the highest accuracy rate in single-person scenarios, which reaches 96.13%, 3.69% higher than that of the STGCN algorithm. Additionally, the accuracy rate of the 2s-STEGCN algorithm in multi-person scenarios is also significantly improved. Therefore, the results of this experiment can provide reference and theoretical support for subway passenger action recognition.

关键词

行为识别时空图卷积目标检测姿态估计

Keywords

action recognitionspatio-temporal graph convolutionalobject detectionpose estimation

references

HE Deqiang, LIU Chenyu, JIN Zhenzhen, et al. Fault diagnosis of flywheel bearing based on parameter optimization variational mode decomposition energy entropy and deep learning[J]. Energy, 2022, 239: 122108.

肖红升, 贺德强, 杨严杰, 等. 基于可靠度的列车多部件预防性机会维修策略研究[J]. 铁道科学与工程学报, 2019, 16(4): 1033-1040.

XIAO Hongsheng, HE Deqiang, YANG Yanjie, et al. Research on the preventive opportunistic maintenance strategy of train multi-components based on reliability[J]. Journal of Railway Science and Engineering, 2019, 16(4): 1033-1040.

WANG Heng, SCHMID C. Action recognition with improved trajectories[C]// 2013 IEEE International Conference on Computer Vision. December 1-8, 2013, Sydney, NSW, Australia. IEEE, 2014: 3551-3558.

周航, 刘於锡, 龚越, 等. 基于密集轨迹和光流二值化图的行为识别算法[J]. 计算机工程与应用, 2022, 58(20): 174-180.

ZHOU Hang, LIU Yuxi, GONG Yue, et al. Action recognition algorithm based on dense trajectories and optical flow binarization image[J]. Computer Engineering and Applications, 2022, 58(20): 174-180.

LIU Jiaheng, GUO Jinyang, XU Dong. APSNet: toward adaptive point sampling for efficient 3D action recognition[J]. IEEE Transactions on Image Processing, 2022, 31: 5287-5302.

吕亚兰, 安建伟. 基于特征融合双流网络的人体行为识别[J]. 电子测量技术, 2020, 43(20): 121-126.

LÜ Yalan, AN Jianwei. Human behavior recognition based on feature fusion double-stream network[J]. Electronic Measurement Technology, 2020, 43(20): 121-126.

WANG Zhongwen, LU Haozhu, JIN Junlan, et al. Human action recognition based on improved two-stream convolution network[J]. Applied Sciences, 2022, 12(12): 5784.

DU Yong, WANG Wei, WANG Liang. Hierarchical recurrent neural network for skeleton based action recognition[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015: 1110-1118.

LIU Jun, SHAHROUDY A, XU Dong, et al. Spatio-temporal LSTM with trust gates for 3D human action recognition[C]// European Conference on Computer Vision. Cham: Springer, 2016: 816-833.

ZHANG Pengfei, LAN Cuiling, XING Junliang, et al. View adaptive recurrent neural networks for high performance human action recognition from skeleton data[C]// 2017 IEEE International Conference on Computer Vision (ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017: 2136-2145.

LI Shuai, LI Wanqing, COOK C, et al. Independently recurrent neural network (IndRNN): building A longer and deeper RNN[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 5457-5466.

KE Qiuhong, BENNAMOUN M, AN Senjian, et al. A new representation of skeleton sequences for 3D action recognition[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 4570-4579.

KIM T S, REITER A. Interpretable 3D human action analysis with temporal convolutional networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 1623-1631.

LI Bo, DAI Yuchao, CHENG Xuelian, et al. Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep CNN[C]// 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). July 10-14, 2017, Hong Kong, China. IEEE, 2017: 601-604.

LIU Hong, TU Juanhui, LIU Mengyuan. Two-stream 3D convolutional neural network for skeleton-based action recognition[EB/OL]. 2017: arXiv: 1705.08106. https://arxiv.org/abs/1705.08106https://arxiv.org/abs/1705.08106.

YAN Sijie, XIONG Yuanjun, LIN Dahua. Spatial temporal graph convolutional networks for skeleton-based action recognition[EB/OL]. 2018: arXiv: 1801.07455. https://arxiv.org/abs/1801.07455https://arxiv.org/abs/1801.07455.

SHI Lei, ZHANG Yifan, CHENG Jian, et al. Two-stream adaptive graph convolutional networks for skeleton-based action recognition[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2020: 12018-12027.

BAI Jing, YU Wentao, XIAO Zhu, et al. Two-stream spatial-temporal graph convolutional networks for driver drowsiness detection[J]. IEEE Transactions on Cybernetics, 2022, 52(12): 13821-13833.

FANG Haoshu, XIE Shuqin, TAI Y W, et al. RMPE: regional multi-person pose estimation[C]// 2017 IEEE International Conference on Computer Vision (ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017: 2353-2362.

WANG Qilong. WU Banggu, ZHU Pengfei, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 11531-11539.

WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020: 11531-11539.

HU Jie, SHEN Li, ALBANIE S, et al. Squeeze-and-excitation networks[EB/OL]. 2017: arXiv: 1709.01507. https://arxiv.org/abs/1709.01507https://arxiv.org/abs/1709.01507.

SHI Lei, ZHANG Yifan, CHENG Jian, et al. Two-stream adaptive graph convolutional networks for skeleton-based action recognition[EB/OL]. 2018: arXiv: 1805.07694. https://arxiv.org/abs/1805.07694https://arxiv.org/abs/1805.07694.

ZHOU Shuren, CHEN Le, SUGUMARAN V. Hidden two-stream collaborative learning network for action recognition[J]. Computers Materials & Continua, 2020, 63(3): 1545-1561.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于计算机视觉的轨道交通站内客流识别与预测

基于乘客行为识别的地铁站照明控制方法

基于图像增强与深度学习的钢轨表面缺陷检测