基于RDSNet的毫米波雷达人体跌倒检测方法

元志安 周笑宇 刘心溥 卢大威 邓彬 马燕新

元志安, 周笑宇, 刘心溥, 等. 基于RDSNet的毫米波雷达人体跌倒检测方法[J]. 雷达学报, 2021, 10(4): 656–664. doi: 10.12000/JR21015
引用本文: 元志安, 周笑宇, 刘心溥, 等. 基于RDSNet的毫米波雷达人体跌倒检测方法[J]. 雷达学报, 2021, 10(4): 656–664. doi: 10.12000/JR21015
YUAN Zhian, ZHOU Xiaoyu, LIU Xinpu, et al. Human fall detection method using millimeter-wave radar based on RDSNet[J]. Journal of Radars, 2021, 10(4): 656–664. doi: 10.12000/JR21015
Citation: YUAN Zhian, ZHOU Xiaoyu, LIU Xinpu, et al. Human fall detection method using millimeter-wave radar based on RDSNet[J]. Journal of Radars, 2021, 10(4): 656–664. doi: 10.12000/JR21015

基于RDSNet的毫米波雷达人体跌倒检测方法

doi: 10.12000/JR21015
基金项目: 国家自然科学基金(61871386),湖南省杰出青年基金(2019JJ20022)
详细信息
    作者简介:

    元志安(1997–),男,山东潍坊人,国防科技大学电子科学学院硕士研究生,主要研究方向为智能信息感知与处理

    周笑宇(1997–),男,重庆北碚人,国防科技大学电子科学学院硕士研究生,主要研究方向为三维点云目标跟踪

    刘心溥(1997–),男,河北石家庄人,国防科技大学电子科学学院硕士研究生,主要研究方向为三维点云补全

    卢大威(1981–),男,湖北安陆人,国防科技大学电子科学学院讲师,主要研究方向为雷达系统仿真、雷达信号处理、随机有限集滤波与扩展目标跟踪

    邓彬:邓 彬(1981–),男,山东邹城人,国防科技大学电子科学学院副研究员,主要研究方向为合成孔径雷达、太赫兹雷达微动与成像

    马燕新(1989–),男,江西永新人,国防科技大学气象海洋学院讲师,主要研究方向为三维计算机视觉

    通讯作者:

    卢大威 davidloo.nudt@gmail.com

  • 责任主编:黄磊 Corresponding Editor: HUANG Lei
  • 中图分类号: TN957.51

Human Fall Detection Method Using Millimeter-wave Radar Based on RDSNet

Funds: The National Natural Science Foundation of China (61871386), The Natural Science Fund for Distinguished Young Scholars of Hunan Province (2019JJ20022)
More Information
  • 摘要: 随着人口老龄化的到来,跌倒检测逐渐成为研究热点。针对基于毫米波雷达的人体跌倒检测应用,该文提出了一种融合卷积神经网络和长短时记忆网络的距离多普勒热图序列检测网络(RDSNet)模型。首先通过卷积神经网络对距离多普勒热图进行特征提取得到特征向量,然后将动态序列对应的特征向量序列依次输入长短时记忆网络,进而学习得到热图序列的时间相关性信息,最后通过分类器网络得到检测结果。利用毫米波雷达采集了不同对象的多种人体动作,构建了距离多普勒热图数据集。对比试验表明,所提出的RDSNet网络模型检测准确率可达到96.67%,计算时延小于50 ms,而且具有良好的泛化能力,可为跌倒检测和人体姿态识别提供新的技术思路。

     

  • 图  1  跌倒检测系统构成

    Figure  1.  The pipeline of fall detection

    图  2  LSTM网络单元

    Figure  2.  The unit of LSTM network

    图  3  RDSNet网络结构

    Figure  3.  The structure of RDSNet

    图  4  实验流程

    Figure  4.  The process of experiments

    图  5  实验场景和雷达视角示意图

    Figure  5.  Experimental scene and radar perspective diagram

    图  6  4种动作分类RDSNet混淆矩阵

    Figure  6.  RDSNet confusion matrix for 4 types of action classification

    表  1  动作设计及距离多普勒热图序列

    Table  1.   Motion design and range Doppler heat map sequence

    动作距离多普勒热图序列
    跌倒
    挥手
    起立
    静止
    走动
    翻身
    下载: 导出CSV

    表  2  不同检测方法结果对比

    Table  2.   Comparative experimental results of different detection methods

    分类方法模型传感器动作种类检测准确率(%)实时
    穿戴式数据融合检测法[3]加速度计+RGBD相机499.00
    基于CNN的手机跌倒检测模型[4]加速度计+陀螺仪+线性加速度计298.51
    基于RNN的可穿戴跌倒检测器[21]加速度计296.70
    非穿戴式基于深度学习的多患者行为检测[13]毫米波雷达684.49
    基于RGB-D跌倒检测算法[7]RGBD相机398.00
    复杂场景下的跌倒检测算法[8]RGBD相机289.80
    基于超宽带雷达的跌倒检测算法[12]超宽带雷达(3个)290.00
    智能家居超宽带雷达跌倒检测[11]超宽带雷达297.50
    老年人智能检测系统[9]WiFi684.60
    RDSNet毫米波雷达298.33是(<50 ms)
    496.67
    693.33
    下载: 导出CSV

    表  3  相同传感器下不同网络结构的结果对比

    Table  3.   The results of different network structures under the same sensor

    方法模型网络结构动作种类检测准确率(%)实时
    基于深度学习的多患者行为检测[13]CNN684.49
    基于LKCNN的跌倒检测算法[14]LKCNN295.24
    基于Bi-SLAM的人体行为连续分类检测[15]Bi-LSTM691.00
    基于Bi-SLAM的多模态人体姿态识别与跌倒检测[16]Bi-LSTM688.90
    本文测试CNN689.00是(<5 ms)
    LSTM676.66是(<35 ms)
    CNN+LSTM298.33是(<50 ms)
    CNN+LSTM693.33
    下载: 导出CSV

    表  4  不同网络结构组合结果对比

    Table  4.   Comparison results of different network structure combinations

    CNN(层)LSTM(层)准确率(%)漏警率(%)虚警率(%)时延(ms)
    1189.0011.0015<38
    2196.673.338<50
    3192.677.336<69
    4167.0033.0026<80
    1291.678.3321>100
    2295.334.6711>100
    3284.0016.0016>100
    4261.3338.6734>100
    下载: 导出CSV
  • [1] 师昉, 李福亮, 张思佳, 等. 中国老年跌倒研究的现状与对策[J]. 中国康复, 2018, 33(3): 246–248. doi: 10.3870/zgkf.2018.03.021

    SHI Fang, LI Fuliang, ZHANG Sijia, et al. The present situation and countermeasures of the study of senile falls in China[J]. Chinese Journal of Rehabilitation, 2018, 33(3): 246–248. doi: 10.3870/zgkf.2018.03.021
    [2] DHARUNGEERAN N and JAFARALI J. Sensors-based wearable systems for monitoring of human movement and falls[J]. International Journal of Modern Trends in Engineering and Science, 2014, 1(3): 64–69.
    [3] GASPARRINI S, CIPPITELLI E, GAMBI E, et al. Proposal and Experimental Evaluation of Fall Detection Solution Based on Wearable and Depth Data Fusion[M]. LOSHKOVSKA S and KOCESKI S. Advances in Intelligent Systems and Computing. Cham: Springer, 2016, 399: 99–108. doi: 10.1007/978-3-319-25733-4_11.
    [4] 吕艳, 张萌, 姜吴昊, 等. 采用卷积神经网络的老年人跌倒检测系统设计[J]. 浙江大学学报: 工学版, 2019, 53(6): 1130–1138. doi: 10.3785/j.issn.1008-973X.2019.06.012

    LV Yan, ZHANG Meng, JIANG Wuhao, et al. Design of elderly fall detection system using CNN[J]. Journal of Zhejiang University:Engineering Science, 2019, 53(6): 1130–1138. doi: 10.3785/j.issn.1008-973X.2019.06.012
    [5] CIPPITELLI E, FIORANELLI F, GAMBI E, et al. Radar and RGB-depth sensors for fall detection: A review[J]. IEEE Sensors Journal, 2017, 17(12): 3585–3604. doi: 10.1109/JSEN.2017.2697077
    [6] ZHANG Jing, LI Wanqing, OGUNBONA P O, et al. RGB-D-based action recognition datasets: A survey[J]. Pattern Recognition, 2016, 60: 86–105. doi: 10.1016/j.patcog.2016.05.019
    [7] ABOBAKR A, HOSSNY M, ABDELKADER H, et al. RGB-D fall detection via deep residual convolutional LSTM networks[C]. 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, 2018: 1–7. doi: 10.1109/DICTA.2018.8615759.
    [8] FENG Qi, GAO Chenqiang, WANG Lan, et al. Spatio-temporal fall event detection in complex scenes using attention guided LSTM[J]. Pattern Recognition Letters, 2020, 130: 242–249. doi: 10.1016/j.patrec.2018.08.031
    [9] BAO Nan, WU Chengyang, LIANG Qiancheng, et al. The intelligent monitoring for the elderly based on WiFi signals[C]. 18th Pacific-Rim Conference on Advances in Multimedia Information Processing, Harbin, China, 2018: 883–892. doi: 10.1007/978-3-319-77380-3_85.
    [10] HU Yuqian, ZHANG Feng, WU Chenshu, et al. A wifi-based passive fall detection system[C]. 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020: 1723–1727. doi: 10.1109/ICASSP40776.2020.9054753.
    [11] MOKHTARI G, ZHANG Q, and FAZLOLLAHI A. Non-wearable UWB sensor to detect falls in smart home environment[C]. 2017 IEEE International Conference on Pervasive Computing and Communications Workshop, Kona, Italy, 2017: 274–278. doi: 10.1109/PERCOMW.2017.7917571.
    [12] MAITRE J, BOUCHARD K, and GABOURY S. Fall detection with UWB radars and CNN-LSTM architecture[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(4): 1273–1283. doi: 10.1109/JBHI.2020.3027967
    [13] JIN Feng, ZHANG Renyuan, SENGUPTA A, et al. Multiple patients behavior detection in real-time using mmwave radar and deep CNNs[C]. 2019 IEEE Radar Conference, Boston, USA, 2019: 1–6.
    [14] WANG Bo, GUO Liang, ZHANG Hao, et al. A millimetre-wave radar-based fall detection method using line kernel convolutional neural network[J]. IEEE Sensors Journal, 2020, 20(22): 13364–13370. doi: 10.1109/JSEN.2020.3006918
    [15] SHRESTHA A, LI Haobo, LE KERNEC J, et al. Continuous human activity classification from FMCW radar with bi-LSTM networks[J]. IEEE Sensors Journal, 2020, 20(22): 13607–13619. doi: 10.1109/JSEN.2020.3006386
    [16] LI Haobo, SHRESTHA A, HEIDARI H, et al. Bi-LSTM network for multimodal continuous human activity recognition and fall detection[J]. IEEE Sensors Journal, 2020, 20(3): 1191–1201. doi: 10.1109/JSEN.2019.2946095
    [17] HOCHREITER S and SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735–1780. doi: 10.1162/neco.1997.9.8.1735
    [18] HUBEL D H and WIESEL T N. Receptive fields of single neurones in the cat’s striate cortex[J]. The Journal of Physiology, 1959, 148(3): 574–591. doi: 10.1113/jphysiol.1959.sp006308
    [19] 田壮壮, 占荣辉, 胡杰民, 等. 基于卷积神经网络的SAR图像目标识别研究[J]. 雷达学报, 2016, 5(3): 320–325. doi: 10.12000/JR16037

    TIAN Zhuangzhuang, ZHAN Ronghui, HU Jiemin, et al. SAR ATR based on convolutional neural network[J]. Journal of Radars, 2016, 5(3): 320–325. doi: 10.12000/JR16037
    [20] 韩昭蓉, 黄廷磊, 任文娟, 等. 基于Bi-LSTM模型的轨迹异常点检测算法[J]. 雷达学报, 2019, 8(1): 36–43. doi: 10.12000/JR18039

    HAN Zhaorong, HUANG Tinglei, REN Wenjuan, et al. Trajectory outlier detection algorithm based on Bi-LSTM model[J]. Journal of Radars, 2019, 8(1): 36–43. doi: 10.12000/JR18039
    [21] LUNA-PEREJÓN F, DOMÍNGUEZ-MORALES M J, and CIVIT-BALCELLS A. Wearable fall detector using recurrent neural networks[J]. Sensors, 2019, 19(22): 4885. doi: 10.3390/s19224885
  • 加载中
图(6) / 表(4)
计量
  • 文章访问数:  393
  • HTML全文浏览量:  47
  • PDF下载量:  141
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-02-26
  • 修回日期:  2021-07-13
  • 网络出版日期:  2021-07-26
  • 刊出日期:  2021-08-28

目录

    /

    返回文章
    返回