Estimating heart rate and rhythm via 3D motion tracking in depth video

Cheng Yang, Gene Cheung, Vladimir Stankovic

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Low-cost depth sensors, such as Microsoft Kinect, have potential for non-intrusive, non-contact health monitoring that is robust to ambient lighting conditions. However, captured depth images typically suer from low bit-depth and high acquisition noise, and hence processing them to estimate biometrics is dicult. In this paper, we propose to capture depth video of a human subject using Kinect 2.0 to estimate his/her heart rate and rhythm (regularity); as blood is pumped from the heart to circulate through the head, tiny oscillatory head motion due to Newtonian mechanics can be detected for periodicity analysis. Specifically, we first restore a captured depth video via a joint bit-depth
enhancement / denoising procedure, using a graph-signal smoothness prior for regularization. Second, we track an automatically detected head region throughout the depth video to deduce 3D motion vectors. The detected vectors are fed back to the depth restoration module in a loop to ensure that the motion information in two modules are consistent, improving performance of both restoration and motion tracking in the process. Third, the computed 3D motion vectors are projected onto its principal component for 1D signal analysis, composed of trend removal, band-pass filtering, and wavelet-based motion denoising. Finally, the heart rate is estimated via Welch power spectrum analysis, and the heart rhythm is computed via peak detection. Experimental results
show accurate estimation of the heart rate and rhythm using our proposed algorithm as compared to rate and rhythm estimated by a portable oximeter.
LanguageEnglish
Pages1625-1636
Number of pages13
JournalIEEE Transactions on Multimedia
Volume19
Issue number7
Early online date20 Feb 2017
DOIs
Publication statusPublished - 1 Jul 2017

Fingerprint

Restoration
Oximeters
Signal analysis
Biometrics
Power spectrum
Spectrum analysis
Mechanics
Blood
Lighting
Health
Monitoring
Sensors
Processing
Costs

Keywords

  • 3D motion tracking
  • depth sensor
  • motion vectors
  • heart rate
  • heart rhythm
  • non-invasive heart monitoring

Cite this

@article{eca84d7bba74413396f7323ea263117d,
title = "Estimating heart rate and rhythm via 3D motion tracking in depth video",
abstract = "Low-cost depth sensors, such as Microsoft Kinect, have potential for non-intrusive, non-contact health monitoring that is robust to ambient lighting conditions. However, captured depth images typically suer from low bit-depth and high acquisition noise, and hence processing them to estimate biometrics is dicult. In this paper, we propose to capture depth video of a human subject using Kinect 2.0 to estimate his/her heart rate and rhythm (regularity); as blood is pumped from the heart to circulate through the head, tiny oscillatory head motion due to Newtonian mechanics can be detected for periodicity analysis. Specifically, we first restore a captured depth video via a joint bit-depthenhancement / denoising procedure, using a graph-signal smoothness prior for regularization. Second, we track an automatically detected head region throughout the depth video to deduce 3D motion vectors. The detected vectors are fed back to the depth restoration module in a loop to ensure that the motion information in two modules are consistent, improving performance of both restoration and motion tracking in the process. Third, the computed 3D motion vectors are projected onto its principal component for 1D signal analysis, composed of trend removal, band-pass filtering, and wavelet-based motion denoising. Finally, the heart rate is estimated via Welch power spectrum analysis, and the heart rhythm is computed via peak detection. Experimental resultsshow accurate estimation of the heart rate and rhythm using our proposed algorithm as compared to rate and rhythm estimated by a portable oximeter.",
keywords = "3D motion tracking, depth sensor, motion vectors, heart rate , heart rhythm, non-invasive heart monitoring",
author = "Cheng Yang and Gene Cheung and Vladimir Stankovic",
note = "(c) 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.",
year = "2017",
month = "7",
day = "1",
doi = "10.1109/TMM.2017.2672198",
language = "English",
volume = "19",
pages = "1625--1636",
journal = "IEEE Transactions on Multimedia",
issn = "1520-9210",
number = "7",

}

Estimating heart rate and rhythm via 3D motion tracking in depth video. / Yang, Cheng; Cheung, Gene; Stankovic, Vladimir.

In: IEEE Transactions on Multimedia, Vol. 19, No. 7, 01.07.2017, p. 1625-1636.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Estimating heart rate and rhythm via 3D motion tracking in depth video

AU - Yang, Cheng

AU - Cheung, Gene

AU - Stankovic, Vladimir

N1 - (c) 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Low-cost depth sensors, such as Microsoft Kinect, have potential for non-intrusive, non-contact health monitoring that is robust to ambient lighting conditions. However, captured depth images typically suer from low bit-depth and high acquisition noise, and hence processing them to estimate biometrics is dicult. In this paper, we propose to capture depth video of a human subject using Kinect 2.0 to estimate his/her heart rate and rhythm (regularity); as blood is pumped from the heart to circulate through the head, tiny oscillatory head motion due to Newtonian mechanics can be detected for periodicity analysis. Specifically, we first restore a captured depth video via a joint bit-depthenhancement / denoising procedure, using a graph-signal smoothness prior for regularization. Second, we track an automatically detected head region throughout the depth video to deduce 3D motion vectors. The detected vectors are fed back to the depth restoration module in a loop to ensure that the motion information in two modules are consistent, improving performance of both restoration and motion tracking in the process. Third, the computed 3D motion vectors are projected onto its principal component for 1D signal analysis, composed of trend removal, band-pass filtering, and wavelet-based motion denoising. Finally, the heart rate is estimated via Welch power spectrum analysis, and the heart rhythm is computed via peak detection. Experimental resultsshow accurate estimation of the heart rate and rhythm using our proposed algorithm as compared to rate and rhythm estimated by a portable oximeter.

AB - Low-cost depth sensors, such as Microsoft Kinect, have potential for non-intrusive, non-contact health monitoring that is robust to ambient lighting conditions. However, captured depth images typically suer from low bit-depth and high acquisition noise, and hence processing them to estimate biometrics is dicult. In this paper, we propose to capture depth video of a human subject using Kinect 2.0 to estimate his/her heart rate and rhythm (regularity); as blood is pumped from the heart to circulate through the head, tiny oscillatory head motion due to Newtonian mechanics can be detected for periodicity analysis. Specifically, we first restore a captured depth video via a joint bit-depthenhancement / denoising procedure, using a graph-signal smoothness prior for regularization. Second, we track an automatically detected head region throughout the depth video to deduce 3D motion vectors. The detected vectors are fed back to the depth restoration module in a loop to ensure that the motion information in two modules are consistent, improving performance of both restoration and motion tracking in the process. Third, the computed 3D motion vectors are projected onto its principal component for 1D signal analysis, composed of trend removal, band-pass filtering, and wavelet-based motion denoising. Finally, the heart rate is estimated via Welch power spectrum analysis, and the heart rhythm is computed via peak detection. Experimental resultsshow accurate estimation of the heart rate and rhythm using our proposed algorithm as compared to rate and rhythm estimated by a portable oximeter.

KW - 3D motion tracking

KW - depth sensor

KW - motion vectors

KW - heart rate

KW - heart rhythm

KW - non-invasive heart monitoring

U2 - 10.1109/TMM.2017.2672198

DO - 10.1109/TMM.2017.2672198

M3 - Article

VL - 19

SP - 1625

EP - 1636

JO - IEEE Transactions on Multimedia

T2 - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

IS - 7

ER -