Plethysmography (PPG) extracts blood flow information, or pleth, using a contact measurement of skin color changes due to blood flow. Today, various forms of this technology are utilized for health and wellness measurements. Devices placed on fingers, common in most doctors’ office visits, and smart wristwatch devices, like the Apple Watch, are used to obtain physiological metrics such as heart rate and blood oxygenation. While wrist smart watches are becoming more common, they are still a luxury device and are inaccessible to many people. Furthermore, these devices take measurements without context. Heart rate, for example, can vary greatly upon performing different activities like eating, talking, or sleeping. Without context, the value of the measurement can be minimized.
Development of rPPG
In the early 2010s, a novel way to measure blood flow information was developed at the Massachusetts Institute of Technology using a camera, called remote plethysmography (rPPG) [1]. This can offer several benefits over the previously mentioned PPG devices. Today, cameras can be found on almost every smartphone, laptop, tablet or other devices. Utilizing these cameras, the activity of a person is seen to provide context for the rPPG measurement.
There have been numerous improvements in rPPG since its original publication in 2012. Recently, deep learning-based rPPG methods have been shown to measure blood flow extremely well with excellent correlation to finger-based PPG signals [2,3]. However, these deep learning approaches can be difficult to comprehend; posing challenges to receiving approval from regulatory bodies and trust-building among physicians.
Comparison: Smart Spectra vs Wrist-Based PPG
Below, we show examples of Presage Technologies’ rPPG measurement algorithm, Smart Spectra. We compare its ability to measure blood flow pleth accurately in a dynamic interview scenario with a wrist-based PPG sensor. Smart Spectra measures pleth with entirely explainable algorithms and no deep learning for signal extraction, other than to identify the face location.
To demonstrate the ability to accurately measure pleth, we utilized the UBFC-PHYS dataset [4]. The dataset is composed of 35 frames per second videos of subjects and PPG traces from a wrist-based PPG sensor (Empatica E4) worn by the subject in an interview-type scenario.
Fig. 1(A) shows the pleth comparison between the wrist sensor and Smart Spectra from the recording in Video 1. In this video, the subject was asked to count down numbers. The pleth from Smart Spectra and the wrist sensor was found to be 84% correlated. The video was time-shifted to eliminate the pulse arrival time difference between the face and wrist.
To further analyze the correlation between the two sensors, we investigated the measured heart rate vs time. Heart rate can be computed from pleth by looking at how many beats or oscillations are in the pleth per min. Fig. 1(B) shows the spectrogram, or measured heart rate vs time, between Smart Spectra (Presage rPPG) and the wrist-based sensor. The spectrogram was computed using a sliding 5-second sampling window. The spectrograms between Smart Spectra and the wrist sensor are very well matched with a correlation of 97% and both measuring a consistent heart rate of ~73 beats per minute (BPM).
Video 1
Fig. 1 (A) Pleth comparison from Presage Video rPPG (Smart Spectra) and a wrist-based PPG sensor in Video 1, a number counting down scenario. (B) Spectrograms from Presage Video rPPG (Smart Spectra) and wrist-based PPG computed with a 5-second sliding window. The darker color indicates a higher heart rate signal whereas the red indicates a low heart rate signal.
The Beauty of Smart Spectra Pleth Shape
The precise measurement of pleth is crucial to computing additional health metrics, such as heart rate variability and blood pressure [5,6]. An example of a pleth pulse measured by Smart Spectra is shown in Fig. 2. This pulse is from the black dotted box in Fig. 1(A). The pulse shape has excellent agreement with the classical PPG signal shape. In addition, several features can be measured; proving to be valuable inputs to blood pressure estimation from PPG pleth.
Smart Spectra Motion Robustness vs Wrist-Based Sensor
Wrist-based sensors do not always provide clean PPG signals and can be noisy. If the sensor loses contact with skin, or there is a lot of motion in the arm, it can result in many artifacts in the PPG signal. To overcome this, wrist PPG manufacturers perform an abundant amount of time-averaging to measure heart rates. This is why sometimes a smart watch may have blackouts, or fail to show the output of a heart rate that is varying significantly in a short span of time [7].
The noise pollution in wrist-based sensors was evident in another video analyzed in the UBFC-PHYS dataset. In this video, Video 2, the subject was participating in a job interview-type scenario. The resulting pleths and spectrograms between Smart Spectra and the wrist-based sensor can be seen in Fig. 3. In Fig. 3(A), the noise in the wrist PPG sensor is evident with large spikes and loss of classical PPG shape in several of the pulses. Because of this, heart rate variability or blood pressure measurements are unlikely from this wrist PPG data.
The Smart Spectra pleth had much cleaner pulse shapes and no evidence of erratic spikes. This highlights Smart Spectra’s ability to deal with motion. Upon examination of the spectrograms in Fig. 3(B), the underperformance of the wrist-based sensor to measure heart rate is clearly evident. Smart Spectra produces a clean measurement of heart rate showing the variance in heart rate from ~86 BPM to ~115 BPM. The wrist PPG sensor spectrogram is extremely noisy making heart rate measurements in this data unlikely.
Video 2
Fig. 3 (A) Pleth comparison from Presage Video rPPG (Smart Spectra) and a wrist based PPG sensor in Video 2, a job interview scenario. (B) Spectrograms from Presage Video rPPG (Smart Spectra) and wrist based PPG computed with a 5 second sliding window. The darker color indicates higher heart rate signal whereas the red indicates low heart rate signal.
Smart Spectra Advantage
The two videos analyzed show that Presage Technologies’ Smart Spectra is able to measure pleth accurately even in scenarios where wrist-based sensor data is corrupted with noise. rPPG also has context in its measurement. If the patient’s face is no longer visible, erroneous pleth is not produced by Smart Spectra. Contact-based PPG sensors do not have this luxury and can be presented with many challenges in accurately measuring pleth when skin contact is lost or large degrees of motion are present.
Smart Spectra’s clean pleth measurement shows how accurate rPPG can truly be and provides an avenue for physiological measurement to anyone with a smartphone. This is only the beginning of Smart Spectra’s physiological measurement abilities with more features such as blood pressure coming soon.
For more information, contact Presage Technologies. We would love to hear about your intended use case.
References
[1] Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., & Freeman, W. (2012). Eulerian Video Magnification for Revealing Subtle Changes in the World.
[2] Nowara, E., McDuff, D., & Veeraraghavan, A. (2020). The Benefit of Distraction: Denoising Remote Vitals Measurements using Inverse Attention. 1–14. http://arxiv.org/abs/2010.07770
[3] Speth, J., Vance, N., Flynn, P., Bowyer, K., & Czajka, A. (2021). Unifying frame rate and temporal dilations for improved remote pulse detection. Computer Vision and Image Understanding, 210(July), 103246. https://doi.org/10.1016/j.cviu.2021.103246
[4] R. Meziati Sabour, Y. Benezeth, P. De Oliveira, J. Chappé, F. Yang. "UBFC-Phys: A Multimodal Database For Psychophysiological Studies Of Social Stress", IEEE Transactions on Affective Computing, 2021.
[5]Hoog Antink, C., Mai, Y., Peltokangas, M., Leonhardt, S., Oksala, N., & Vehkaoja, A. (2021). Accuracy of heart rate variability estimated with reflective wrist-PPG in elderly vascular patients. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-87489-0
[6] Mohebbian, M. R., Dinh, A., Wahid, K., & Alam, M. S. (2020). Blind, Cuff-less, Calibration-Free and Continuous Blood Pressure Estimation using Optimized Inductive Group Method of Data Handling. Biomedical Signal Processing and Control, 57, 101682. https://doi.org/10.1016/j.bspc.2019.101682
[7] Admin. “Can Fitbit and Apple Watch Do What Dhyana Does?: Dhyana.” Meditation Device, Smart Dhyana, 1 Dec. 2020, https://www.smartdhyana.com/blog/can-fitbit-and-apple-watch-do-what-dhyana-does/.