Video-based estimation of human physiological signals

<p dir="ltr">The estimation of human physiological signals such as heart rate, blood pressure, skin temperature, respiration rate, and blood glucose is crucial to the detection and monitoring of various diseases. Physiological signal estimation is typically performed utilizing contact-based devices, such as pulse oximeters (for measurement of heart rate and respiration rate), contact thermometers (for measurement of temperature), or glucometers (for measurement of blood glucose), which focus on explicit signal measurement through device-subject contact. Contact-based devices are often cost-prohibitive and find limited utilization in non-clinical settings due to the risk of infection. Non-contact-based approaches utilize data from modalities such as images, videos, audio clips or radio signals to estimate the vital signs of people, in place of explicit signal measurement through device-subject contact. Non?contact-based vital sign estimation approaches reduce the risk of infection due to the lack of contact between the device and the subject. Of the afore?mentioned modalities, the video modality is the preferred modality for remote estimation of vital signs, due to 1) the optical nature of changes in physiological signals and 2) the ubiquity of video recording devices. </p><p dir="ltr">Despite these advantages, video-based physiological signal estimation faces critical challenges. The non-contact nature makes increases the susceptibility to environmental and demographic heterogeneity such as lighting variations and subject variability which can substantially reduce robustness and generalizability. This thesis investigates the impact of such heterogeneity on the robustness of physiological signal estimation and introduces dataset-driven methodologies designed to mitigate these challenges. Prior research in video-based physiological monitoring has predominantly focused on estimating heart rate and respiration rate using facial video data. In contrast, the exploration of video-based methods for estimating other clinically significant signals, such as skin temperature, blood pressure, and blood glucose, remains limited. To address this gap, this work not only advances dataset-driven strategies for managing environmental and demographic heterogeneity, but also proposes novel estimation algorithms. These algorithmic contributions aim to complement data-driven approaches, thereby enhancing the generalizability and robustness of video-based physiological signal estimation across diverse and challenging data distributions. </p><p dir="ltr">The first chapter of this thesis explores the impact of the aforementioned environmental and demographic heterogeneity utilizing existing state-of-the-art video-based heart rate estimation approaches on a dataset collected under challenging conditions. This study found that existing video-based approaches for physiological signal estimation exhibit high variance when tested under challenging, previously unseen conditions such as lighting variations and subject variability. The result obtained reveals that existing video-based physiological signal estimation approaches are predisposed to perform optimally on less challenging, previously encountered data distributions. To this end, the REVIT dataset is developed, consisting of human subject videos and associated ground truth physiological sign measurements (such as heart rate) collected in multiple outside-the-lab settings for the remote estimation of human physiological signs. The REVIT dataset is uniquely designed to capture a wide range of environmental heterogeneity (such as variable lighting and background) encountered in outside-the-lab settings, thereby enabling the development of more robust physiological signal estimation approaches. Through statistical hypothesis testing, it is demonstrated that the use of the REVIT dataset results in a significant improvement in the robustness of existing video-based physiological signal estimation approaches. Further?more, this work highlights the impact of existing data distributions toward controlled laboratory conditions, underscoring their limitations in real world applicability. </p><p dir="ltr">Building on the analysis of environmental and demographic heterogeneity and the development of data-driven strategies to mitigate its effects, the subsequent chapters in this work shift focus toward algorithmic approaches for enhancing the robustness of video-based physiological signal estimation. In particular, the design of robust estimation methods for physiological signals beyond heart rate and respiration rate is explored, namely skin temperature, blood pressure, and blood glucose. Across these approaches, emphasis is placed on ensuring reliable performance under heterogeneous environmental and demographic conditions, thereby advancing algorithmic strategies as a complementary pathway to improve the robustness of video-based physiolog?ical estimation. </p><p dir="ltr">The second chapter of this thesis develops V-TEMP, a video-based approach to detect elevated skin temperature. V-TEMP leverages the correlation between skin temperature and the angular reflectance distribution of light from the skin to empirically differentiate between skin at an elevated temperature and skin at non-elevated temperature. The uniqueness of this correlation is demonstrated through 1) revealing the existence of a difference in the angular reflectance distribution of light from skin-like and non-skin-like material and 2) exploring the consistency of the angular reflectance distribution of light in materials exhibiting optical properties similar to human skin. The robustness of V-TEMP in challenging conditions is demonstrated by evaluating the efficacy of elevated skin temperature detection on subject videos recorded in 1) laboratory controlled environments and 2) outside-the?lab environments. </p><p dir="ltr">The third chapter of this thesis develops V-BPE, a video-based approach that estimates an individual’s blood pressure. V-BPE leverages the time difference of the blood pulse arrival at two different locations in the body (Pulse Transit Time) and the inverse relation between the blood pressure and the velocity of blood pressure pulse propagation in the artery to analytically estimate the blood pressure. Statistical hypothesis testing, reveals that Pulse Transit Time-based approaches to estimate blood pressure require knowledge of subject specific blood vessel parameters, such as the length of the blood vessel. To address this, V-BPE utilizes a combination of computer vision techniques and demographic information (such as the height and the weight of the subject) to capture and incorporate the aforementioned subject specific blood vessel parameters into the estimation of blood pressure. The robustness of V-BPE in challenging conditions is demonstrated by evaluating the efficacy of blood pressure estimation in demographically heterogeneous, outside-the-lab conditions. </p><p dir="ltr">The fourth chapter of this thesis develops V-BGE, a video-based approach that estimates an individual’s blood glucose level. V-BGE estimates blood glucose by analytically leveraging the differential absorption and scattering characteristics of visible (RGB) and near-infrared (NIR) radiation by blood and tissue. V-BGE is grounded in the Beer-Lambert law, which relates substance concentration to light absorption, and is extended to account for physiological factors such as tissue composition, blood scattering, and atmospheric effects. Given that blood glucose exhibits significant absorption primarily in the NIR region (due to dominant water absorption in the RGB range), V-BGE utilizes combined RGB and NIR data to compute absorbance and estimate physiological parameters (e.g., hematocrit) that influence light scattering within biological tissue. The robustness of V-BGE in challenging conditions is demonstrated by evaluating through a case study focused on blood glucose estimation among critical populations in demographically heterogeneous, outside-the-lab conditions. </p><p dir="ltr">Together, these contributions advance the field of non-contact physiological monitoring by addressing three critical signals, namely skin temperature, blood pressure, and blood glucose, which remain largely unaddressed in prior video-based research. By coupling dataset-driven strategies with algorithmic innovations, this thesis demonstrates that robust and generalizable solutions can be achieved even in heterogeneous environments. Developing such algo?rithms for diverse physiological signals is essential for broadening the scope of remote health monitoring, enabling non-invasive, and scalable solutions that address critical gaps in healthcare accessibility and disease management.</p>