Advancing Multimodal Sensing and Robotic Interfaces for Chronic Care

<p dir="ltr">The healthcare system prioritizes reactive care for acute illnesses, often overlooking the ongoing needs of individuals with chronic conditions that require long-term management and personalized care. Addressing this gap through technology can empower patients to better manage their conditions, greatly enhancing quality of life and independence. Multi modal sensing, incorporating inertial, acoustic, and vision-based sensors, within mobile form factors like wearables and probes, has the ability to enable continuous, quantitative monitoring of physiological and behavioral changes, while also serving as interfaces for individuals to manage and control aspects of their care. Building on this concept, this thesis intro duces sensing technologies, devices, and algorithms aimed at improving the management of chronic conditions. </p><p dir="ltr">Focusing initially on dermatological conditions, particularly chronic itch diseases such as eczema, we present acousto-mechanic wearable sensing hardware and machine learning models for monitoring of scratching be havior. Next, we present work in 3D reconstruction of the skin surface using GelSight tactile sensing integrated into a probe, offering a validated tool for skin surface analysis and wrinkle depth estimation, with potential applications in diagnosis and treatment monitoring. </p><p dir="ltr">Extending beyond dermatology, we develop active, shared, and passive control interfaces for individuals with severe motor impairments to use caregiving robots. In the space of active control, we present HAT, a head worn inertial device for robot teleoperation. For shared control, we refine HAT to blend autonomy with user intent, validated in a 7-day in-home deployment with a non-speaking individual with quadriplegia. We also introduce VoicePilot, a speech-based interface powered by large language models, supporting flexible and natural robot control and validated with older adults. Lastly, for passive control, we present WAFFLE, a wearable system that uses inertial and throat microphone data to estimate bite timing in robot-assisted feeding, generalizing across users, robot types, and dining contexts. </p><p dir="ltr">Together, these systems advance the state of the art in multimodal sensing and robotic interfaces for chronic care, contributing novel algorithms, validated hardware, and empirical insights that deepen our understanding of how portable, multimodal technologies can support long-term health management and promote human autonomy</p>