Structural Vibration Based Occupant Activity Recognition in Real-world Environments

Automated monitoring of humans and animals can facilitate better health and productivity. Prior monitoring approaches use video, which requires line-of-sight and high processing power, or motion detection, which has difficulty separating subtle activities. Wearable sensors can address these issues but are vulnerable to human forgetfulness and animal destructiveness. This thesis presents a system that uses structural vibration to monitor animal and human activities, along with a framework to apply the system to general activity recognition applications. 

The system described in this thesis characterizes the structural vibration caused by two types of activities: those that cause various types of excitation and those that change the distribution of static load on the structure. It also detects simultaneously occurring activities. Behavioral and domain knowledge are used to adapt the system to changing environments, and it is evaluated on humans in a home office environment and on pigs at an operational pig farm. 

With this in mind, the primary contributions of this thesis are: 

• A discussion of the characteristics of excitation inducing activities that affect a structure’s dynamic load, where we use domain knowledge to address the challenge of different surface structures affecting the signal, compare variation in activities between different human subjects, and evaluate our analysis with different structures and human behavior variations in a home office environment. 

• A analysis of the characteristics of load distribution changing activities that affect the structure’s distribution of static load, including a discussion of how they and excitation inducing activities are differently affected by activity overlap, and a framework for parallel classification of overlapping activities.

 • Deployment and evaluation experiences at an operating pig farm for over three months, including descriptions of three iterations of hardware and how they survived the environment, and a discussion of the trade-offs and considerations that were made. This was the first system to use structural vibration to sense animal activity, and was able to recognize piglet nursing as well as detect the onset of birth a day in advance.

The work in my thesis addresses three major challenges. First, indoor activities often create vibration on different surfaces of the building structure, which may respond differently to vibration. We address this with  characterization of our different structures, and an analysis of sensor placement and sensor combinations. Second, small amounts of data makes classification challenging. We address this with domain knowledge about structural characteristics of different kinds of activities. Third, our real-world deployments deal with varying noise sources and structures that change characteristics over time, due to environmental factors. We address with by using multiple redundant sensors to add information from different points in the structure, and by incorporating behavioral knowledge of time-series activities to make our model more robust. 

We present results from a series of experiments in a home office and from a real-world farm deployment for over six months. The deployed system was able to achieve a daily pen-level status profile of up to 90% accuracy, which tracks nursing activity and sow lying activity. This status profile can be used to predict birth a day in advance, which has great potential to save piglet lives during the birthing process.