Mobile Behaviometrics: Behavior Modeling from Heterogeneous Sensor Time-Series

In the era of ubiquitous and mobile computing, the penetration of mobile devices equipped with various embedded sensors makes it possible to capture the physical and virtual context of the user and his/her surrounding environment. There is also an increasing demand to model human behaviors based on those data to enable context-aware computing and people-centric applications, which utilize users’ behavior pattern to improve the existing services or develop new services. Over the decades, we have seen tremendous success in biometrics technologies being used in all types of applications based on the physical attributes of the individual such as face, fingerprints, voice and iris. Inspired by this, we introduce a new concept Mobile Behaviometrics, which uses algorithms and models to measure and quantify unique human behavioral patterns in place of human bio-attributes. Behaviometrics algorithms take multiple data from various sensors as input and fuse them to build behavioral models which are capable of producing application specific quantitative analysis on the unique individuals that were the originators of the data. We have three main challenges: statistical modeling of the sensor data, difficulty of multi-dimensional sensor fusion and structural activity recognition. Based on our observation of the similarity between human behavior and natural language, we apply natural language processing (NLP) techniques to statistically model heterogeneous sensor data and investigate various applications based on the derived models. We also investigate adapting conventional machine learning techniques to human behavioral modeling for non time-sensitive behavior sequences. To overcome the difficulty of multi-dimensional sensor fusion, we develop a hybrid approach to first learn behavioral factors for feature dimension reduction and then infer optimal way of fusing models from different factors. We study a novel Helix algorithm to induce the underlying grammar of human activities and ultimately solve the structural activity recognition. Our analytical work has been incorporated into a mobile security application SenSec, which allows us to conduct extensive experiments on behavioral modeling and its applications through passive sensing. Through the deployment and field tests of SenSec, we conclude that the proposed modeling algorithms can effectively identify users and their unique behavioral patterns and eventually detect anomalies from suspicious behaviors. Our SenSec application and its foundation application MobiSens¹ are publicly available. They continuously collect huge amount of mobile sensing data which will enable a broader range of research in modeling and analyzing human behaviors in the near future.