Situated Intelligent Interactive Systems

The recent wide usage of Interactive Systems (or Dialog Systems), such as Apple’s Siri has attracted a lot of attention. Besides personal assistants, dialog systems can also be applied in various domains, such as education, health care and entertainment. However, none of the current systems coordinate well with their users, leading to less than ideal performance. The ultimate goal for dialog research is to transform these current systems into real intelligent systems that can communicate with users effectively and naturally. There are three major challenges to this ultimate goal: first, how to make systems cooperate with users in a natural manner; second, how to provide adaptive and personalized experience to each user; and last, how to make multiple task systems transition from one task to another fluidly to achieve overall conversation effectiveness and naturalness. To address these challenges, I proposed a theoretical framework, Situated Intelligence (SI).

In the SI framework, we argue that three capabilities are needed to achieve natural and high quality conversations: (1) situation awareness, which can be approximated by multimodal information; (2) a rich repertoire of conversation strategies to regulate systems’ situational contexts; and (3) a global planning policy that optimally chooses among different conversation strategies at run-time to achieve an overall natural conversation flow. We applied the SI framework to three different types of conversations: non-task-oriented conversations (e.g. social chat), task-oriented conversations (e.g. job interview training) and conversations that interleave both task and no-task content (e.g. movie promotion) and made a number of contributions in both algorithm development and end-to-end system building.