Theory of Mind in Multi-Agent Systems

<p dir="ltr">The ability to model the internal mental states of others– known as theory of mind — is a crucial aspect of human social intelligence. In this work we present an interpretable family of approaches to modeling theory of mind within artificial agents based on concept learning, and explore this in the context of deep imitation and reinforcement learning. We posit that endowing artificial agents with theory of mind increases both model transparency and trust from a human perspective as well as task performance with respect to navigating social dynamics in both competitive and cooperative multi-agent scenarios. </p><p dir="ltr">The first part of this work focuses on theory of mind as a framework for modeling artificial agents in imitation learning. Completed work develops a modular neural framework to explicitly model theory of mind from observed trajectories, and introduces concept whitening as an approach to ensuring interpretability of learned policies. The efficacy of this approach is demonstrated with experiments on data from human participants on a search and rescue task in Minecraft. </p><p dir="ltr">The second part of this work focuses on higher-level theory of mind inference within the context of multi-agent reinforcement learning. Completed work extends our concept-based approach to interpretability by introducing an information theoretic variant to concept learning via bottleneck while incorporating residual latent knowl edge. Experiments in a variety of cooperative, competitive, and mixed multi-agent scenarios show that introducing higher-order theory of mind inference as an intrinsic reward for agents can lead to policies with improved coordination, strategy, and efficiency of communication. </p><p dir="ltr">The final part of this work focuses on deception via theory of mind in zero sum games. We present an approach for improving performance in zero-sum Multi Agent Reinforcement Learning (MARL) by rewarding agents for being deceptive. Our framework utilizes opponent theory of mind (ToM) modeling error over beliefs and intents as a signal to induce deceptive behavior. We extend this framework to higher-order ToM reasoning, where beyond 0th-order beliefs about the environment, an agent aims to confound an opponent’s 1st and 2nd order beliefs (i.e. beliefs about beliefs). We present empirical results in Barrage Stratego, Kuhn Poker, and Mafia, where we find that higher-order deceptive policies consistently outperform baseline and lower-order deceptive policies.</p>