Co-op Tile World: A Benchmarking Environment for Human-AI Cooperative Interaction

Cooperative AI research is widely regarded as a next milestone in the development of advanced AI systems--where AI agents do not operate in isolation, but instead act as autonomous and social members of a larger network.

This research begins by examining cooperative behaviors through the lens of reinforcement learning. Unlike autonomous agents that perform tasks independently within an environment, cooperative tasks introduces a broader notion of context. In cooperative scenarios, the environment expands beyond the tangible task space to include intangible social elements such as norms and cultural expectations. Also, agents should now consider the presence of other agents as a group or individual, while the conditions for cooperation also plays into. Accordingly, an agent’s action set must include both autonomous and social behaviors. In response to these actions, agents receive not only task-related rewards, but also social rewards, and both the task state and social state evolve over time.

Social abilities are especially crucial in cooperative tasks between humans and agents, since successful cooperation hinges on providing participants with a comprehensive combination of task-related feedback and social cues. Feedback and cues allow participants to dynamically adjust their behaviors in pursuit of shared goals. Humans, for instance, do not evaluate cooperation solely based on final outcomes; they continuously adapt based on the perceived quality of interaction. Therefore, it is essential to develop agents with diverse social capabilities—built on top of reliable autonomous capabilities—and to study how those agents cooperate with real human partners.

Yet, there is currently no suitable environment for conducting cooperative AI research in a human-in-the-loop setting. While existing Multi-Agent Reinforcement Learning (MARL) environments are commonly used to benchmark agent performance, they primarily focus on AI-to-AI interactions and lack the necessary mechanisms to showcase an agent’s social capabilities—an essential component when interacting with humans.

To address this gap, this research introduces a custom benchmarking environment called Coop Tile World, specifically designed to support experimentation on socially cooperative AI and human cooperation. Furthermore, this study leverages the environment to evaluate cooperative performance between human players and AI agents, parameterizing the agent’s social feature—its alignment with the human’s decisions in the collective decision-making process—across two levels of task difficulty.

This work lays the groundwork for developing socially grounded cooperative agents that generate rich social feedback throughout the course of interaction. Through empirical studies using this environment, the research offers foundational insights into the nature of human–machine cooperation. For instance, our user study revealed that task difficulty had a significant main effect on cooperative performance. We also observed trends indicating that the agent’s alignment mode—and its interaction with task difficulty—influenced cooperative outcomes. These findings from human-in-the-loop experiments with diverse social agents carry broad implications for designing AI companions, socially assistive robots, and AI partners in cooperative games—agents that not only perform tasks autonomously, but also engage in context-aware social interaction.