Carnegie Mellon University
Browse

Simulating Realistic Human Activity Using Large Language Model Directives

Download (783.22 kB)
report
posted on 2023-10-02, 21:35 authored by Dustin UpdykeDustin Updyke, Thomas PodnarThomas Podnar, Sean HuffSean Huff

In this report, we explore how activities generated from the GHOSTS Framework’s non-player character (NPC) client, including software usage, compare to activities produced by GHOSTS’ default behavior and large language models (LLMs). We also explore how the underlying results compare in terms of complexity and sentiment. In our research, we leveraged the advanced natural language processing capabilities of generative artificial intelligence (AI) systems, specifically LLMs (i.e., OpenAI’s GPT-3.5 Turbo and GPT-4) to guide virtual agents (i.e., NPCs) in the GHOSTS Framework, a tool that simulates realistic human activity on a computer. We devised a configuration to fully automate activities by using an LLM, where text outputs become executable agent directives. Our preliminary findings indicate that an LLM can generate directives that result in coherent, realistic agent behavior in the simulated environment. However, the complexity of certain tasks and the translation of directives to actions present unique challenges. This research has potential implications for enhancing the realism of simulations and pushing the boundaries of AI applications within human-like activity modeling. Further studies are recommended to optimize agent understanding and response to LLM directives. 

History

Publisher Statement

This material is based upon work funded and supported by the Department of Defense under Contract No. FA8702-15-D-0002 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The view, opinions, and/or findings contained in this material are those of the author(s) and should not be construed as an official Government position, policy, or decision, unless designated by other documentation. References herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by Carnegie Mellon University or its Software Engineering Institute. This report was prepared for the SEI Administrative Agent AFLCMC/AZS 5 Eglin Street Hanscom AFB, MA 01731-2100. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. [DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see Copyright notice for non-US Government use and distribution.

Copyright Statement

Copyright 2023 Carnegie Mellon University.

Usage metrics

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC