In our work as advisors in software and AI engineering, we are often asked about the efficacy of large language model (LLM) tools like Copilot, GhostWriter, or Tabnine. Recent innovation in the building and curation of LLMs demonstrates powerful tools for the manipulation of text. By finding patterns in large bodies of text, these models can predict the next word to write sentences and paragraphs of coherent content. The concern surrounding these tools is strong – from New York schools banning the use of ChatGPT to Stack Overflow and Reddit banning answers and art generated from LLMs. While many applications are strictly limited to writing text, a few applications explore the patterns to work on code, as well. The hype surrounding these applications ranges from adoration (“I’ve rebuilt my workflow around these tools”) to fear, uncertainty, and doubt (“LLMs are going to take my job”). In the Communications of the ACM, Matt Welsh goes so far as to declare we’ve reached “The End of Programming.” While integrated development environments have had code generation and automation tools for years, in this post I will explore what new advancements in AI and LLMs mean for software development.
Publisher Statement
This material is based upon work funded and supported by the Department of Defense under Contract No. FA8702-15-D-0002 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The view, opinions, and/or findings contained in this material are those of the author(s) and should not be construed as an official Government position, policy, or decision, unless designated by other documentation. References herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by Carnegie Mellon University or its Software Engineering Institute.
This report was prepared for the SEI Administrative Agent AFLCMC/AZS 5 Eglin Street Hanscom AFB, MA 01731-2100
NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see Copyright notice for non-US Government use and distribution.Date
2023-02-20Copyright Statement
Copyright 2023 Carnegie Mellon University.