What Can Generative AI Red-Teaming Learn from Cyber Red-Teaming?
report
posted on 2025-07-17, 19:05 authored by Anusha SinhaAnusha Sinha, James LucassenJames Lucassen, Keltin GrimesKeltin Grimes, Michael FefferMichael Feffer, Mary SotoMary Soto, Hoda HeidariHoda Heidari, Nathan VanhoudnosNathan Vanhoudnos<p dir="ltr">Red teaming, a security practice rooted in adversarial emulation, has been widely applied across various domains, including cybersecurity and artificial intelligence (AI). This paper investigates the applicability of established cyber red-teaming methodologies to the evaluation of generative AI systems, addressing the growing need for robust security assessments in AI-driven applications. Through a pair of systematic literature reviews, we synthesize existing generative AI red-teaming approaches and analyze their alignment with established practices in cyber red-teaming.</p>
History
Publisher Statement
This material is based upon work funded and supported by the Department of Defense under Contract No. FA8702-15-D-0002 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The view, opinions, and/or findings contained in this material are those of the author(s) and should not be construed as an official Government position, policy, or decision, unless designated by other documentation. References herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by Carnegie Mellon University or its Software Engineering Institute. This report was prepared for the SEI Administrative Agent AFLCMC/AZS 5 Eglin Street Hanscom AFB, MA 01731-2100. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. [DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see Copyright notice for non-US Government use and distribution.Copyright Statement
Copyright 2025 Carnegie Mellon University.Usage metrics
Categories
No categories selectedLicence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC