What Lies Beneath the Surface? Evaluating LLMs for Offensive Cyber Capabilities

What Lies Beneath the Surface? Evaluating LLMs for Offensive Cyber Capabilities through Prompting, Simulation & Emulation Large Language Models (LLMs) show remarkable aptitude for analyzing code and employing software, leading to concerns about potential misuse in enabling autonomous or AI-assisted offensive cyber operations (OCO). Current LLM risk assessments present a false sense of security by primarily testing models' responses to open-ended hacking challenges in isolated exploit/action scenarios, a bar which today's off-the-shelf LLMs largely fail to meet. This fails to quantify graduated risks that LLMs may be capable of being adapted or guided by a malicious adversary to enable specific preferred tactics and techniques. In effect, this has left cyber defenders without a confident answer to the question "Does this LLM actually pose an offensive cyber threat to my system?" We address this gap by developing a more granular and repeatable means to measure, forecast, and prioritize defenses to near-term operational OCO risks of LLMs. In this talk, we present a rigorous, multifaceted methodology for evaluating the extent to which a given LLM has true offensive cyber capabilities. This methodology includes not only LLM prompt and response evaluation mechanics but also high-fidelity cyber-attack simulations and emulation test scenarios on real cyber targets. In effect, with our evaluation framework, selected LLMs are put through a barrage of repeatable tests, scenarios, and settings to elicit whether ever increasing levels of offensive cyber capabilities exist within the model's capacity. For this talk, we will detail our LLM evaluation methodology, technical implementation and tooling, provide results from our initial round of LLM evaluations, and have a real demonstration of an LLM evaluation for offensive cyber capabilities. Copyright 2024 The MITRE Corporation. ALL RIGHTS RESERVED. Approved for public release. Case 24-1222. By: Michael Kouremetis | Principal Adversary Emulation Engineer, MITRE Marissa Dotter | Senior Artificial Intelligence Engineer, MITRE Alex Byrne | Applied Cybersecurity Engineer, MITRE Dan Martin | Senior Offensive Security Engineer, MITRE Ethan Michalak | Cybersecurity Engineer, MITRE Gianpaolo Russo | Principal Engineer, MITRE Michael Threet | Principal AI Research Engineer, MITRE Full Abstract and Presentation Materials: https://ift.tt/OrkSG2W

source https://www.youtube.com/watch?v=p9T4gWds54o