Black Hat USA 2025 | From Prompts to Pwns: Exploiting and Securing AI Agents

The flexibility and power of large language models (LLMs) are now well understood, driving their integration into a wide array of real-world applications. Early use cases, such as retrieval-augmented generation (RAG), followed rigid, predictable workflows where models interacted with external systems in tightly controlled sequences. While these systems were easier to optimize and secure, they often resulted in inflexible, single-purpose tools. In contrast, modern agentic systems leverage expanded input modalities, such as speech and vision, and use more sophisticated inference strategies, such as dynamic chain-of-thought reasoning. These advancements allow them to act independently on users' behalf to automate increasingly complex workflows, often involving sensitive data and systems. As their utility increases, so too does their attack surface: more usability means broader access to data, greater ability to execute actions, and significantly more opportunity for exploitation. In this talk, we will explore the emerging security challenges posed by agentic AI systems. We demonstrate the implications of this significant shift through internal assessments and proof-of-concept exploits developed by our AI Red Team, targeting a range of agentic applications, from popular open-source tools to enterprise systems. These exploits all leverage the same core finding: that LLMs are uniquely vulnerable to malicious input, and exposure to such input can have a significant impact on the trust of downstream actions. In short, we lay out what can go wrong when agentic systems vulnerable to adversarial inputs are deployed within enterprise environments. We conclude by discussing how NVIDIA addresses the security of emerging agentic workflows, and our principles for designing agent interactions in ways that mitigate risk, emphasizing a security-first foundation for safe and scalable adoption. By: Rebecca Lynch | Offensive Security Researcher, NVIDIA Rich Harang | Principal Security Architect, NVIDIA Presentation Materials Available at: https://ift.tt/FjcC9HR

source https://www.youtube.com/watch?v=zipgr080EQU