AI vision systems see differently than humans do. When platforms downscale uploads to save compute, the mathematical properties of interpolation algorithms create exploitable artifacts. In this presentation, we'll show how to craft images which use invisible pixel perturbations to reveal malicious prompts after downscaling, triggering unauthorized tool execution across Google Gemini, Vertex AI, Google Assistant, and Genspark. Beyond image downscaling, we'll explore the broader attack surface, including audio transformations, dithering algorithms, and other preprocessing steps that become prompt injection vectors. You'll learn to fingerprint vulnerable systems using test patterns that reveal specific downscaling implementations across AI libraries. We'll demo Anamorpher, our open-source tool for automated attack generation, with both Python APIs and visual interfaces, as well as examine practical mitigations from displaying actual processed images to implementing design patterns resistant to prompt injection, such as the action selector pattern.
By:
Suha Hussain | AI Research Engineer, Product Security, Harvey
Kikimora Morozova | Security Researcher, Trail of Bits
https://ift.tt/MuZqL6N
source https://www.youtube.com/watch?v=rHvFGz7_67k
-
Unmasking State-Sponsored Mobile Surveillance Malware from Russia, China, and North Korea – Threat Actors, Tactics, and Defense Strategies S...
-
Germany recalled its ambassador to Russia for a week of consultations in Berlin following an alleged hacker attack on Chancellor Olaf Scho...
No comments:
Post a Comment