Black Hat Europe 2025 | Flaw And Order: Finding The Needle In The Haystack Of CodeQL Using LLMs

Running CodeQL's built-in queries on Redis gave me over 6,800 potential issues. Doable, maybe. But when I tried FFmpeg, I got over 51,000. That's way too much for me. And how many of those are real vulnerabilities? Probably around 0.01%. The sheer number of false positives makes static code analysis impractical - who wants to manually sift through tens of thousands of results just to find a few actual security flaws? To fix this, we built an open-source tool that fuses CodeQL with an LLM-driven agent. This agent autonomously navigates the code, running targeted queries to extract only the relevant context. On top of that, we introduced Guided Questioning, an advanced reasoning technique that keeps the LLM focused, improving accuracy even for complex vulnerabilities. Using this approach, we reduced false positives by up to 97% and uncovered more than a dozen real-world security issues in Linux, Apache, FFmpeg, Bullet3, Libvips, libretro, Linenoise, and other widely used open-source projects. By: Simcha Kosman | Senior Security Researcher, Cyberark https://ift.tt/NSzZIgc

source https://www.youtube.com/watch?v=gcsIUqb6s_8