Black Hat USA 2025 | Enhancing Command Line Classification with Benign Anomalous Data

Anomaly Detection Betrayed Us, so We Gave It a New Job: Enhancing Command Line Classification with Benign Anomalous Data Anomaly detection in cybersecurity has long promised the ability to identify threats by highlighting deviations from expected behavior. For classifying malicious command lines, however, its practical application often results in high false positive rates, making it expensive and inefficient. But is that the whole story for command line anomaly detection? With recent innovations in AI, is there a new angle that we have yet to explore? In this Briefing, we will explore that question by developing a pipeline that does not depend on anomaly detection as a point of failure. By combining anomaly detection with large language models (LLMs), we can confidently identify critical data that can be used to augment a dedicated command line classifier. Using anomaly detection to feed a different process avoids the potentially catastrophic false positive rates of an unsupervised method. Instead, we create improvements in a supervised model targeted towards classification. Unexpectedly, the success of this method did not depend on anomaly detection locating malicious command lines. We gained a valuable insight: anomaly detection, when paired with LLM-based labeling, yields a remarkably diverse set of benign command lines. Leveraging this benign data when training command line classifiers significantly reduces false positive rates. Furthermore, it allows us to use plentiful existing data without the needles in a haystack that are malicious command lines in production data. Attendees will gain an understanding of the methodology of our experiment, highlighting how diverse benign data identified through anomaly detection broadens the classifier's understanding and contributes to creating a more resilient detection system. By shifting focus from solely aiming to find malicious anomalies to harnessing benign diversity, we offer a potential paradigm shift in command line classification strategies. Learn how to easily implement this method in your detection systems at a large scale and low cost. By: Ben Gelman | Senior Data Scientist, Sophos Sean Bergeron | Senior Data Scientist, Sophos Presentation Materials Available at: https://ift.tt/UAroVyd

source https://www.youtube.com/watch?v=om5x9aFrnLE