AutoBnB-RAG: enhancing multi-agent incident response
An extension of AutoBnB that incorporates RAG into multi-agent incident response simulations.
Incident response (IR) requires fast, coordinated, and well-informed decision-making to contain and mitigate cyber threats. While large language models (LLMs) have shown promise as autonomous agents in simulated IR settings, their reasoning is often limited by a lack of access to external knowledge. In this work, we present AutoBnB-RAG, an extension of the AutoBnB framework that incorporates retrieval-augmented generation (RAG) into multi-agent incident response simulations. Built on the Backdoors & Breaches (B&B) tabletop game environment, AutoBnB-RAG enables agents to issue retrieval queries and incorporate external evidence during collaborative investigations. We introduce two retrieval settings: one grounded in curated technical documentation (RAG-Wiki), and another using narrative-style incident reports (RAG-News). We evaluate performance across eight team structures, including newly introduced argumentative configurations designed to promote critical reasoning. To validate practical utility, we also simulate real-world cyber incidents based on public breach reports, demonstrating AutoBnB-RAG's ability to reconstruct complex multi-stage attacks. Our results show that retrieval augmentation improves decision quality and success rates across diverse organizational models. This work demonstrates the value of integrating retrieval mechanisms into LLM-based multi-agent systems for cybersecurity decision-making.
Latest publications
EPSVec: Efficient and private synthetic data generation via dataset vectors
A differentially-private lightweight alternative that steers LLM generation using dataset vectors.
ICMLCritique-guided distillation for robust reasoning via refinement
A training framework that decouples critique consumption from critique generation.
ICMLCommonLID: Re-evaluating state-of-the-art language identification performance on web data
A community-driven, human-annotated LID benchmark for the web domain, covering 109 languages.
ACL