AutoBnB-RAG: enhancing multi-agent incident response
An extension of AutoBnB that incorporates RAG into multi-agent incident response simulations.
Incident response (IR) requires fast, coordinated, and well-informed decision-making to contain and mitigate cyber threats. While large language models (LLMs) have shown promise as autonomous agents in simulated IR settings, their reasoning is often limited by a lack of access to external knowledge. In this work, we present AutoBnB-RAG, an extension of the AutoBnB framework that incorporates retrieval-augmented generation (RAG) into multi-agent incident response simulations. Built on the Backdoors & Breaches (B&B) tabletop game environment, AutoBnB-RAG enables agents to issue retrieval queries and incorporate external evidence during collaborative investigations. We introduce two retrieval settings: one grounded in curated technical documentation (RAG-Wiki), and another using narrative-style incident reports (RAG-News). We evaluate performance across eight team structures, including newly introduced argumentative configurations designed to promote critical reasoning. To validate practical utility, we also simulate real-world cyber incidents based on public breach reports, demonstrating AutoBnB-RAG's ability to reconstruct complex multi-stage attacks. Our results show that retrieval augmentation improves decision quality and success rates across diverse organizational models. This work demonstrates the value of integrating retrieval mechanisms into LLM-based multi-agent systems for cybersecurity decision-making.
Latest publications
Routing with generated data
A setting in which routers are trained on generated queries and answers produced from high-level task descriptions. (ACL)
ACLCommonLID: Re-evaluating language identification performance
A community-driven, human-annotated LID benchmark for the web domain, covering 109 languages. (ACL)
ACLMacaron: Controlled, human-written benchmark
A template-first benchmark that factorizes reasoning type and cultural aspect across question languages. (ACL)
ACL