AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generation

AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generation

Perrina, Filippo (University of Padova)
Marchiori, Francesco (University of Padova)
Conti, M. (TU Delft Cyber Security; University of Padova)
Verde, Nino Vincenzo (Leonardo S.p.A.)

2023

Cyber Threat Intelligence (CTI) reporting is pivotal in contemporary risk management strategies. As the volume of CTI reports continues to surge, the demand for automated tools to streamline report generation becomes increasingly apparent. While Natural Language Processing techniques have shown potential in handling text data, they often struggle to address the complexity of diverse data sources and their intricate interrelationships. Moreover, established paradigms like STIX have emerged as de facto standards within the CTI community, emphasizing the formal categorization of entities and relations to facilitate consistent data sharing. In this paper, we introduce AGIR (Automatic Generation of Intelligence Reports), a transformative Natural Language Generation tool specifically designed to address the pressing challenges in the realm of CTI reporting. AGIR’s primary objective is to empower security analysts by automating the labor-intensive task of generating comprehensive intelligence reports from formal representations of entity graphs. AGIR utilizes a two-stage pipeline by combining the advantages of template-based approaches and the capabilities of Large Language Models such as ChatGPT. We evaluate AGIR’s report generation capabilities both quantitatively and qualitatively. The generated reports accurately convey information expressed through formal language, achieving a high recall value (0.99) without introducing hallucination. Furthermore, we compare the fluency and utility of the reports with state-of-the-art approaches, showing how AGIR achieves higher scores in terms of Syntactic Log-Odds Ratio (SLOR) and through questionnaires. By using our tool, we estimate that the report writing time is reduced by more than 40%, therefore streamlining the CTI production of any organization and contributing to the automation of several CTI tasks.

Cyber Threat Intelligence
Natural Language Generation
Threat Reports
STIX

http://resolver.tudelft.nl/uuid:b416e921-936b-4b96-a451-2d0089d7c987

https://doi.org/10.1109/BigData59044.2023.10386116

IEEE

2024-07-22

979-8-3503-2446-4

Proceedings of the 2023 IEEE International Conference on Big Data (BigData)

2023 IEEE International Conference on Big Data (BigData), 2023-12-15 → 2023-12-18, Sorrento, Italy

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2023 Filippo Perrina, Francesco Marchiori, M. Conti, Nino Vincenzo Verde