A reduction in phishing threats is of increasing importance to organizations. One part of this effort is to provide training to employees, so that they are able to identify and avoid phishing emails. Yet further, simulated phishing emails are used to test whether employees will both identify and report a suspicious email. We worked with a partner bank to examine a repository of many thousands of reported emails from a behavioural perspective. We divide reported emails into categories and examine reporting trends over time relative to training and phishing simulation campaigns. Among our findings, the level of reporting of benign emails is comparable to the number of malicious emails reported, and we see indications that training and simulations amplify the reporting of benign emails. Our analysis uncovers reporting patterns for unique reporters per email campaign as a promising indicator for the security-related culture around phishing prevention. Evidence from our analysis informs recommendations, such as providing reporting infrastructure for reporting not only malicious emails, but also benign but suspicious work-related emails, in a manner that minimises the disruption for users erring on the side of caution when assessing emails.

Large- and medium-sized organizations employ various security systems to protect their assets. These systems, often developed by different vendors, focus on different threats and usually work independently. They generate separate and voluminous alerts that have to be monitored and analyzed by often overburdened security analysts. Prior work has tried to support analysts by better correlating and prioritizing alerts. In this work, we propose to combine the wisdom of individual security systems using an Integration Layer (IL). We validated our idea by deploying the IL in a large global organization (50,000+ employees) running four very different security detection systems. We did so by using end-to-end red-team exercises to generate real attack data. For training, we labeled our dataset with evaluations directly from the incident response team instead of using the escalated decisions of the first/second tier Security Operation Center (SOC) analysts as in prior works. We showed that our approach considerably reduces the number of alerts requiring investigation while maintaining very high performance on multi-step attack detection - Matthews correlation coefficient (MCC) reaches 0.998. The substantial dependence of the model on features derived from the different security systems supports the viability of our integration methodology. The explainability layer added to our system gives analysts insights into why a particular case is marked as an attack or non-attack. Based on the test results, our approach has been added to the production setup.