Exposing intrusion campaigns has become a geopolitical tool, with governments and commercial firms publishing threat intelligence reports about hacking attempts and modus operandi. U.S. government officials have explained this as not just a defensive practice but also as a way to ‘impose cost’ on attackers by forcing them to develop new infrastructure, tools, and techniques, consuming their scarce resources. We empirically examine this claim by analyzing attacker behavior before and after the publication of indicators of compromise (IOCs). Using IOC feeds from two leading commercial providers – deemed to best enable detection of sophisticated threats – we matched IOCs against a large dataset of real-world network traffic metadata. This enabled us to generate sightings retroactively, capturing malicious activity up to 150 days before and after publication. Unlike prior work focused on post-publication malicious activity, our method provides a more complete view over time. Our results show that most IOCs point to resources that attackers had already abandoned by the time of IOC publication, limiting their utility for detecting ongoing attacks and undermining the idea of ‘imposing costs’. Statistical modeling further reveals that publication status has low explanatory power for sightings, suggesting that confounding variables exist. We also observed a 30-day delay between the peak of threat actor activity and IOC publication for one provider. This study is the first empirical assessment linking threat intelligence publication to attacker behavior, bridging computer science and international relations. ... Read more

The number of published software vulnerabilities is increasing every year. How do organizations stay in control of their attack surface despite their limited staff resources? Prior work has analyzed the overall software vulnerability ecosystem as well as patching processes within organizations, but not how these two are connected.We investigate this missing link through semi-structured interviews with 22 organizations in critical infrastructure and government services. We analyze where in these organizations the responsibility is allocated to collect and triage information about software vulnerabilities, and find that none of our respondents is acquiring such information comprehensively, not even in a reduced and aggregated form like the National Vulnerability Database (NVD). This means that information on known vulnerabilities will be missed, even in critical infrastructure organizations. We observe that organizations apply implicit and explicit coping mechanisms to reduce their intake of vulnerability information, and identify three trade-offs in these strategies: independence, pro-activeness and formalization.Although our respondents' behavior is in conflict with the widely accepted security advice to collect comprehensive vulnerability information about active systems, no respondents recall having experienced a security incident that was associated with missing information on a known software vulnerability. This suggests that, given scarce resources, reducing the intake of vulnerability information by up to 95% can be considered a rational strategy. Our findings raise questions about the allocation of responsibility and accountability for finding vulnerable systems, as well as suggest changing expectations around collecting vulnerability information. ... Read more

We tracked the largest volunteer security information sharing community known to date: the COVID-19 Cyber Threat Coalition, with over 4,000 members. This enabled us to address long-standing questions on threat information sharing. First, does collaboration at scale lead to better coverage? And second, does making threat data freely available improve the ability of defenders to act? We found that the CTC mostly aggregated existing industry sources of threat information. User-submitted domains often did not make it to the CTC's blocklist as a result of the high threshold posed by its automated quality assurance using VirusTotal. Although this ensured a low false positive rate, it also caused the focus of the blocklist to drift away from domains related to COVID-19 (1.4%-3.6%) to more generic abuse, such as phishing, for which established mitigation mechanisms already exist. However, in the slice of data that was related to COVID-19, we found promising evidence of the added value of a community like the CTC: just 25.1% of these domains were known to existing abuse detection infrastructures at time of listing, as compared to 58.4% of domains on the overall blocklist. From the unique experiment that the CTC represented, we draw three lessons for future threat data sharing initiatives. ... Read more

Commercial threat intelligence is thought to provide unmatched coverage on attacker behavior, but it is out of reach for many organizations due to its hefty price tag. This paper presents the first empirical assessment of the services of commercial threat intelligence providers. For two leading vendors, we describe what these services consist of and compare their indicators with each other. There is almost no overlap between them, nor with four large open threat intelligence feeds. Even for 22 specific threat actors – which both vendors claim to track – we find an average overlap of only 2.5% to 4.0% between the indicator feeds. The small number of overlapping indicators show up in the feed of the other vendor with a delay of, on average, a month. These findings raise questions on the coverage and timeliness of paid threat intelligence.

We also conducted 14 interviews with security professionals that use paid threat intelligence. We find that value in this market is understood differently than prior work on quality metrics has assumed. Poor coverage and small volume appear less of a problem to customers. They seem to be optimizing for the workflow of their scarce resource – analyst time – rather than for the detection of threats. Respondents evaluate TI mostly through informal processes and heuristics, rather than the quantitative metrics that research has proposed. ... Read more