The spread of Child Sexual Abuse Material (CSAM) and Terrorist Content Online (TCO) remains a pressing societal issue. Various organizations rely on hash databases to detect, flag, and remove harmful content. These databases function as storage of digital fingerprints of previously identified illegal material, enabling automated platform filtering. However, the effectiveness and reliability of such databases rely on the verification processes used to determine what content qualifies for inclusion.

This thesis investigates the characteristics of verification processes in CSAM and TCO hash databases, with a particular focus on triple verification. Using a multiphase mixed-methods design, the study integrates qualitative insights from stakeholder interviews, an annotation experiment, and a follow-up focus group with annotators.

The interviews with experts highlighted variations in verification workflows, ranging from single-rater decisions to triple verification models. While triple verification is seen as a standard for increasing trust and minimizing false positives, its feasibility in terms of emotional toll and volume has been questioned. Thematic insights centered around benefits (e.g., legal considerations), challenges (e.g., emotional toll, inconsistent thresholds), necessity (e.g., utility and impact), future opportunities (e.g., automation), and differences between CSAM and TCO workflows.

In the experiment, two raters from the Dutch National Police classified 2,031 real potentially illegal items under two different conditions. In the blind phase, raters voted independently, whereas in the non-blind phase, prior votes were visible. Overall inter-rater agreement rose from 89.4% in the blind condition to 97.1% in the non-blind condition. A statistically significant association was found between voting order and agreement rates, suggesting that seeing one or two prior votes can subtly influence rater alignment.

The focus group offered further insight into the found disagreements. A key theme was the importance of recognizing image series: individual images were often reclassified as illegal when identified as part of a known CSAM series. Age estimation was also a recurring source of ambiguity, particularly when visual quality was poor or when victims’ physical development and ethnicity made assessment difficult. Raters relied on indicators such as skin texture, body proportions, and dental features, though these cues were often interpreted differently.

The findings emphasize the need for verification systems that are both flexible and context-sensitive. Not all cases require the same level of scrutiny: while baseline CSAM could be classified with fewer checks, ambiguous cases require more checks. Rather than enforcing uniformity, organizations should accommodate interpretive differences while safeguarding consistency and accountability.

Today, huge volumes of information flow at unprecedented speeds, and large organisations like the Dutch National Police face significant challenges. Efficiently sharing, processing, and prioritizing information is essential but has become increasingly difficult to achieve. These chal- lenges often hinder their ability to make sound and timely decisions. This study investigates innovative methods to enhance collaborative decision-making within such complex environ- ments.
The main goal of this study is to research and test the potential of Artificial Swarming Intelli- gence to enable collective ranking of information based on its importance. The resulting Proof of Concept offers a new method for improving the efficiency of information sharing, enhancing collective intelligence, and facilitating the decision-making processes within the police force.
Drawing inspiration from natural swarming behaviour seen in species like bees, the platform allows participants to rank multiple pieces of information during collaborative sessions. This process helps to highlight the most significant topics, enabling quicker access to critical in- sights.
The Proof of Concept is structured as a digital platform designed to improve real-time decision- making. For our case study, we focused on the policy advisors of the Dutch National Police. The Proof of Concept operates as a client-server model, ensuring that user interactions are efficient while providing live updates on rankings. This allows participants to engage actively and see the evolving importance of various pieces of information.
However, despite the progress achieved, several limitations became evident. First, some envisioned functionalities were not fully developed or implemented within the project’s time frame. In addition, current design faces challenges in scalability, particularly when engaging larger groups of users. Issues related to user accessibility and potential biases in decision- making processes also came into view. To guide further work, we have highlighted key areas for future research and provided specific considerations that address these limitations, aiming to support the Proof of Concept’s development into a complete and fully scalable platform.
Ultimately, this Proof of Concept presents a promising approach to enhancing decision-making processes at the Dutch National Police. It lays the foundation for future research and devel- opment, with opportunities to refine its functionality and expand its use within the organisation. By improving how information is shared and prioritised, the Dutch National Police can enhance its collective intelligence and improve its decision-making processes.