The spread of Child Sexual Abuse Material (CSAM) and Terrorist Content Online (TCO) remains a pressing societal issue. Various organizations rely on hash databases to detect, flag, and remove harmful content. These databases function as storage of digital fingerprints of previously identified illegal material, enabling automated platform filtering. However, the effectiveness and reliability of such databases rely on the verification processes used to determine what content qualifies for inclusion.

This thesis investigates the characteristics of verification processes in CSAM and TCO hash databases, with a particular focus on triple verification. Using a multiphase mixed-methods design, the study integrates qualitative insights from stakeholder interviews, an annotation experiment, and a follow-up focus group with annotators.

The interviews with experts highlighted variations in verification workflows, ranging from single-rater decisions to triple verification models. While triple verification is seen as a standard for increasing trust and minimizing false positives, its feasibility in terms of emotional toll and volume has been questioned. Thematic insights centered around benefits (e.g., legal considerations), challenges (e.g., emotional toll, inconsistent thresholds), necessity (e.g., utility and impact), future opportunities (e.g., automation), and differences between CSAM and TCO workflows.

In the experiment, two raters from the Dutch National Police classified 2,031 real potentially illegal items under two different conditions. In the blind phase, raters voted independently, whereas in the non-blind phase, prior votes were visible. Overall inter-rater agreement rose from 89.4% in the blind condition to 97.1% in the non-blind condition. A statistically significant association was found between voting order and agreement rates, suggesting that seeing one or two prior votes can subtly influence rater alignment.

The focus group offered further insight into the found disagreements. A key theme was the importance of recognizing image series: individual images were often reclassified as illegal when identified as part of a known CSAM series. Age estimation was also a recurring source of ambiguity, particularly when visual quality was poor or when victims’ physical development and ethnicity made assessment difficult. Raters relied on indicators such as skin texture, body proportions, and dental features, though these cues were often interpreted differently.

The findings emphasize the need for verification systems that are both flexible and context-sensitive. Not all cases require the same level of scrutiny: while baseline CSAM could be classified with fewer checks, ambiguous cases require more checks. Rather than enforcing uniformity, organizations should accommodate interpretive differences while safeguarding consistency and accountability.

... Read more

Everyday use of ChatGPT spans study and work, but surveys and usage logs capture different parts of that behaviour. This thesis compares what people say in a short survey with what their anonymised ChatGPT export shows, without linking individuals. The design is simple and auditable: donors share platform-native logs, prompts are mapped to clear task families using compact example prototypes and all comparisons are made at the group level. To keep both sources directly comparable, the same frames are used on each side: intensity (how often and how long), timing (broad dayparts on one time base), input form (prompt-length bands) and task portfolio (main task families with concise subtasks). Analyses focus on full distributions and effect sizes rather than single averages. The core message is practical: self-reports give a workable signal for “how much,” while donation-based logs add detail on “how” people interact and “what” they use the tool for. Short, one-line, iterative or more technical exchanges are easy to miss in surveys, so using both sources together gives a more realistic picture for policy, training and procurement. The thesis closes with guidance on measuring and monitoring everyday use in organisations and education, with privacy and reproducibility built in. ... Read more

In today's digitally connected world, the spread of conspiracy theories on social media poses a significant challenge to societal trust and public discourse. This thesis aims to develop a model for identifying conspiracy theories on Telegram, a platform known for its private nature and the use of channels to disseminate information. The main research question guiding this study is: *How can conspiracy theories be identified on Telegram?* To address this, the research leverages graph theory, machine learning, and topic modeling.

The first step involved modeling the network structure of Telegram channels using graph theory. Channels were represented as nodes, and forwarded messages as directed, weighted edges, allowing for the analysis of network structures and the identification of influential channels. Various centrality measures, including weighted degree centrality, betweenness centrality, and viral message centrality, were computed to assess the influence of channels within the network.

A fine-tuned multilingual BERT (m-BERT) model was used to classify Telegram messages as either conspiracy-related or not. This model was trained on a manually labeled dataset and demonstrated robust performance.

To identify specific conspiracy theories, BERTopic, a topic modeling technique, was applied to the messages classified as conspiracy-related. The resulting topics were then analyzed using the OpenAI API, which linked them to known conspiracy theories. The study found that all identified topics could be connected to existing conspiracy theories, suggesting that the model is effective in detecting these narratives on Telegram.

The research also included the validation of the model using a fictive conspiracy theory, "The Verdant Shadow Conspiracy," created specifically for this purpose. This validation demonstrated the model's ability to detect even simulated conspiracies, though it highlighted the importance of continuous refinement and expansion of the dataset.

The contributions of this thesis are twofold: scientifically, it advances the understanding of how to model and analyze Telegram networks to detect conspiracy theories, and societally, it offers a framework that can be used to monitor and potentially mitigate the spread of harmful misinformation. Future research should focus on expanding the dataset, refining the model, and exploring interdisciplinary approaches to further enhance the detection and understanding of conspiracy theories on Telegram. ... Read more

In recent years, the field of artificial intelligence (AI) has witnessed rapid advancements, particularly in the domain of text-to-image generative AI (T2I GenAI) models. These models, including Stable Diffusion and DALL-E, have demonstrated remarkable capabilities, enabling the creation of lifelike images from textual descriptions. Their applications span across various fields such as graphic design and personalized content generation. However, alongside these advantages, there are significant downsides. These include the generation of unsafe content, such as sexually explicit and violent images, which can be exploited for disinformation or harassment. Furthermore, such graphic content can mentally harm (young) individuals. Some research exists that examines filter performance. However, a gap exists in understanding how these filters’ effectiveness varies across different languages. Analyzing filter effectiveness across languages may reveal weak spots that can be utilized to generate unsafe content. This research therefore aids in the ethical use of T2IGenAI models. To address the aforementioned knowledge gap, the main research question of this study is: ”How do T2IGenAI models’ safety filters perform in blocking unsafe content from English and Dutch prompts?” The main research question is unravelled into several sub-questions. Addressing these offers a comprehensive answer to the main research question. The sub-questions are formulated as follows: 1. How do blocking rates vary between English and Dutch across different domains and prompt types across the T2IGenAI models? 2. How accurately do the images generated by T2IGenAI models reflect the content of the input prompts in English and Dutch across various domains and prompt types? 3. How do unsafe scores differ between English and Dutch for various domains and prompt types in the T2IGenAI models? 4. How do the T2IGenAI models compare in their blocking rates, image-text correspondence scores, and unsafe scores for both English and Dutch prompts? The research focuses on two particularly concerning domains: violence and sexually explicit content. Two T2IGenAI models were evaluated within this research, namely Stable Diffusion (V1-5) and DALLE (3). These models were selected based on their distinct filtering mechanisms, accessibility, and price. Two types of prompts are used, being ’direct’ and ’diluted’ prompts. First, 500 ’direct’ prompts are developed for both domains in English and Dutch. These prompts have very few details that are considered safe. An example is ’A soldier shoots another man through his head’. The diluted sentences are based on the direct prompts that are initially blocked. Contrary to their direct counterpart, diluted prompts have extra, safe, details added to the unsafe direct prompt. An example is ’Amid a landscape draped in the soft hues of dusk, a camouflaged soldier stands still in the dark mud while he shoots an innocent man through his head.’ Using this dilution tactic will provide additional information on filter circumvention strategies. The developed prompts are inserted automatically inserted into these models using the programming software Python. The T2IGenAI model filter performance is evaluated based on its blocking rate, text alignment (imagetext correspondence degree), and unsafe grading. The data from the first two quantitative metrics are 2 3 automatically generated and analyzed within the Python environment. The latter, being qualitative in nature, requires manual annotation of the developed images. These images are graded based on a novel grading scheme, thereby quantifying the annotation process. This research considers a ’weak spot’ when there is a significant difference in blocking rate across language, when this language has a similar or higher text alignment score, and when this language has more unsafe images. The outcomes from all three metrics are synthesized into the following results: • For most domains, it is concluded that although initial blocking rates may be significantly lower across languages, this does not mean that more unsafe content will be generated. • One weak spot is revealed within the sexually explicit domain, where images were generated using the Stable Diffusion model. Here, Dutch blocking rates for diluted prompts were significantly lower (37%) compared to English prompts (75%). The two-sample KS test proved that there was no statistically proven relationship between a lower alignment score for Dutch prompts. Moreover, fewer safe images were generated for Dutch prompts (88.89%) compared to the English diluted prompt (93.06%), albeit the difference is small. • DALL E outperforms Stable Diffusion in terms of blocking rates across all domains and prompt types. Stable Diffusion does not flag violent prompts at all, signified by the 0% blocking rate. For sexually explicit content, DALL E scores 95% (English) and 94% (Dutch), whereas Stable Diffusion only presents blocking rates of 58% and 11% for English and Dutch prompts, respectively. This indicates a less sophisticated initial filter mechanism. • Contrary to the blocking rates, text alignment scores are often lower for DALL E images. Safety scores are slightly better for DALL E within the sexually explicit domain. The violence domain often scores worse compared to Stable Diffusion. • DALL E’s content moderation system has seen improvements since it implemented a prompt rewriting policy. Non-English prompts are first translated into English. Moving forward, DALL E further rewrites the prompt, adding extra details. This is also performed for English prompts. Doing so, they decrease the probability of generating unsafe content, while improving the image aesthetics. Using ’Perspective API’ it was found that often the prompt’s unsafe characteristics indeed did decrease, with the exemption of Dutch violence prompts. This research further builds on existing research regarding T2IGenAI model filter performance. In contrast to previous research, our work makes significant contributions to the scientific landscape by conducting a cross-language analysis that compares the performance of safety filters across English and Dutch prompts, thereby examining potential linguistic variability in AI safety mechanisms. Additionally, the introduction of a novel image grading scheme improves previous research, since the image output is not only analyzed quantitatively but also qualitatively. Furthermore, this study provides critical insights into the impact of prompt rewriting on safety filter performance. While previous work by Hwang et al. (2024) explored DALL-E’s prompt rewriting policy, it only considered English prompts. Furthermore, no research existed that quantitatively evaluated whether the rewritten prompt is actually safer. Our research addresses this gap. The societal relevance of this research lies in its potential to enhance the safety and reliability of text-toimage generative AI models. By identifying, addressing, and informs readers about the weaknesses in current AI safety filters, particularly across different languages and domains, this study can contribute to the development of more robust safeguards against the generation of harmful content. This is crucial in a digital age where AI-generated images can significantly impact public perception and information dissemination. Improved safety mechanisms can help mitigate the risks associated with disinformation, and inappropriate content, thereby fostering a safer and more trustworthy online environment. Keywords: Text-to-Image Generative AI, AI Safety Filters, Stable Diffusion, DALL-E, Content Moderation, Prompt Dilution. ... Read more

The rapid rise of short-format video platforms such as TikTok and YouTube shorts in the last 5 years has fundamentally transformed the way that users consume content. These platforms rely more than any other on recommendation systems to provide content to users. These systems analyze user behaviors on certain types of content to learn their interests and deliver content to users directly, limiting direct user inputs. While this innovative approach to content delivery simplifies the user experience and generally increases user satisfaction, there are concerns that they can lead to an increase in rabbit holes. These rabbit holes are defined as situations where users are recommended increasingly narrow content which can reinforce biases and limit diverse perspectives and content.

The scientific body on recommendation systems has proven time and time again that they generate rabbit holes. However, a gap exists in the understanding how exactly these rabbit holes form specifically on Short-Format Video (SFV) Platforms, such as YouTube Shorts or TikTok. While many studies have attempted to audit recommendation systems there is a lack of methodology to effectively prove the importance of different users actions in both influencing the output of the algorithm as well as the generation of rabbit holes. To address this knowledge gap, the main research question of this study is:

"How can the feature importance of user actions in algorithmic recommendation systems on Short-Form Video Platforms, such as YouTube Shorts, be analyzed to understand their role in leading users down a rabbit hole?"

This research question can be divided into two sub-questions, which when answered will provide a comprehensive understanding to the main research question. These sub-questions are as follows:

1. How can a tool be built to collect a large dataset of algorithmic recommendations based on user actions?
2. How can this dataset be analyzed to understand the key user actions that lead to falling into a rabbit hole on YouTube Shorts?

As seen by the sub questions, this research will focus on distinct phases and objectives. The first will be to develop a tool that is able to fully simulate a human user of YouTube Shorts, the SFV platform of interest in the study, with full user interests, actions and decision making. This will permit the study to then focus on the second phase which will be to run this automation tool with varying user personas each interacting with content in different ways and recording all the output of the algorithm. This tool is novel in that it utilizes Large Language Models (LLMs) prompts in order to understand the content of the videos it is watching and decide how to react (positively or negatively) based on the video classification and the user interests. This will allow the researchers to understand how different user actions influence the algorithm and permit it to learn the user interests. It will also permit the researchers to investigate if rabbit holes form on YouTube Shorts as well as the user actions that intensify them.
The tool is built to collect a time series of video recommendations with two key data variables for each point: the video category and the reaction of the user (positive or negative).

A time series of 550 videos is collected for the watch, like, share, dislike and skip reactions. The runs for the watch and like are ran at varying probabilities of the reaction happening to moderate their influence. The research then analyses these time series by plotting the running average of videos that positively aligned with the user interests as well as the different video categories recommended to the user. These plots reveal which actions are most important in influencing the algorithm and pertaining to user interests as well as how rabbit holes form. These results are presented below:

- Au automated tool to fully simulate users on YouTube Shorts was successfully developed and can now be used perform algorithmic audits. A full explanation of the development and architecture of the bot/tool is provided in the methodology section in order. The tool itself is also provided to the scientific community on GitHub in order for other researchers to perform more audits.
- Among user interactions, watching videos is identified as the most influential in guiding the YouTube Shorts algorithm. Liking also has a significant impact, though less than watching. In contrast, sharing and disliking videos have minimal influence on the algorithm’s output.
- The results also confirms that the YouTube Shorts algorithm has a strong tendency to create rabbit holes by progressively narrowing down content recommendations based on user engagement, particularly through actions like watching and liking videos. It is also shown that the consistency in user actions is key in the intensity of rabbit holes and that to avoid rabbit hole.

From these results, this study aims to advance the understanding of how YouTube Shorts' recommendation algorithms foster algorithmic rabbit holes. The study introduces a novel automated bot for simulating user behavior, offering a new tool for auditing algorithmic processes on short-form video platforms. By providing insights into the mechanics of content personalization, this work contributes to ongoing discussions about algorithm transparency and ethical considerations in digital content curation.

The research on auditing SFV platforms for the rabbit hole effect aligns seamlessly with the overarching theme of the CoSEM program within the Technology Policy Management faculty. By delving into the intricacies of algorithmic recommendation systems on SFV platforms, the study intersects policy and technology within the context of a complex system. The objective is not only to understand the dynamics of these algorithms comprehensively but also to provide a tool for policy makers to analyse and understand the algorithms shaping our world. The results of this thesis can help platform developers optimize algorithms to reduce rabbit holes, guide policymakers in crafting regulations to promote transparency, and inform users on how to engage more critically with content. ... Read more

The Dutch banking sector is mandated to identify and report transactions that may signify money laundering (ML) activities. Banks have been reliant on rule-based transaction monitoring (TM) systems that flag transactions exceeding predefined thresholds. While such systems are instrumental in filtering potential ML transactions, the inherently small prevalence rate of ML occurring in the vast majority of financial transactions causes these systems to produce only a limited number of flagged transactions. Furthermore, as flagged transactions are only those surpassing certain rule thresholds, the alerts are biased toward presumed risk distributions. Consequently, this causes the performance regarding transactions that go unreviewed but should have been flagged, so-called false negatives, to remain unknown. This lack of understanding is a critical gap in the efficacy of current anti-money laundering (AML) controls and motivates the need for better insights into this False Negative Rate (FNR). Addressing this critical need for enhanced discernment of FNR, this study aims to improve this knowledge gap by answering the following research question: ‘To what extent could a supervised machine learning classifier, when trained on historical alerts, assist Dutch banks in estimating the False Negative Rate of rule-based transaction monitoring systems concerning unreviewed transactions?’. To achieve this goal, this study adopts a mixed-methods research design by combining a literature review with seven interviews with domain specialists to acquire insights into transactional ML typologies and the underlying indicators and thresholds employed in existing rule-based TM systems. The study further extended to the development of eight different types of supervised machine learning classifiers, applied both using their default settings as well as with balancing measures in place when possible. These classifiers were trained on two synthetically generated datasets of both 180 million transactions, one with a high and one with a low ML prevalence rate, indicating the relative frequency of ML transactions. These mirrored real transactional patterns in order to evaluate the feasibility of estimating the FNR pertaining to unreviewed transactions utilizing historical data on flagged transactions. In addition to establishing the effect of both different ML prevalence rates on performance, we also explored whether combining data from multiple financial institutions into one shared information perspective could be of additional value. The findings indicate that the classifiers struggle to accurately predict the FNR, especially in scenarios of low ML prevalence and without combining information from multiple institutions. There is a significant discrepancy between the actual (0.729 to 0.988) and predicted FNRs (0.156 to 0.649), even in higher ML prevalence settings. The low performance, as evidenced by poor Area Under Precision-Recall Curve (AUPRC) and Matthews Correlation Coefficient (MCC) scores, highlights the challenges in using machine learning for ML transaction detection in Dutch banking, calling for further research and development of more advanced detection models Despite the restrained success in accurately estimating FNR through supervised machine learning classifiers, the insights derived from this research are of considerable value. They prompt a critical examination of the current TM systems and suggest a pivot toward more sophisticated machine learning techniques for FNR estimation... ... Read more