Hate speech detection on social media platforms remains a challenging task. Manual moderation by humans is the most reliable but infeasible, and machine learning models for detecting hate speech are scalable but unreliable as they often perform poorly on unseen data. Therefore, human-AI collaborative systems, in which we combine the strengths of humans' reliability and the scalability of machine learning, offer great potential for detecting hate speech. While methods for task handover in human-AI collaboration exist that consider the costs of incorrect predictions, insufficient attention has been paid to estimating these costs. In this work, we propose a value-sensitive rejector that automatically rejects machine learning predictions when the prediction's confidence is too low by taking into account the users' perception regarding different types of machine learning predictions. We conducted a crowdsourced survey study with 160 participants to evaluate their perception of correct, incorrect and rejected predictions in the context of hate speech detection. We introduce magnitude estimation, an unbounded scale, as the preferred method for measuring user perception of machine predictions. The results show that we can use magnitude estimation reliably for measuring the users' perception. We integrate the user-perceived values into the value-sensitive rejector and apply the rejector to several state-of-the-art hate speech detection models. The results show that the value-sensitive rejector can help us to determine when to accept or reject predictions to achieve optimal model value. Furthermore, the results show that the best model can be different when optimizing model value compared to optimizing more widely used metrics, such as accuracy. ... Read more

Governments require companies to be able to explain where their data is coming from and going to. Our client helps these companies by creating maps of their data landscapes. This is the concept of data lineage. There are various issues that arise in the workflow of figuring out and building data lineage diagrams. Our contributions here are providing a model for what data lineage diagram instances are, and an interactive web application that can be used to visualize and edit these diagrams in an intuitive way. One of the core challenges of this project has been to combine the client's business perspective with our knowledge of computer science. Starting with our research by figuring out the client's use cases, analysing them for their feasibility within the constraints of the project. The team used both Scrum and an agile approach to develop a product that matches the client's expectations and needs throughout the project. The product was tested by adhering to the five metrics defined by the Consortium for IT Software Quality. The final product contains the desired functionality and allows building data lineage diagrams using company data from the client through external APIs. The client is eager to use the product and has provided additional opportunities for the team to work further on the product. ... Read more