Machine Learning models are begin increasingly used within the industry such as by financial institutions, governments and commercial companies. In the past few years, there have been several incidents where these ML models show discriminatory behavior towards particular groups of people, leading to unfair decisions that can have negative impacts on the lives of these people. Therefore, eliminating bias and ensuring fairness within these models is crucial to the societal expectations these institutions would like to meet. While there are tools and research towards technical mitigation methods for bias and unfairness. There is a lack of focus on the process of implementing these tools and methods within the industry, to develop ML models with the consideration of fairness at an early stage. In particular, there is a lack of specification on what fairness goals and objectives the ML model should accomplish. Without having this clarity, industry stakeholders can apply the tools and algorithmic unfairness mitigation methods but if the fairness requirements are not defined, then a) one is still not sure whether the ML model is solving the correct fairness goals and b) one is still not sure what the trade-offs and feasibility within different fairness goals, and available resources may look like. To address this, we design a Multi-Level Fairness Framework (digital workflow) that aims towards supporting stakeholders within the industry to perform Requirements Engineering, specifically elicitation and modeling, for fairness in a Machine Learning Model.

In recent years financial fraud has seen substantial growth due to the advent of electronic financial services opening many doors for fraudsters. Consequently, the industry of fraud detection has seen a significant growth in scale, but moves slowly in comparison to the ever-changing nature of fraudulent behavior. As the monetary losses associated with financial fraud continue to grow, so does the need for efficient automated decision making systems. Simple decision making rules are often still the industry standard and only show decent results in the short-term, as reverse-engineering such rules is an easy task for smart fraudsters. Supervised learning systems as automated fraud detectors have shown promising results across the field, but are plagued by challenges uniquely prevalent in the field. Disproportional class imbalance in fraudulent transactions, as well as fraudsters continually adopting new schemes make training robust and generally applicable machine learning models an arduous task. This work introduces a novel machine learning pipeline, which makes use of carefully selected synthetic samples of this minority class to augment the training dataset of the supervised model. Synthetic samples representing fraudulent transactions are filtered based on a novel technique to quantify their expected performance as an adversarial example, using both data-driven and human-expert-driven techniques. By providing the supervised model with high-quality synthetic adversarial examples, we aim to improve its generalizability to never-seen-before fraudulent behavior and, in turn, improve its robustness to the volatile nature of financial fraud. Our results show that weakly-supervised models trained on our augmented datasets are able to detect 7% more fraudulent transactions compared to a baseline model trained on the standard dataset, at the cost of a 1% increase in false positives. Our calculations further show that applying this system could lead to a decrease of 1/6 in monetary losses incurred by financial fraud.

The increasing dangers of unfairness in machine learning (ML) are becoming a frequent subject of discussion, both, in academia and popular media. Recent literature focused on introducing and assessing algorithmic solutions to bias in ML. However, there is a disconnect between these solutions and practitioners' needs. By interviewing 30 ML practitioners of diverse backgrounds across 16 countries, and presenting them with a simulated use case, our study aims to investigate common fairness practices among professionals and how these are influenced by their backgrounds. The results reveal a superlative disparity among academia and industry practitioners. We also identify different practices in fairness and data exploration stages, influenced by the educational background as well as the level of experience of practitioners. Our study also finds how demographics have an impact on several aspects, such as willingness to accept and support legal actions taken against ML discrimination. In accordance with our findings, we suggest several actions that can be taken to improve fairness solutions, and we also highlight future directions for fairness research that can cause a positive impact on the way fairness is perceived by practitioners.

To study how to involve the end-users in the development of machine learning explainability, this project has chosen the context of bird species identification. It intends to develop a platform where the end-users can learn bird knowledge while contributing to building the explainability of machine learning models. Among all the methods that equips machine learning models with explainability, this project adopts a framework called SECA (Semantic Concept Extraction and Analysis). In this framework, we require human-made-annotations to be made to the saliency maps of training photos to provide semantically understandable explanations to the end-users. On the other hand, we hope that the process of making annotations will also benefit the human annotators’ skills in bird species identification, in order to motivate their participation. Two main goals of the user research were: to understand the users’ needs for learning and to know their capability in making the annotations needed by the project owners. The user research started with qualitative and quantitative research to understand the current practices of the bird hobbyists, to define the target user groups, which were the birders with zero or little expertise. Then, in order to link their learning needs to the capability of machine learning explanations, three prototypes were built to collect their feedback. It was found out that they didn’t care much about the justification or transparency of bird ID apps, compared to learning knowledge in distinguishing birds. Then came the annotation test when we found the participants were able to finish the annotation task with high correctness (>93% on average). And the most popular annotations of each task were 100% correct. Finally, we built a functional, high-fidelity prototype with experiential interfaces and interactions, and tested it among 3 of the target users. They had positive feedback on the prototype’s usability and the overall workflow, which proved the feasibility of our concepts. Recommendations on usability were drawn at the end of this test. Throughout the research and design phases in this project, we have developed an approach to involve end-users in the annotation process of an explainable bird species identification model for their own benefit of fun and learning, which could potentially be applied to broader deployments

Machine learning is still one of the most rapidly growing fields, and is used in a variety of different sectors such as education, healthcare, financial modeling etc(Jordan and Mitchell 2015). However, along with this demand for machine learning algorithms, there comes a need for ensuring that these algorithms are fair and contain little to no bias. Tools like Fairlearn and AI Fairness 360(AIF360) allows developers and data scientists to examine their codebase according to specified fairness metrics and mitigate any fairness related issues. This study aims to determine how practitioners use the separate toolkits and whether their practices are differ by the toolkit they choose to use. To do this, we conducted 29 think-aloud interviews with industry practitioners to understand how they would use Fairlearn and AIF360 in practice. The results show that fairness is a socio-technical challenge. While the toolkit does allow for participants to be structured in their approach, and raise awareness for fairness related harms, at the end of the day the fairness toolkit only provides technical help to find harms that the individual was already aware about. Based on the findings, we then suggest the design for a fairness toolkit that can help practitioners approach fairness in the most ideal manner. This toolkit would include a way to have interdisciplinary collaboration, have a larger focus on explainability, and give clear guidance to its users regarding fairness related harms.

Machine learning can still make harmful mistakes. A solution would be tacit knowledge. Machine learning needs this type of knowledge to improve. An example of such knowledge that can help make the system draw better logical conclusions would be: if presented with an open fridge, then it could deduct that the food will go bad. Tacit knowledge or common-sense knowledge refers to the type of knowledge which is acquired through experience, the kind only humans can create. GWAPs (game with a purpose) have shown quite promising results for acquiring such knowledge. Unfortunately, it could still contain errors due to users who only want to harm the game data, etc. and there is no method for validating such knowledge without involving humans somehow. Therefore, from the previously stated problem, our goal has emerged - develop a method for validating an existing data set and for later training machine learning models using a GWAP. There has been work done before using GWAPs to elicit such information, yet they are limited in the sense that their main focus is set on data collection, not validation. Since very few projects looked into it, we decided to investigate a new GWAP which has as main purpose tacit knowledge validation. The main question which we aim to answer is "How can we elicit and validate tacit knowledge using a game with the following settings: single-player, textual concepts, goal: associate words with their concepts." The game presents hints to the users and they have to guess, as fast as possible and with the least amount of tries as possible, which answer is correct from the 6 options that are provided. The evaluation of the game will be made using standard metrics such as games played, time spent playing, number of users, etc. The conclusion is that the GWAP, even with the lack of data, was quite capable of analyzing the quality of the data set and reached a conclusion that is easily confirmed by a mere look over the initial data set.

The manual process of collecting and labelling data required for machine learning tasks is labour-intensive, expensive, and time consuming. In the past, efforts have been made to crowdsource this data by either offering people monetary incentives, or by using a gamified approach where users contribute to databases as a side-effect of playing an enjoyable game. However, most of these efforts focus on using a competitive setting to incentivize players. This sometimes results in users spamming the dataset for personal gains. Research is lacking in how a collaborative setup, where players work together to make decisions by consensus, can be used to source knowledge that is more accurate and reliable. This paper describes the design and evaluation of SceneFinder, a game that aims to crowdsource reliable and diverse textual data about scenes (such as rooms, parks, monuments, etc) and the tacit knowledge relevant to them, such as information about their contents, their purpose and their surroundings. SceneFinder makes use of a collaborative setup that elicits a relevance based ranking of facts about these scenes, that distinguishes it from existing games in the field.

Despite the ever-growing advances in artificial intelligence (AI), common sense acquisition and reasoning is still comparingly in their early stages to other fields in AI. To further advance this field, it is necessary to collect large amounts of common sense facts or tacit knowledge to train such AI models. One effective way is to use crowdsourcing and games to make the collection process both widely available and fun at the same time. Currently, there are no tools to collect discriminative tacit knowledge efficiently and accurately. In this work, we propose FindItOut: a crowdsourcing tacit knowledge elicitation solution with multiplayer gamification and images to collect general-purpose and discriminative tacit knowledge. We show that using FindItOut, we can both efficiently and accurately collect discriminative tacit knowledge while also being fun and engaging.

The ability to identify and mitigate various risks and harms of using Machine Learning models in industry is an essential task. Specifically because these may produce harmful outcomes for stakeholders, including unfair or discriminatory results. Due to this there has been substantial research into the concepts of fairness and its metrics, bias and its mitigation, and algorithmic harms and their sources. Various toolkits have been created to guide practitioners to reflect on these topics and provide suggestions on algorithmic solutions to mitigate these risks. However, it is not yet known how widely these toolkits are used and how they are perceived in terms of usefulness. In this project, practitioners were interviewed in order to determine to what extend do envisioned practices of practitioners without experience with fairness toolkits differ from those with the experience. The two toolkits considered were the IBM AI Fairness360 and Microsoft FairLearn. The data collected from the interviews suggests that there could be fewer differences in practices of practitioners with experience and without experience with toolkits, than those with training or work roles in ethics and fairness in ML and those without. This suggests that experience the toolkit itself is not indicative of a more thorough approach to identifying and mitigating harms in fair Machine Learning.

This thesis looks at how to characterize weaknesses in machine learning models that are used for detecting privacy-sensitive data in images with the help of crowdsourcing. Before we can come up with a method to achieve a goal, we first need to make clear what we consider privacy-sensitive data. We took the General Data Protection Regulation (GDPR) as a starting point, and performed a crowdsourcing task to see how workers interpret this regulation. Interpreting legal texts can be difficult, there is room for interpretation and the perception of a legal text can change over time. Therefore, we need to take the input of the crowd, next to our own input, to operationalize this regulation to use in this context. Next, we took a machine learning model for detecting privacy-sensitive data in images in order to retrieve saliency maps, which helps us with explaining the inner-working of the model. Subsequently, the saliency maps are inspected through a crowdsourcing task, with the established privacy definition, to find out the strengths and weaknesses. From the results, we see that crowd workers can be efficiently used to find the strengths and weaknesses of a machine learning model, while keeping the privacy definition in mind. Workers are able to consistently apply their views about privacy across different images, whilst also increasing the trust people have in the machine learning model. This shows us that we can use crowdsourcing efficiently in a fairly difficult context of privacy, and paves the way for a more sophisticated approach to privacy-sensitive elements in images, and even for contexts other than privacy.

Tacit knowledge, unlike explicit knowledge, is not easily codifiable, yet important for machine learning models. This research explores a method to gather tacit knowledge about humor using a simple text-based party game, building on the existing idea of using games to gather tacit knowledge from crowds of people. Players propose prompts, which will then be answered by other players. They will then vote to determine which of the two answers to each prompt is the funniest. The engagement of the players with the game is measured and tacit knowledge is obtained from the jokes. With a large and diverse enough group of participants across games, a variety of tacit knowledge can be extracted.