S.E. Carter
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
7 records found
1
Large language models (LLMs) are widely used tools that assist us by answering various questions. Humans implicitly use contrast as a natural way to think about and seek explanations (i.e., "Why A and not B?"). Explainability is a challenging aspect of LLMs, as we do not truly understand how good the LLM answers are. The challenge is understanding to what extent LLMs can generate effective contrastive self-explanations for users. We introduce the Contrastive Self-Explanation Method (CoSEM) to narrow the gap between LLMs and explainability. It generates contrastive self- explanations and evaluates them through automation and a user study on generality, usefulness, readability, and relevance. Our results indicate that LLMs are capable of generating effective contrastive self-explanations. Lexical analysis of contrastive explanation indicates that explanations are not less general than the text those explain, and semantic analysis shows that more complex models generalize self-explanations more consistently. Although it is challenging to evaluate contrast in self-explanations semantically, user study shows that some models (Llama3-8B) help understand the contrast. Moreover, task selection affects how readable users find the explanations, where more self-explanations on general topics (movie reviews) are more readable than more specific topics (medical diagnoses). Lastly, some models, such as Llama3-8B, excel at generating contrastive self-explanations that contain relevant information regarding input text.
...
Large language models (LLMs) are widely used tools that assist us by answering various questions. Humans implicitly use contrast as a natural way to think about and seek explanations (i.e., "Why A and not B?"). Explainability is a challenging aspect of LLMs, as we do not truly understand how good the LLM answers are. The challenge is understanding to what extent LLMs can generate effective contrastive self-explanations for users. We introduce the Contrastive Self-Explanation Method (CoSEM) to narrow the gap between LLMs and explainability. It generates contrastive self- explanations and evaluates them through automation and a user study on generality, usefulness, readability, and relevance. Our results indicate that LLMs are capable of generating effective contrastive self-explanations. Lexical analysis of contrastive explanation indicates that explanations are not less general than the text those explain, and semantic analysis shows that more complex models generalize self-explanations more consistently. Although it is challenging to evaluate contrast in self-explanations semantically, user study shows that some models (Llama3-8B) help understand the contrast. Moreover, task selection affects how readable users find the explanations, where more self-explanations on general topics (movie reviews) are more readable than more specific topics (medical diagnoses). Lastly, some models, such as Llama3-8B, excel at generating contrastive self-explanations that contain relevant information regarding input text.
Machine Learning (ML) algorithms have the potential to reproduce biases that already exist in society, a fact that leads to scholarly work trying to quantify algorithmic discrimination through fairness metrics. Although there are now a plethora of metrics, some of them are even contradictory, making fairness become a problem of knowing which measurement to choose over another. Consequently, scholars began considering that fairness should be discussed by placing algorithms in their social contexts. Since (1) these social aspects are related to structures of discrimination and (2) feminism aims to criticise discrimination against the marginalised, I introduce the possibility of analysing the social context of ML algorithms through a feminist lens. By doing this, I highlight social and political aspects that are equally important to consider for a faithful discussion on fairness: corporate lobbying, the lack of diverse hiring which leads to fairness discussions that do not consider the experiences of marginalised groups and, lastly, the broader context that an algorithm is used in. Moreover, I emphasise how feminist ethics of care constitute an essential framework for a conversation about actually implementable fairness solutions, since it shows the need to listen to both the marginalised community and to the developers who might want to build fairer ML but currently cannot. Having built a bridge between the hegemony and the feminist camp, I highlight how Northpointe’s (now Equivant’s) Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) algorithm can be considered biased against black people. Through this, I illustrate how feminist considerations bring clarity to fairness debates by helping choose a fairness metric or by claiming that an algorithm is unfair by nature and should be abolished. To follow, I use the same feminist critiques to draw attention to the possible weak points of current sociotechnical solutions. For instance, the EU AI Act risks being too susceptible to company lobbying, leading to not strict enough regulations. Furthermore, the AI committee should ensure that they hire a diverse group of people in order to develop regulations that positively consider all marginalised groups. Lastly, I highlight how ethics education is essential for creating a new generation of responsible engineers. Considering this, I emphasise the urgency of making ethics courses at TU Delft (and not only) more interdisciplinary by interacting more with critique points coming from the social sciences. This will open up possibilities for more research tackling fairness from a multitude of perspectives.
...
Machine Learning (ML) algorithms have the potential to reproduce biases that already exist in society, a fact that leads to scholarly work trying to quantify algorithmic discrimination through fairness metrics. Although there are now a plethora of metrics, some of them are even contradictory, making fairness become a problem of knowing which measurement to choose over another. Consequently, scholars began considering that fairness should be discussed by placing algorithms in their social contexts. Since (1) these social aspects are related to structures of discrimination and (2) feminism aims to criticise discrimination against the marginalised, I introduce the possibility of analysing the social context of ML algorithms through a feminist lens. By doing this, I highlight social and political aspects that are equally important to consider for a faithful discussion on fairness: corporate lobbying, the lack of diverse hiring which leads to fairness discussions that do not consider the experiences of marginalised groups and, lastly, the broader context that an algorithm is used in. Moreover, I emphasise how feminist ethics of care constitute an essential framework for a conversation about actually implementable fairness solutions, since it shows the need to listen to both the marginalised community and to the developers who might want to build fairer ML but currently cannot. Having built a bridge between the hegemony and the feminist camp, I highlight how Northpointe’s (now Equivant’s) Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) algorithm can be considered biased against black people. Through this, I illustrate how feminist considerations bring clarity to fairness debates by helping choose a fairness metric or by claiming that an algorithm is unfair by nature and should be abolished. To follow, I use the same feminist critiques to draw attention to the possible weak points of current sociotechnical solutions. For instance, the EU AI Act risks being too susceptible to company lobbying, leading to not strict enough regulations. Furthermore, the AI committee should ensure that they hire a diverse group of people in order to develop regulations that positively consider all marginalised groups. Lastly, I highlight how ethics education is essential for creating a new generation of responsible engineers. Considering this, I emphasise the urgency of making ethics courses at TU Delft (and not only) more interdisciplinary by interacting more with critique points coming from the social sciences. This will open up possibilities for more research tackling fairness from a multitude of perspectives.
Algorithmic Fairness: Encouraging Exclusionary Diversity
(instead of Inclusionary Pluriversality)
AI is becoming significantly more impactful in society, especially with regard to decision-making. Algorithmic fairness is the field wherein the fairness of an AI algorithm is defined, subsequently evaluated, and ideally improved. This paper uses a fairness decision tree to critique certain notions of algorithmic fairness through a postcolonial lens by applying Gayatri Spivak's theory of the subaltern alongside other postcolonial principles. A definition and criteria for a subaltern population in AI are provided, depicting that AI and algorithmic fairness rely on subaltern marginalization, silence, and faux inclusion. A theoretical case analysis is then conducted to illustrate how demographic parity, even in cases where it is the best fairness metric, does not include the subaltern. Algorithmic fairness often defines fairness through neoliberalism, assigning a ``cost'' to ethical considerations, wherein morality is second to profits and utility. Furthermore, a large proportion of the ``justice'' conducted through AI is surface level and may actually cause more harm in the long run. A proposal is made to seriously consider not using any AI in socially relevant, complex situations.
...
AI is becoming significantly more impactful in society, especially with regard to decision-making. Algorithmic fairness is the field wherein the fairness of an AI algorithm is defined, subsequently evaluated, and ideally improved. This paper uses a fairness decision tree to critique certain notions of algorithmic fairness through a postcolonial lens by applying Gayatri Spivak's theory of the subaltern alongside other postcolonial principles. A definition and criteria for a subaltern population in AI are provided, depicting that AI and algorithmic fairness rely on subaltern marginalization, silence, and faux inclusion. A theoretical case analysis is then conducted to illustrate how demographic parity, even in cases where it is the best fairness metric, does not include the subaltern. Algorithmic fairness often defines fairness through neoliberalism, assigning a ``cost'' to ethical considerations, wherein morality is second to profits and utility. Furthermore, a large proportion of the ``justice'' conducted through AI is surface level and may actually cause more harm in the long run. A proposal is made to seriously consider not using any AI in socially relevant, complex situations.
A study on bias against women in recruitment algorithms
Surveying the fairness literature in the search for a solution
Algorithms have a more prominent presence than ever in the domain of recruitment. Many different tasks ranging from finding candidates to scanning resumes are handled more and more by algorithms and less by humans. Automating these tasks has led to bias being exhibited towards different unprivileged groups, among these women. This has prompted a need to find solutions to this bias in order to achieve fairness in algorithms for everyone. This survey analyses the state of the literature on fairness and bias against women with a focus on recruitment algorithms. It has been found that a plethora of methods to achieve and measure fairness exist, with many of the technical methods having only been tested in a controlled environment and not in a production environment. Companies are not very forthcoming in sharing how they ensure fairness, which complicates the development of the field. There are many limitations to the current methods, however due to widespread usage of algorithms in the recruitment process, there is still a need for solutions to exist while these critiques are being addressed. It is vital for fairness to not only consider technical solutions, but also social solutions and to be aware of the limitations of each approach. Solving the issue of bias against women in algorithms requires sometimes realising that the best solution does not involve an algorithm at all, or in the case where an algorithm is applied, a critical engineer that is aware of the limitations and possibilities of different methods to achieve fairness. Future work is in addressing some major critiques of the current fairness literature and reducing the bias in society that often leads to algorithmic bias.
...
Algorithms have a more prominent presence than ever in the domain of recruitment. Many different tasks ranging from finding candidates to scanning resumes are handled more and more by algorithms and less by humans. Automating these tasks has led to bias being exhibited towards different unprivileged groups, among these women. This has prompted a need to find solutions to this bias in order to achieve fairness in algorithms for everyone. This survey analyses the state of the literature on fairness and bias against women with a focus on recruitment algorithms. It has been found that a plethora of methods to achieve and measure fairness exist, with many of the technical methods having only been tested in a controlled environment and not in a production environment. Companies are not very forthcoming in sharing how they ensure fairness, which complicates the development of the field. There are many limitations to the current methods, however due to widespread usage of algorithms in the recruitment process, there is still a need for solutions to exist while these critiques are being addressed. It is vital for fairness to not only consider technical solutions, but also social solutions and to be aware of the limitations of each approach. Solving the issue of bias against women in algorithms requires sometimes realising that the best solution does not involve an algorithm at all, or in the case where an algorithm is applied, a critical engineer that is aware of the limitations and possibilities of different methods to achieve fairness. Future work is in addressing some major critiques of the current fairness literature and reducing the bias in society that often leads to algorithmic bias.
Influence of Data Processing on the Algorithm Fairness vs. Accuracy Trade-off
Building Pareto Fronts for Equitable Algorithmic Decisions
Algorithmic bias due to training from biased data is a widespread issue. Bias mitigation techniques such as fairness-oriented data pre-, in-, and post-processing can help but usually come at the cost of model accuracy. For this contribution, we first conducted a literature review to get a better insight into the potential trade-offs. We followed by implementing a Python program to test how the Disparate Impact Remover (DIR) pre-processing and Reject Option Classification (ROC) post-processing techniques impacted the fairness and accuracy metric values of a Logistic Regression model trained on data from the Adult Income dataset. The implementation also allows for building Pareto fronts that trade off fairness and accuracy metrics of choice, thus offering a blend of perspectives on fairness. Our findings give insight into how combined fairness methods influence the trade-off, but our implementation can be extended to explore such trade-offs using other datasets, models, and fairness methods.
...
Algorithmic bias due to training from biased data is a widespread issue. Bias mitigation techniques such as fairness-oriented data pre-, in-, and post-processing can help but usually come at the cost of model accuracy. For this contribution, we first conducted a literature review to get a better insight into the potential trade-offs. We followed by implementing a Python program to test how the Disparate Impact Remover (DIR) pre-processing and Reject Option Classification (ROC) post-processing techniques impacted the fairness and accuracy metric values of a Logistic Regression model trained on data from the Adult Income dataset. The implementation also allows for building Pareto fronts that trade off fairness and accuracy metrics of choice, thus offering a blend of perspectives on fairness. Our findings give insight into how combined fairness methods influence the trade-off, but our implementation can be extended to explore such trade-offs using other datasets, models, and fairness methods.
From Data to Decision
Investigating Bias Amplification in Decision-Making Algorithms
This research investigates how biases in datasets influence the outputs of decision-making algorithms, specifically whether these biases are merely reflected or further amplified by the algorithms. Using the Adult/Census Income dataset from the UCI Machine Learning Repository, the research explores biases through the lens of three machine learning models: Logistic Regression, Decision Tree, and Random Forest. The analysis reveals that all models exhibit varying degrees of bias, dependent on the fairness metrics applied: Demographic Parity, Disparate Impact, Equal Opportunity, Equalized Odds. It has been found that higher accuracy does not necessarily equate to fairness. The findings emphasize the complex nature of algorithmic bias and the challenge of achieving fairness in automated decision-making systems. This research contributes to the understanding of bias amplification in algorithms and underscores the need for continued efforts to develop fairer decision-making systems in various sectors.
...
This research investigates how biases in datasets influence the outputs of decision-making algorithms, specifically whether these biases are merely reflected or further amplified by the algorithms. Using the Adult/Census Income dataset from the UCI Machine Learning Repository, the research explores biases through the lens of three machine learning models: Logistic Regression, Decision Tree, and Random Forest. The analysis reveals that all models exhibit varying degrees of bias, dependent on the fairness metrics applied: Demographic Parity, Disparate Impact, Equal Opportunity, Equalized Odds. It has been found that higher accuracy does not necessarily equate to fairness. The findings emphasize the complex nature of algorithmic bias and the challenge of achieving fairness in automated decision-making systems. This research contributes to the understanding of bias amplification in algorithms and underscores the need for continued efforts to develop fairer decision-making systems in various sectors.