Influence of Data Processing on the Algorithm Fairness vs. Accuracy Trade-off

Abstract

Algorithmic bias due to training from biased data is a widespread issue. Bias mitigation techniques such as fairness-oriented data pre-, in-, and post-processing can help but usually come at the cost of model accuracy. For this contribution, we first conducted a literature review to get a better insight into the potential trade-offs. We followed by implementing a Python program to test how the Disparate Impact Remover (DIR) pre-processing and Reject Option Classification (ROC) post-processing techniques impacted the fairness and accuracy metric values of a Logistic Regression model trained on data from the Adult Income dataset. The implementation also allows for building Pareto fronts that trade off fairness and accuracy metrics of choice, thus offering a blend of perspectives on fairness. Our findings give insight into how combined fairness methods influence the trade-off, but our implementation can be extended to explore such trade-offs using other datasets, models, and fairness methods.