Using explanations to improve subjective experience of control in situations of delayed control effects

Abstract

Problem Statement. The responsibility problem in the AI field has been taken seriously. The point of the problem is how to prevent AI systems from making moral decisions for whom they cannot be held accountable. One of the directions to address the responsibility problem is Meaningful Human Control (MHC). Most of the work focused on theoretical definition and measurement ex- plorations and few researchers investigate it by combining application scenarios. Research Question. One of the previous works identified that explanations could be used to improve the subjective feeling of MHC when the effect of con- trol is delayed. Therefore, we wanted to study what type of explanations could facilitate the improvement and how explanations achieved that. Method. We conducted an expert study to obtain advice on selecting explanation types by a ranking question questionnaire and structured questions. Due to the findings that information sharing could improve Situation Awareness (SA), we conducted a pilot study and a user study to study the effect of explanations on different levels of SA and the human feeling of MHC. We used the Situation Awareness Global Assessment Technique (SAGAT) and a Five-point Likert scale question- naire to measure the effect of explanations on SA and MHC. Moreover, we compared the effect of sub-explanations we used in the user study (consequen- tial and counterfactual explanations) with a questionnaire. Results. It was shown that the explanations help to get more overall SA and SA in the pro- jection level. It was also shown that higher SA scores are associated with a better feeling of MHC. However, our result showed that the explanations have no significant effect on the subjective experience of MHC. The findings also in- dicated that a high frequency of playing computer games can result in a good subjective experience of control. Comparisons of the sub-explanations on quali- tative and quantitative analysis demonstrated that counterfactual explanations made a better impression on participants in most respects. Discussion and Conclusion. From the results, we concluded that explanations do increase the degree of SA of the task to a certain extent, but they do not affect the experi- enced control. We also found that the computer gaming experience may provide higher cooperation engagement and cohesion, resulting in increased experienced control. As for the sub-explanations, the counterfactual ones were overall bet- ter than the consequential ones by providing more information and making the participants feel the robot’s intelligence.