Print Email Facebook Twitter Survey of Affect Representation Schemes used in Vision-Based Automatic Affect Prediction Title Survey of Affect Representation Schemes used in Vision-Based Automatic Affect Prediction: A Systematic Literature Review Author Serrano Ruber, Lucia (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Dudzik, B.J.W. (mentor) Raman, C.A. (mentor) Liem, C.C.S. (graduation committee) Degree granting institution Delft University of Technology Programme Computer Science and Engineering Project CSE3000 Research Project Date 2023-06-28 Abstract In human-human interactions, the majority of information is conveyed through body language, specifically facial expressions. Consequently, researchers have been interested in improving human-computer interactions through developing systems with automatic understanding of body language and facial expressions. This technology is especially useful due to its broad range of applications in fields such as healthcare, education, and safety & security. Vision-based automatic affect recognition (AAR) systems aim to predict a subject’s affective state based on visual input such as image or video. These systems analyze and classify subjects’ facial expressions and body language using affect representation schemes (ARS), most often classified as either categorical or dimensional. This paper explores the current state of ARS used in vision-based AAR through a systematic literature review following PRISMA guidelines. We selected 53 papers from WebOfScience according to our eligibility criteria which included computer science papers written in English proposing a vision-based AAR system targeting single subjects, and excluded studies dealing exclusively with micro-expressions or group affect recognition. Additionally, given the time limitation imposed on this research we excluded papers that were not readily accessible with our TU Delft license, used multimodal input, or did not use a dataset included in our predefined list. For this exploration we specifically look at the schemes used, the popularity and trends of usage, motivations, and psychological basis. From the 53 reviewed papers, all of the papers target utilitarian emotions using at least one discrete ARS. The most commonly used schemes classify affective states into happiness, sadness, fear, anger, surprise, and disgust. While the majority of papers are lacking in providing explicit reasoning for their choices, most ARS are based grounded in psychological theories. Our results show an established norm within this area of research. However, they also evidence a lack of displayed critical thought in the selection of schemes. This oversight limits potential for future AAR research. Subject Affect Representation SchemeAutomatic Affect PredictionSystematic Literature Review To reference this document use: http://resolver.tudelft.nl/uuid:a7e4e5b5-af49-4080-b3ce-577e59e27993 Part of collection Student theses Document type bachelor thesis Rights © 2023 Lucia Serrano Ruber Files PDF CSE3000_Final_Paper.pdf 801.7 KB Close viewer /islandora/object/uuid:a7e4e5b5-af49-4080-b3ce-577e59e27993/datastream/OBJ/view