Social robots are increasingly deployed in fields such as health care and education to support users through social interactions. Nonetheless, these robots mostly rely on black-box machine learning methods that lack awareness of the mental states of their users, which often leads to unnatural behavior. To address this, we propose three model-based techniques for real-time estimation of invisible mental states of humans. Each method adapts the extended Kalman filter and incorporates a validated dynamic model of human mental states. These mental state estimators are designed for human-robot social interactions and personalize their parameters using initial user data. When tested with 10 human participants interacting with a NAO robot, the mental state estimators reduced the average error in estimation and prediction of mental states across all participants by 3% (i.e., from 12% to 9%), with improvements of up to 13% for individual participants. These results demonstrate the potential of integrating such state estimators into the behavioral control systems of social robots to enhance their awareness of the mental states of users.

Interactive machines should establish and maintain meaningful social interactions with humans. Thus, they need to understand and predict the mental states and actions of humans. Based on Theory of Mind (ToM), in order to understand and interact with each other, humans develop cognitive models of one another. Our main goal is to provide a mathematical framework based on ToM to improve the understanding of interactive machines regarding the perception, cognition, and decision-making of humans. Most state-of-the-art models of behavioral theories based on machine learning are focused on input-output black-box representations. Thus, they lack transparency and generalizability, and exhaustive training procedures are needed to personalize them for various humans. Moreover, these models lack dynamics, i.e., they do not mathematically describe the evolution of the mental states and actions of humans in time. Following a systems-and-control-theoretic point-of-view, we represent for the first time the perception, cognition, and decision-making of humans via a dynamic, mathematical framework by introducing a novel formalization and an extension to Fuzzy Cognitive Maps (FCMs). The resulting models are given in a general state-space representation, which can be used by interactive machines within known model-based state estimation and control methods. In a case study, the resulting models were identified and validated for 21 participants, in scenarios where predicting the intentions and behavior of the participants required understanding the dynamics of their mental procedures. The results of these experiments show that our model is capable of incorporating the dynamics to estimate the intentions and predict the behavior of the participants, with an accuracy of, respectively, 81.55% and 66.06%. Moreover, we compared our model with a state-of-the-art formalization of human cognition, which was made dynamic using our introduced FCM framework. Our model, which in addition to the elements of the state-of-the-art model included emotions, personality traits, and biases (thus providing a more transparent insight about the mental procedures of the participants) showed 6.25% and 2.45% more accuracy in, respectively, estimating the intentions and predicting the behavior of the participants.