Despite advancements in socially aware navigation, robots still often behave inappropriately in social environments. To ensure successful application, robots must detect the human perceived appropriateness of their navigation behaviors. This paper presents a novel dataset covering a complete range of perceived appropriateness and uniquely incorporates human emotion and attention to facilitate the detection of perceived appropriateness of robot social navigation in pathways (PARSNiP). It is created based on a series of human-robot interaction experiments with 30 participants and a mobile robot. Several typical machine learning models are utilized to evaluate the dataset and analyze the contributions of different features in detecting perceived appropriateness. The results indicate that incorporating emotional and attentional features can significantly improve the accuracy of perceived appropriateness detection. There was an increase from 63% to 68% using algorithm-predicted emotional and attentional features, and a further increase to 79% with the emotion and attention data reported by the participants. With the dataset, researchers could train machine learning models to enable robots to detect perceived appropriateness accurately, fostering adaptations that improve their responsiveness and accuracy in social interactions. The dataset is available for download at https://github.com/duibcuiegiosahxois/PARSNiP.git, and videos will be shared upon request by contacting Y.Zhou-13@tudelft.nl.

Technology-enhanced learning systems, specifically multimodal learning technologies, use sensors to collect data from multiple modalities to provide personalized learning support beyond traditional learning settings. However, many studies surrounding such multimodal learning systems mostly focus on technical aspects concerning data collection and exploitation and therefore overlook theoretical and instructional design aspects such as feedback design in multimodal settings. This paper explores multimodal learning systems as a critical part of technology-enhanced learning used for capturing and analyzing the learning process to exploit the collected multimodal data to generate feedback in multimodal settings. By investigating various studies, we aim to reveal the roles of multimodality in technology-enhanced learning across various learning domains. Our scoping review outlines the conceptual landscape of multimodal learning systems, identifies potential gaps, and provides new perspectives on adaptive multimodal system design: intertwining learning data for meaningful insights into learning, designing effective feedback, and implementing them in diverse learning domains.

Applicability testing of constructive computational mechanisms (CCMs) is a new challenge for both the academia and the industry. The overwhelming majority of the existing validation approaches focuses on the internal validity of CCMs (e.g. consistency, bias), while there is a shortage of efficient approaches for assessing the external validity (e.g. applicability, reusability). The objective of this paper is to clarify the concepts and criteria, and to develop an approach for a systematic evaluation of the applicability of a given CCM to cases that were not considered at design time. The approach is adapted from the validation square approach (VSA). The adapted methodology (A-VSA) makes it possible to evaluate CCMs from (a) theoretical structural, (b) empirical structural, (c) theoretical performance, and (d) empirical performance dimensions. Altogether eight indicators are introduced that support the evaluation process. The effectiveness of the A-VSA was confirmed through a case study, in which a specific CCM is considered and the strategy of the A-VSA was operationalized with three completely different application cases. As evidenced by the results, the proposed A-VSA establishes a tight coupling among the enablers embraced by a CCM and the aspects of theoretical and empirical validation, which approves the approach to be an efficient tool for defining the range and/or the extent of applicability. The advantage of the A-VSA is that it offers a way to transfer qualitative applicability evaluation into quantitative applicability assessment, which allows the use of both subjective statements and mathematical modeling in applicability testing. The results of the assessment can guide the adaptation work of a CCM when applied to an out-of-domain application.

Maintaining a complex system, such as a modern production line, is a knowledge-intensive task. Many firms use maintenance reports as a decision support tool. However, reports are often poor quality and tedious to compile. A Conversational User Interface (CUI) could streamline the reporting process by validating the user's input, eliciting more valuable information, and reducing the time needed. In this paper, we use a Technology Probe to explore the potential of a CUI to create instructional maintenance reports. We conducted a between-groups study (N = 24) in which participants had to replace the inner tube of a bicycle tire. One group documented the procedure using a CUI while replacing the inner tube, whereas the other group compiled a paper report afterward. The CUI was enacted by a researcher according to a set of rules. Our results indicate that using a CUI for maintenance reports saves a significant amount of time, is no more cognitively demanding than writing a report, and results in maintenance reports of higher quality.

We present a novel DC motor driven soft robotic fish which is optimized for speed and efficiency based on experimental, numerical and theoretical investigation into oscillating propulsion. Our system achieves speeds up to 0.85 m/s, outperforming the previously reported fastest free swimming soft robotic fish by a significant margin of 27%. A simple and effective wire-driven active body and passive compliant body are used to mimic highly efficient thunniform swimming. The efficient DC motor to drive the system decreases internal losses compared to other soft robotic oscillating propulsion systems which are driven by one or multiple servo motors. The DC motor driven design allows for swimming at higher frequencies. The current design has been tested up to a tailbeat frequency of 5.5 Hz, and can potentially reach much higher frequencies.

Bicycle production has not changed much over the last 100 years, it is still performed mainly by manual labor in mass production. During the global pandemic, the demand for ecologically friendly and customized transport has increased. Hence, customers start to impose the same requirements on bikes as on cars: they want more customized products and short delivery time. This publication describes an approach to transform bicycle manufacturing towards human-robot co-production to enable smaller batch sizes and production on-shoring. We list the challenges of this transformation, our applied methods, and presents preliminary results of the cobot-driven prototypes.

Cyber-physical systems (CPSs) have sophisticated control mechanisms that help achieve optimal system operations and services. These mechanisms, imply considering multiple signal inputs in parallel, to timely respond to varying working conditions. Despite the advantages that control mechanisms convey, they bring new challenges in terms of failure prevention. The compensatory action the control exerts cause a fault masking effect, hampering fault diagnosis. Likewise, the multiple information inputs CPSs have to process can affect the timely system response to faults. This article proposes a failure prognosis method, which combines time series-based forecasting methods with statistically based classification techniques in order to investigate system degradation and failure forming on system levels. This method utilizes a new approach based on the concept of the system operation mode (SOM) that offers a novel perspective for health management that allows monitoring the system behavior, through the frequency and duration of SOMs. Validation of this method was conducted by systematically injecting faults in a cyber-physical greenhouse testbed. The obtained results demonstrate that the degradation and fault forming process can be monitored by analyzing the changes of the frequency and duration of SOMs. These indicators made possible to estimate the time to failure caused by various failures in the conducted experiments.

Nautical traffic management in The Netherlands is shifting from local traffic control to corridor traffic management. Current traffic management systems do not sufficiently support operators in perceptual and cognitive process to interpret and understand the large amounts of information needed for corridor traffic management. Newly developed user interface concepts aim to overcome deficiencies of current interface designs that insufficiently support situation awareness assessment. The effects of these new user interfaces, however, are insufficiently known due to the intricate relations between situation awareness, task performance, and workload. The objective of this study is to evaluate the effects of the three previously developed user interface concepts on operators’ situation awareness, task performance, and workload to gain better insights into the benefits and limitations of the user interface design concepts. The effects were tested in a simulator environment. The results show that user interface features of an integrated user interface allowed operators to apply more effective information processing, which resulted in better task performance. Features of a context-dependent adaptable user interface triggered proactive behavior of operators, which resulted in better task performance for tasks in which operators require insight into future activities of the elements in the environment.

Hazard-intense applications of cyber-physical systems (CPSs), such as the evacuation of a building on fire, require personalized informing on the basis of a real-time assessment of dynamic context. In this paper, a context-dependent message construction mechanism (CD-MCM) is proposed with the objectives (i) to inform people about the emergence and development of a situation unsafe for them, and (ii) to instruct them what they have to do according to an adaptively computed action plan. To achieve this, the concepts of 'situation' and 'impact indicator' have been introduced in order to facilitate the computation of personalized action plans and to send messages about the level of danger and the requested actions. In both activities, the inferred implications of situations are used as the basis of informing the involved people. The messages are adapted to actual situations. In addition, the concept of 'relevance indicator' was utilized to assess the significance of the standing situations for the concerned people in a quasi-real-time manner. The level of danger was evaluated for each person by totaling the values of the situation-related relevance indicators. This was also used to select the proper message templates from the predefined alternatives in the process of message construction. The personalized messages were generated based on the chosen message template and the various message components describing concerned situations or providing instructions. The CD-MCM was validated in a simulated indoor fire evacuation guiding application. In the practical evaluation of the quality of the generated messages, a sample of test people was involved. The results of the evaluation show that the messages generated by the proposed CD-MCM lead to more effective messaging about the personal context and the expected actions than the messages constructed by using static context information only. The reason is that the proposed template-based message construction mechanism facilitates the appropriateness as well as the articulation of the contents of context-sensitive personalized messages.

This paper proposes to use Virtual Reality scenarios to explore the reaction of stakeholders within an innovation process in the context of the introduction of robots working in close collaboration with users. The goal is to design the system upfront in such a way, that it is not perceived as a threat to the worker or his/her job. Within the responsible research and innovation approach, the introduction of new technology needs to be accompanied by a careful investigation of the thoughts and feelings of all stakeholders. Especially workers who are currently not working with robots but their workspace is currently undergoing an Industry 4.0 driven transformation, experience fear, that this new technology will make their jobs redundant. On the other hand, it can be observed, that successful robot interaction processes, on the one hand, increase the overall productivity, but also can enhance human well-being. The feeling of 'teamwork' with the artificial intelligence entity can develop to be equally positive and motivating. To be able to design future workspaces which will result in a 'teamwork' perception instead of the 'fear' perception, the use of VR can be applied.

How to visualize recorded production data in Virtual Reality? How to use state of the art Augmented Reality displays that can show robot data? This paper introduces an opensource ICT framework approach for combining Unity-based Mixed Reality applications with robotic production equipment using ROS Industrial. This publication gives details on the implementation and demonstrates the use as a data analysis tool in the context of scientific exchange within the area of Mixed Reality enabled human-robot co-production.

Smart CPSs (S-CPSs) have been evolving beyond what was identified by the traditional definitions of CPSs. The objective of our research is to investigate the concepts and implementations of reasoning processes for S-CPSs, and more specifically, the frameworks proposed for the fuzzy front end of their reasoning mechanisms. The objectives of the paper are: (i) to analyze the framework concepts and implementations of CPS, (ii) to review the literature concerning system-level reasoning and its enablers from the points of view of the processed knowledge, building awareness, reasoning mechanisms, decision making, and adaptation. Our findings are: (i) awareness and adaptation behaviors are considered as system-level smartness of S-CPSs that are not achieved by traditional design approaches; (ii) model-based and composability approaches insufficiently support the development of reasoning mechanisms for S-CPSs; (iii) frameworks for development of reasoning in S-CPS should support compositional design. Based on the conclusions above, we argue that coping with the challenges of compositionality requires both software-level integration and holistic fusion of knowledge by means of semantic transformations. This entails the need for a multi aspect framework that is able to capture at least conceptual, functional, architectural, informational, interoperation, and behavioral aspects. It needs further investigation if a compositionality enabling framework should appear in the form of a meta-framework (abstract) or in the form of a semantically integrated (concrete) framework.

Malfunction or breakdown of certain mission critical systems (MCSs) may cause losses of life, damage the environments, and/or lead to high costs. Therefore, recognition of emerging failures and preventive maintenance are essential for reliable operation of MCSs. There is a practical approach for identifying and forecasting failures based on the indicators obtained from real life processes. We aim to develop means for performing active failure diagnosis and forecasting based on monitoring statistical changes of generic signal features in the specific operation modes of the system. In this paper, we present a new approach for identifying emerging failures based on their manifestations in system signals. Our approach benefits from the dynamic management of the system operation modes and from simultaneous processing and characterization of multiple heterogeneous signal sources. It improves the reliability of failure diagnosis and forecasting by investigating system performance in various operation modes, includes reasoning about failures and forming of failures using a failure indicator matrix which is composed of statistical deviation of signal characteristics between normal and failed operations, and implements a failure indicator concept that can be used as a plug and play failure diagnosis and failure forecasting feature of cyber-physical systems. We demonstrate that our method can automate failure diagnosis in the MCSs and lend the MCSs to the development of decision support systems for preventive maintenance.