This master’s thesis is an exploration of force feedback in offshore applications with the intention of teleoperating heavy machinery. Specifically, Allseas provided their field joint coating machine handling for offshore pipelay as the subject and scope of this study. The anti-sway use case is identified through literature research, user and expert interviews, and observation of machine handling. Due to the nature of offshore dynamics coupled with heavy machine movement, sway of these machines is relevant to all specifics of handling and placing the machine on the pipe. Operators rely on their intuition and experience when manually operating these machines. For the proposed teleoperation of machine handling, the operator is now distanced from the machine, removing their sense of control, direct force contact, and now making them trust in automation. This is where a force feedback telemanipulator can increase situational awareness and human machine performance. This is achieved through returning the direct contact forces back to the operator and providing them intuitive control from afar. With more research, rapid prototyping, and machine simulation, Paddy (the teleoperation force feedback test set up) is designed and developed. The proof-of-concept test set up is built using open-source frameworks such as Stanford Hapkit and Vanderbilt Simulink model, but with substantial redesign for offshore anti-sway use case. To validate the design and use case, a user test with 12 Allseas engineers is conducted. The task is to mitigate the swing of the FJC simulation via the handheld manipulator with and without force feedback. The results imply that the main hypotheses are valid: (1) force feedback yields faster stabilization times, (2) the perceived workload from NASA-TLX scores is lower for force feedback (3) the user requirements and ease of use UMUX-Lite scores is favourable with force feedback, and (4) embodiment and hand placement influences expectations and feeling the feedback mechanisms. Future research could explore different feedback types (assistance vs. error prevention), feedback basis (sway angle vs. angular velocity) or further confirm these hypotheses. Force feedback is not typically studied in industrial engineering. Thus, design guidelines are created based on this thesis exploration, emphasizing that force feedback is not an add on, but should be considered from the start of the design process. Lastly, recommendations for force feedback in Allseas and other relevant applications are stated for increased situational awareness, task efficiency, and enhanced HMI.

Background This project belongs to the IEBB project, which advocates for the renovation of ‘zero energy houses’ (ZEH) that synergize insulation and heat pump systems for energy conservation. The ZEH’s energy-saving efficiency depends on user behaviour. From a systems perspective, improper human interactions hinder energy saving, while occupants may find system responses that conflict with their preferences. This study delves into these conflicts and crises, highlighting the variance in post-occupancy adaptation influenced by individual perceptions and familial interplay. The project’s zenith aim is to tailor adaptation, ensuring co-performance between the heat pump system and households. Key Insight In researching residents’ responses to the ‘routine crisis’ introduced by a new system, I discovered that these ‘crises’ could be productive, stimulating household engagement and fostering harmonious interactions with ZEH systems. This insight gave rise to the concept of the ‘enacted interface’ - a distinctive bridge between residents and their automated homes that supports the adaptation process. Through analysis, I identified elements that amplify residents’ engagement with ‘crises’ and observed the influence of family diversity on individual perceptions and responses to them. The results are manifested in two interrelated frameworks: one describing the ubiquity of ‘crises’ and another characterising the classification and impact of different elements. Design This design aims to make households curious about ‘crises’ and guide them towards tailored adaptations to new tech. The ‘Clock’ thermostat provides a consistent interface for temperature adjustments, while the ‘Feeling Message Board’ suggests lifestyle tips based on the user’s emotional input and changing scenarios. Both reinforce the system’s ability to communicate contextual and real-time status, motivating users to actively engage with its functions. Additionally, the design stimulates family discussions about the indoor environment and promotes collaborative responses to ‘crises’. Through co-performance of the system’s dynamic feedback and the household’s proactive exploration, the aim is to facilitate tailored adaptation.

This thesis describes the design and development of a conversational assistant (CA) for standardized issue descriptions in manufacturing. The CA was specifically targeted at the acquisition of data on standardized issue descriptions and at the same time reduce operator friction with the system. CAs can be used to handle and process repetitive data, such as gathering data at scale. For Diversey specifically, the conversational assistant can gather data on issues and in the future on root causes, solutions and best practices. At Diversey, agile manufacturing is applied to the production process. Due to the complexity of this approach, the production process has many stoppages. Context analysis at Diversey showed that operators primarily resort to their own intuition and experience to resolve stoppages. Evaluations with operators showed that the system for data acquisition on stoppages currently in place, is an inconvenience and a time sink. The data that was captured with the system was often ambiguous, incomplete and non-descriptive. However, such data can be of significant value to the company as it can be used for process improvement of the production line. A Wizard-of-Oz-like experiment was conducted with the operators where the researcher role-played a CA for acquiring issue description, which resulted in 71 issue capturing dialogues. Even though these dialogues were filled with implicit and explicit knowledge, without proper understanding of the context, the dialogues were too chaotic and unstructured for a machine learning (ML) algorithm to understand and process. For a complete issue description, that a ML algorithm can process, several key entities were identified: machine location, machine component, machine component state, product component and product component state. Furthermore, several challenges were found related to the speech patterns of the operators. Two of these were tackled in the prototype: Usage of synonyms and pronouns. With the open-source framework Rasa, a prototype was developed for a CA that would capture product related issues through a form structure into a database. The form re-quires the following entities: machine location, product component and product component state. The form is used to guide the user to capture good, structured data, while al-lowing some flexibility in the input. With the synonym and pronoun handling features, some of this flexibility is realized. A conversational flow test was conducted to test and improve the CA prototype. Although failing the accuracy requirements set for intents by 7.4% and for entities by 2%, it provided an indication that with some information on the functionality of the CA, the participants were able to correctly capture the set issue. With some training of the operators, it would be feasible to implement such a CA in a manufacturing environment. Through contextual filtering, the model can filter towards the input of the user, allowing for only context-specific issue to be captured. With a qualitative test it was concluded that this feature decreased the duration of issue capturing. The contextual filtering and issue description control provide feedback to the user whether correct information is captured.

ICU survivors have been shown to face memory loss during and after their ICU stays, which contributes the psychological issues. ICU digital diary has been proven as a way of filling the memory gap, thus avoiding the development of these mental issues. EMC has currently been working on the implementation of the digital diary and a tailored strategy should be considered to support ICU nurses in using the digital diary. As ChatGPT’s potential in the field of the ICU, this project starts with the assignment to explore the possibilities of ChatGPT for ICU digital diary in the context of the ICU. Leveraging a multidisciplinary approach, the project amalgamates insights from nurse practices, interface design, and the introduction of ChatGPT technology. Drawing on meticulous literature review, direct observations, and interviews, the results from these activities highlighted nurses’ need for patient information visualisation and guidance in their diary writing, and the possible application that ChatGPT could provide in the digital diary. Based on the insights from research activities, a web application - Patient Diary Toolkit - was designed and developed through three rounds of design iterations. There are basically four main functionalities in this design: reviewing the patient diary, adding messages, diary tracker and ChatGPT-based assistant supporting nurses’ diary writing. This design was evaluated with 15 participants and 5 experts through comparative testing and expert interviews, with the aim of getting insights from their perspectives about the usability, the user experience and the future potential of the design. The results of the evaluation show that the interface is intuitive and easy to understand, and the introduction of the ChatGPT-based assistant offers a practical solution for supporting nurses’ diary writing. Future recommendations including further research, optimisation of the functionality, future-focused solutions for nurses and user experience improvement have been discussed.