Real-time Digital Twin of On-site Robotic Construction Processes in Mixed Reality

Abstract

The use of robotics in construction improves safety and productivity in construction sites. However, there are limitations for construction managers to monitor robotic construction processes. This is due to the lack of well-developed human-robot collaboration interfaces. Digital Twin (DT) and Mixed Reality (MR) are two emerging technologies that can help to address these limitations by enhancing human-robot interaction, for on-site construction processes. However, DT in MR for human-robot interactions in robotic construction processes has not yet been widely studied in research or in practice. This work explores effective human-robot collaboration for automated construction processes using on-site real-time DT of robotic process in a MR construction environment. First, a DT prototype of a robotic construction process establishes a two-way communication between the physical and virtual models of the robot arm. Next, real-time process data is collected from the robot and sent to the visualisation and database platforms. This work describes the workflow in order to send the real-time data to the MR headset for direct visual feedback and direct interaction with the robot arm. The prototype is validated using a case study demonstration of a robotic masonry construction process. By demonstrating this proof of concept for real-time DT of robotic construction processes in MR, this work contributes to construction management for digital fabrication through human-robot collaboration. This work concludes with potential future research directions including data access and manipulation for digital processes in construction sites.