From Digital Twins to Immersive Manufacturing: XR and Gesture-Based Control for Enhancing Human–Robot Collaboration

Digital twins (DTs) and immersive extended reality (XR) interfaces offer new opportunities for intuitive, human-centred interaction in smart manufacturing. However, implementations often lack rigorous validation, quantitative performance analysis and assessment of scalability and robustness, especially in small-scale or resource-constrained manufacturing settings. This work proposes a modular metaverse framework integrating ROS2, Unity and Meta Quest devices to develop interactive, bidirectional DTs enhanced with gesture-based and mixed reality (MR) control. The framework is demonstrated through a lab-scale case study combining a robotic Wire Arc Additive Manufacturing (WAAM) system and a collaborative robot-based laser-cleaning station, showing broad applicability across industrial robotics. To evaluate usability, a preliminary user study with 17 participants was conducted, comparing a standard teach pendant with the proposed XR interface for a tool-inspection task. Results show an 80% reduction in programming time and significant decreases in perceived workload. Measured end-to-end gesture-to-action latency ranged from 430 to 450 ms, representing suitable timeframe for high-level interaction and task initiation. The study provides empirical user-centred evidence for metaverse-enabled interaction and discusses scalability, latency and industrial constraints. Aligned with the human-centric intelligence principles of Industry 5.0, the proposed approach improves accessibility, operator well-being and adaptability, contributing towards more inclusive and robust smart manufacturing systems.