<p>Adaptable, intuitive, and efficient user interfaces are crucial elements for digitalizing industrial processes within the Industry 4.0 framework. User interfaces (UI) are essential components in industrial robot programming and control, especially in modern collaborative scenarios requiring fast robot reprogramming, production changeover, and reliable safety methods. Recent works demonstrate the strength of digital twin (DT) extended reality (XR) interfaces in providing flexible methods for safety evaluation, workspace design and assessment, programming, training, control, and teleoperation. By including the operator in the DT of the physical industrial system, these interfaces provide a powerful means of interaction and constitute a workbench for system efficiency, human-robot collaboration (HRC) assessment, user evaluation and monitoring towards the operator 4.0 paradigm. Nonetheless, XR systems are likely to increase user stress levels, eye fatigue, and sickness. &nbsp;Applications in this field tend to focalize on the overall DT systems performance and evaluation, considering Virtual Reality (VR) UI interaction paradigms barely as a means to an end. A physical interface, usually a tech pendant, mediates traditional robotic arm control and programming. Interactions within the DT VR scenario are generally mediated by the visualization and control hardware and devices. Despite being designed with similar button mapping, VR interaction controllers have different physical and morphological characteristics depending on the producer. Moreover, the interaction with the virtual UI can be delivered in different ways, depending on both hardware capabilities and VR software implementation. These can highly influence user task performance, experience, and workload while interacting with the immersive DT or interface. Our previous work demonstrated the efficiency of DT XR interfaces in robot control by comparing user task performance with a physical teach pendant. Based on previous research results, this work aims at the comparison and evaluation of VR UI interaction methods based on different input system design approaches. This study proposes a virtual interface interaction task implemented in Unity3D software. Interaction with the virtual UI components for robot control and manipulation are provided by three distinct input methods, which present a scaled level of abstraction form hardware-based to hand avatar interaction. In particular, the user interacts with the virtual UI by means of VR device controller button mapping as direct input, traditional VR pointer UI selection an interaction, and physical-based manipulation via user hand avatar. &nbsp;The goal is to assess the most efficient and usable input method for VR-based UIs and provide a starting point for further design, development and evaluation of XR DT interfaces for HRC, programming, and control.</p>

Evaluation of Virtual Reality Interface Interaction Methods for Digital Twin Industrial Robot Programming and Control, A Pilot Study

Paper Proceedings: https://asmedigitalcollection.asme.org/IMECE/IMECE2021/volume/85543

ASME 2021 International Mechanical Engineering Congress and Exposition (IMECE2021)

Supporting the need among researchers worldwide for additional online and multimedia content, ASME now offers the ASME Conference Video Collection. This platform will collect and provide long-term access to conference presentation videos, encompassing all technology areas within ASME's technical conference and events program. About 30,000-40,000 engineers across all mechanical engineering and related disciplines participate in ASME conferences each year. This includes authors of approximately 6,000 to 8,000 technical papers published by ASME, the majority of whom also submit video presentations for our virtual conferences. The video collection will be available as an add-on to existing product packages or as a separate subscription.