Remote Assistance for Repair Tasks Using Augmented Reality

Abstract

In the past three decades, using Augmented Reality (AR) in repair tasks has received a growing amount of attention from researchers, because AR provides the users with a more immersive experience than traditional methods, e.g., instructional booklets, and audio, and video content. However, traditional methods are mostly used today, because there are several key challenges to using AR in repair tasks. These challenges include device limita- tions, object pose tracking, human-computer interaction, and authoring. Fortunately, the research community is investigating these challenges actively.

The vision of this thesis is to move the AR technology towards being widely used in this field. Under this vision, I propose an AR platform for repair tasks and address the challenges of device limitations and authoring. The platform contains a new authoring ap- proach that tracks the real components on the expert’s side to monitor her or his operations. The proposed approach gives experts a novel authoring tool to specify 6DoF movements of a component and apply the geometrical and physical constraints in real-time. To ad- dress the challenge of device limitations, I present a hybrid remote rendering framework for applications on mobile devices. In my remote rendering approach, I adopt a client-server model, where the server is responsible for rendering high-fidelity models, encoding the ren- dering results and sending them to the client, while the client renders low-fidelity models and overlays the high-fidelity frames received from the server on its rendering results. With this configuration, we are able to minimize the bandwidth requirements and interaction latency, since only key models are rendered in high-fidelity mode.

I perform a quantitive analysis on the effectiveness of my proposed remote rendering method. Moreover, I conduct a user study on the subjective and objective effects of the remote rendering method on the user experience. The results show that key model fidelity has a significant influence on the objective task difficulty, while interaction latency plays an important role in the subjective task difficulty. The results of the user study show how my method can benefit the users while minimizing resource requirements. By conducting a user study for the AR remote assistance platform, I show that the proposed AR plat- form outperforms traditional instructional videos and sketching. Through questionnaires provided at the end of the experiment, I found that the proposed AR platform receives higher recommendation than sketching, and, compared to traditional instructional videos, it stands out in terms of instruction clarity, preference, recommendation and confidence of task completion. Moreover, as to the overall user experience, the proposed method has an advantage over the video method.