Autonomous Vehicle Teleoperation interfaces
Autonomous vehicle (AV) technology has the potential to disrupt the transportation system as we know it today by increasing traffic safety, reducing congestion, and changing travel behaviors. Latest technological advancements, in particular in the field of artificial intelligence, enable the evolution of AV technology to the point that operational AVs are already being tested in the streets today. However, it is widely believed, both in academia and industry, that AVs will not be able to resolve all ambiguous traffic situations on their own. Thus, a promising approach to resolve these ambiguities and provide an actionable solution for edge situations is Teleoperation. Teleoperation involves a remote human operator (RO) who can monitor and control the vehicle from afar. When a vehicle encounters a problem in a particular situation, a RO can be called to assess the situation and guide the vehicle until the problem is resolved.
Teleoperation systems for AVs are already in use and are being developed by various automotive companies. However, remote driving is not a trivial task and there are various challenges that a RO must face. For example, since the RO is physically disconnected from the operated AV, she cannot feel the forces that are applied on the teleoperated vehicle or hear its surroundings sounds. Other challenges might be related to the fact that a lot of information should be transmitted from the AV to the RO over the network, and thus, latency might be an issue. Finally, the RO has to gain situation awareness quickly under a heavy cognitive load and impaired visibility conditions.
The purpose of this project was to unveil the major vehicle teleoperation challenges and create design recommendations for future teleoperation interfaces of AVs. In order to unveil the major challenges and issues in AV teleoperation, we conducted semi-structured in-depth interviews with 14 experts from industry and academia, who have extensive experience with AVs, teleoperation, command and control, or the auto-industry in general. To complement the interviews, we also observed eight teleoperation driving sessions of AVs. Based on these interviews and observations, we created a framework of overall tele-driving challenges grouped into six clusters. In addition, we provide initial guidelines for the design of AV teleoperation interfaces. The results of this work may be used to help future engineers, designers, and researchers to design and build remote driving interfaces, which take into account the discovered limitations and challenges.