Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human-Machine Teams (HMT)

Version 1 : Received: 11 November 2020 / Approved: 12 November 2020 / Online: 12 November 2020 (14:48:50 CET)

A peer-reviewed article of this Preprint also exists.

Damacharla, P.; Dhakal, P.; Bandreddi, J.P.; Javaid, A.Y.; Gallimore, J.J.; Elkin, C.; Devabhaktuni, V.K. Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT). Appl. Sci. 2020, 10, 8390. Damacharla, P.; Dhakal, P.; Bandreddi, J.P.; Javaid, A.Y.; Gallimore, J.J.; Elkin, C.; Devabhaktuni, V.K. Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT). Appl. Sci. 2020, 10, 8390.

Abstract

This paper presents a multi-year research study conducted at the University of Toledo, aimed at improving human-machine teaming (HMT) methods and technologies. With the incorporation of artificial intelligence (AI) into 21st-century machines, collaboration between humans and machines has become highly complicated for real-time applications. The penetration of intelligent and synthetic assistants (IA/SA) in virtually every field has opened a path for the new area of HMT. When it comes to crucial tasks such as patient treatment and care, defense, and industrial production, the use of non-standardized HMT technologies may pose a risk to human lives as well as up to billions of taxpayer dollars. A thorough literature survey revealed that there are not many established standards or benchmarks for HMT. In this paper, we aim to address the lack of HMT design and development standards. We propose a method to design an HMT based on a generalized architecture. This design includes the development of an intelligent collaborative system and human team. Followed by processes and metrics to test and validate the proposed model, we present a novel human-in-the-loop (HIL) simulation method using two controlled HMT scenarios; first, emergency care provider (ECP) training, and second, patient treatment by an experienced medic. Both scenarios include humans processing visual data and performing actions that represent real-world applications while responding to a Voice-Based Synthetic Assistant (VBSA) as a collaborator that keeps track of actions. As part of these HMT simulation studies, the impact of each parameter related to machines, humans, and HMT (such as errors made by humans and by the VBSA) is presented from the perspective of performance, rules, roles, and operation limitations. In addition, based on the analysis of results from two case studies, we presented guidelines for designing future HMT. Our conclusions indicate that the proposed HIL can be used to perform standardization studies in the pursuit of developing techniques for benchmarking HMT that can be used in critical situations.

Keywords

artificial agents; human machine teaming; metrics; synthetic agents; human factors; ergonomics; empirical studies

Subject

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.