Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Evaluation of Replies to Voice Queries in Gynecologic Oncology by Virtual Assistants Siri, Alexa, Google, and Cortana

Jamie M Land
,
Edward J. Pavlik
* ,
Elizabeth Ueland
† ORCID logo ,
Sara Ueland
† ,
Nicholas Per
,
Kristen Quick
,
Justin W. Gorski
,
McKayla J. Riggs
,
Megan L. Hutchcraft
ORCID logo ,
Josie D. Llanora
,
Do Hyun Yun
†
Equal Contribution: These authors contributed equally to this work.
Version 1 : Received: 5 May 2023 / Approved: 6 May 2023 / Online: 6 May 2023 (08:42:27 CEST)

A peer-reviewed article of this Preprint also exists.

Land, J.M.; Pavlik, E.J.; Ueland, E.; Ueland, S.; Per, N.; Quick, K.; Gorski, J.W.; Riggs, M.J.; Hutchcraft, M.L.; Llanora, J.D.; Yun, D.H. Evaluation of Replies to Voice Queries in Gynecologic Oncology by Virtual Assistants Siri, Alexa, Google, and Cortana. BioMedInformatics 2023, 3, 553-562. Land, J.M.; Pavlik, E.J.; Ueland, E.; Ueland, S.; Per, N.; Quick, K.; Gorski, J.W.; Riggs, M.J.; Hutchcraft, M.L.; Llanora, J.D.; Yun, D.H. Evaluation of Replies to Voice Queries in Gynecologic Oncology by Virtual Assistants Siri, Alexa, Google, and Cortana. BioMedInformatics 2023, 3, 553-562.

Abstract

1) Background: This study focused on the validity of audible replies to voice queries in gyneco-logic oncology by Siri, Alexa, Google, and Cortana virtual assistants (VAs).; (2) Methods: 21 evaluators analyzed VA audible answers to voice queries related to gynecologic oncology. A 3-tier template was utilized for each voice query: “X?” (A), “What is X?” (B), and “Define X?” (C) in a 24-question panel, allowing questions to be posed in different formats to assess the VAs at a greater depth. Responses were scored using a rubric designed to assess the validity and quality of each answer; (3) Results: For general queries, Google provided the most correct audible replies (ncorrect = 20; 83.3% correct), followed by Alexa (ncorrect = 16; 66.7% correct), Siri (ncorrect = 11; 45.8% correct), and Cortana (ncorrect = 5; 20.8% correct.; For Gynecologic Oncology related queries, Google also provided the most correct audible replies (ncorrect=222; 18.14%), followed by Alexa (ncorrect=75; 6.51%), Siri (ncorrect=55; 5.46%), and Cortana (ncorrect=23; 2.28%). (4) Conclusions: The audible replies to oral queries on gynecologic oncology topics varied between Siri, Alexa, Google, and Cortana. Google’s audible reply accuracy was superior to audible replies by Siri, Alexa, and Cortana. Since there is room for considerable improvement in VA performance, the authors advise caution when using VAs for medical queries in gynecologic oncology.

Keywords

Virtual assistants; Gynecologic; Oncology; Siri; Alexa; Google; Cortana; Validity

Subject

Public Health and Healthcare, Other

Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download PDF

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0
Metrics 0
Get PDF Cite Share 0
Bookmark BibSonomy Mendeley Reddit Delicious
×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.