Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Initial Experience in Developing AI Algorithms in Medical Imaging Based on Annotations Derived From an E-Learning Platform

Version 1 : Received: 29 August 2021 / Approved: 31 August 2021 / Online: 31 August 2021 (11:23:12 CEST)

How to cite: Henkel, M.; Breit, H.; Wiesner, P.; Wasserthal, J.; Parmar, V.; Weikert, T.; Hofmann, V.; Eiden, S.; Schmülling, L.; Appelt, K.; Winkel, D.; Paciolla, F.; Lechtenboehmer, C.A.; Vogt, M.; Binsfeld, L.; Sexauer, R.; Wetterauer, C.; Mertz, K.D.; Sauter, A.; Stieltjes, B. Initial Experience in Developing AI Algorithms in Medical Imaging Based on Annotations Derived From an E-Learning Platform. Preprints 2021, 2021080564. https://doi.org/10.20944/preprints202108.0564.v1 Henkel, M.; Breit, H.; Wiesner, P.; Wasserthal, J.; Parmar, V.; Weikert, T.; Hofmann, V.; Eiden, S.; Schmülling, L.; Appelt, K.; Winkel, D.; Paciolla, F.; Lechtenboehmer, C.A.; Vogt, M.; Binsfeld, L.; Sexauer, R.; Wetterauer, C.; Mertz, K.D.; Sauter, A.; Stieltjes, B. Initial Experience in Developing AI Algorithms in Medical Imaging Based on Annotations Derived From an E-Learning Platform. Preprints 2021, 2021080564. https://doi.org/10.20944/preprints202108.0564.v1

Abstract

Development of supervised AI algorithms requires a large amount of labeled images. Image labelling is both time-consuming and expensive. Therefore, we explored the value of e-learning derived annotations for AI algorithm development in medical imaging. Methods We have developed an e-learning platform that involves image-based single click labelling as part of the educational learning process. Ten radiology residents, as part of their residency training, trained the recognition of pneumothorax on 1161 chest X-rays in posterior-anterior projection. Using this data, multiple AI algorithms for detecting pneumothorax were developed. Classification and localization performance of the models was tested on an independent internal testing dataset and on the public NIH ChestX-ray14 dataset. Results The AI models F1 scores on the internal and the NIH dataset were 0.87 and 0.44, respectively. Sensitivity was 0.85 and 0.80 for classification and specificity 0.96 and 0.48 for classification. F1 scores were 0.72 and 0.66, sensitivity 0.72 and 0.72. False positive rate was 0.36 and 0.32 for localisation. Conclusion Our results demonstrated that e-learning derived annotations are a valuable data source for algorithm development. Further work is needed to include additional parameters such as user performance, consensus of diagnosis, and quality control in the development pipeline.

Keywords

E-learning derived annotations; Pneumothorax; Artificial intelligence; Crowdsourcing; Educational data mining

Subject

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.