preprints.org > medicine and pharmacology > gastroenterology and hepatology > doi: 10.20944/preprints202308.0447.v1

Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Version 1 : Received: 3 August 2023 / Approved: 4 August 2023 / Online: 4 August 2023 (13:32:21 CEST)

Although wireless capsule endoscopy (WCE) detects small bowel diseases effectively, it has some limitations. For example, the reading process can be time-consuming due to the numerous images generated per case, and lesion detection accuracy may rely on the operators' skills and experiences. Hence, many researchers have recently developed deep learning-based methods to address these limitations. However, they tend to select only a portion of the images from a given WCE video and analyze each image individually. In this study, we note that more information can be extracted from the unused frames and temporal relations of sequential frames. Specifically, to increase the accuracy of lesion detection without depending on experts' frame selection skills, we suggest using whole video frames as the input to the deep-learning system. Thus, we propose a new Transformer-based neural encoder that takes the entire video as the input, exploiting the power of the Transformer to extract long-term global correlation within and between the input frames. Subsequently, we can capture the temporal context of the input frames and the attentional features within a frame. Tests on benchmark datasets of four WCE videos showed 95.1% sensitivity and 83.4% sensitivity. These results may significantly advance automated lesion detection techniques for WCE images. Our code is available at https://github.com/syupoh/VWCE-Net.git.

artificial intelligence; Transformer; capsule endoscopy; video-analysis

Medicine and Pharmacology, Gastroenterology and Hepatology

* All users must log in before leaving a comment