Study of Diagnostic Image Legend Quality in the Oral and Maxillofacial Radiology Published Literature

1. Introduction

In the dental literature, images are a rich source of diagnostic and clinical information. They enhance and clarify articles and offer succinct summaries of the research paper or clinical report. It is often said, “A picture is worth a thousand words.” In published radiology literature, the “picture” is usually an essential element of the article. An image, be it a radiograph, an MR image, a technetium scan, or an ultrasound image, can demonstrate a clinical finding or research result more objectively. While most images reveal the anatomic detail, some images are obtained to assess the functionality of the area in question. Examples are bone scans (Technetium scans) as well as Positron Emission Tomography scans (PET, PET/CT and PET/MR). Oftentimes, a reader will quickly scan a journal and will focus on an interesting radiograph or figure and decide whether to read the abstract or even the full article. When a reader’s attention is caught by an image without the context of the paper, it is important that they understand what they are viewing and what message is being sent to the reader via the image without the reader having to first read the entire article. Ideally, this should be the role of every legend that accompanies a figure.

In histopathology, there is a standard format of describing a figure. This includes the tissue and cell type, magnification, and the type of stain/stains used, so that the reader clearly understands what the figure represents. The diagnosis becomes the next key part of the pathologist report. Although the same standard should apply to radiographic images, it often does not appear to be common practice, based on anecdotal information. We hypothesize that figure legends associated with radiographs often do not offer enough information for the reader to understand what the image represents. A legend for a radiograph or an image should have enough information for a novice reader to recognize the image. With the advent of multiple radiographic techniques and modalities, basic parameters such as imaging technique, anatomy, and location or level should be included. A proper legend will improve a manuscript and help improve the reader’s ability to diagnose images in dental and maxillofacial radiology.

2. Materials and Methods

The samples for this study were taken from major peer-reviewed and indexed journals devoted to dental and maxillofacial radiology: Dentomaxillofacial Radiology, Imaging Science in Dentistry, Oral Radiology, and Oral Surgery Oral Medicine Oral Pathology Oral Radiology. Ten figures with the accompanying legends were randomly chosen for each year and journal between the years of 2011 to 2020 for a total of 400 figures and legends. Using Google Apps Script API, a new program was developed to randomize the 400 figures and legends. To evaluate each figure and legend, a Ratings Form was created as shown in Figure 1.

A 5-point rating scale, Image Quality Rating Scale (IQRS), was developed by a board-certified oral maxillofacial radiologist (MM) to evaluate the quality of the legends as shown in Figure 2. Each figure and legend were then reviewed independently by two investigators and categorized using the IQRS.

The ratings data were gathered through a series of survey questions wherein a given legend from a dental journal was displayed alongside a linear scale from which a rating would be chosen. Google Forms was chosen as the user interface for these surveys because of the webapp’s pre-existing survey infrastructure and script automation capabilities. Google Spreadsheet was chosen to log data because of its tools for integration with Google Forms. The Google Apps Script API was used to automate the process of initializing and populating the Google Forms, a choice made given the volume of data to be collected and arranged.

First, the 400 images of legends to be rated were uploaded into Google Drive for storage. Each image was named with a uniquely identifying filename according to its source journal, the year it was published, and an identifying number corresponding to the publication year (e.g. “A_11_09.png” is the ninth image selected from 2011 from journal “A”). There are four journals (A-D), ten years (11-20), and ten identifying numbers (01-10).

Next, the Google Apps Script API was used to iterate through the uploaded image files and transcribe their filenames into an obscured arrangement. The obscured arrangements were unique to the filename and can be transcribed in the reverse to obtain the original filename. Because identifying information about the image must be visible to the user on the survey screen, the obscured arrangement was created to be used as such identifying information. The goal was to avoid possible bias based on information based on journal name or publication year. These converted filenames and their corresponding original filenames were stored on a Google Spreadsheet. A separate Google Spreadsheet was created to serve as the response sheet, to be used as a data log for the Google Forms to be created. Investigators were calibrated prior to the study and when a disagreement occurred, the legends were reviewed together, and a consensus was reached using a third investigator.

Of the three ratings submitted for each of the 400 radiographic legends, a final rating was calculated as thus: two reviewers, A and B, were chosen to be the main determiners of the final rating. If the A and B submitted the same rating for a legend, then the final rating was that rating. If A and B differed in their rating, then the rating of the third reviewer, C, was consulted. In this case, if A and C submitted the same rating, the final rating was A’s rating; conversely, if B and C submitted the same rating, the final rating was B’s rating. If all three ratings differed, then the final rating was the rating among A and B to which C’s rating was closest. Finally, if all three ratings differed, but C’s rating was exactly the average of the ratings of A and B, then the final rating was C’s rating. These cover all the possible cases for the distribution of ratings.

3. Results

The final ratings data were assembled and evaluated for average figure rating score. The average rating scores per journal, including the raw count of each rating that the journal received, are shown in Table 1.

The range of ratings is from 1 to 5, with 1 being the lowest and 5 being the highest rating score. Journal A and Journal C received average rating scores close to 3.0, with average scores of 2.96 and 3.04 respectively. Journal B and Journal D scored slightly lower than that, with average scores of 2.81 and 2.67, respectively. Overall, the journals scored ratings between 2.67 to 3.04. A graphical comparison of the final ratings per journal is shown in Figure 3 as a stacked bar graph. The highest average rating for one journal was 3.04, which is two ratings below the highest rating possible. The overall average rating for all four journals was 2.87.

It is to be noted that this study does not seek to highlight any significant statistical differences in the quality of the legends between the selected journals, but rather emphasize that the overall quality of the legends needs improvement. Further, improved legends will aid the reader in improving the comprehension of each article.

4. Discussion

Figures and images are an integral part of the medical and scientific literature. Figures offer a concise summary of data, and images offer a wealth of information for the reader. In biomedical literature, tables and figures are so essential that many data-mining programs have been developed to help biologists efficiently access images to facilitate their research [1,2,3,4,5,6]. FigSearch is a text-mining classification system for figures in full text articles, and PDFigCapX is a system for figure and caption extraction [1,3].

In the medical literature, Siontis and his team studied whether there was empirical evidence for the selection and presentation of imaging figures in published papers. They selected 12 general and specialty medical journals published in 2005 and evaluated them for selection, study population, provision of quantitative measurements, color scales, and contrast use. A total of 417 images from 212 articles were evaluated. They observed that in 26% or 108 of the images studied, the paper did not clearly state whether the image was actually from the study population. In addition, they observed that a comment about the image presented was only made in 11% or 44 of the images and abnormal versus normal images were presented in only 9% or 37 of the images. Based on these findings, Siontis and his team suggested items to be considered in image selection and reporting. These items included providing the selection process of the published image, source of the image, contrast or normal versus abnormal, quantitative data on the presented case, color signals, and contrast specification. They concluded that the quantitative data was sparse and difficult to place in context with the material being presented, and the documentation of images was often insufficient [7].

The need to improve the reporting of figures and tables was also addressed more pointedly with guidelines for clinical research in urology. Vickers observed that graphs and figures in the medical literature were often of poor quality and sometimes detracted from the findings presented in the paper. He and his team introduced six principles and proposed guidelines for figures and tables so that the data is communicated effectively [8]. These principles included only using figures that improve the readers ability to understand the findings, using the best graph to convey the information and not simply choosing from the pre-selected options from the statistical software, and not using graphs to replace key information that should be presented in the text. The proposed guidelines also recommend that figures should provide an immediate visual presentation of the data and the labels and legends should be clear and complete. The importance of figures and their accompanying legends cannot be overlooked.

In the oral maxillofacial radiology literature, there is anecdotal evidence that radiographic images lack proper legends. There are no published standards as there are in histology. The standard format of describing a figure in histopathology includes the organ tissue, and cell types, as well as magnification and staining, so that the reader clearly understands what the figure represents and the diagnosis. With the advent of multiple new imaging modalities available to the dentist, it can be overwhelming for clinicians to understand the different imaging modalities presented in the literature, and it would be helpful to the reader to indicate the technique and other parameters in the figure legends.

While two-dimensional radiography has been the mainstay of dental radiology, there have been many recent advances in oral and maxillofacial radiology. Medical computed tomography (CT) was developed by Sir Godfrey Hounsfeld in 1967, and while conventional multi-detector computed tomography (MDCT) scanners are too large and expensive for maxillofacial and dental use, cone-beam computed tomography was quickly integrated into dental practices [9,10,11,12,13,14,15,16,17]. CBCT, which was introduced in Europe in 1996 and the United States in 2001, offers a relatively low-dose, high-resolution digital technology providing imaging of all three reference planes. Mozzo laid the foundation for the new revolution in dental imaging by describing how a volumetric computed tomography image could be useful in dentistry [18]. CBCT was quickly integrated into dental practice, as dentists began using the technology for hard tissue imaging of the jaws for a variety of diagnostic and treatment-related tasks.

Magnetic resonance imaging (MRI) is another modality used in dentistry [19,20,21,22,23,24]. MRI relies on the properties of hydrogen atoms and is best suited for the evaluation of soft tissues. In contrast to radiographic imaging, MRI is based on a magnetic field formed by an MRI scanner in which the patient is positioned. The images are generated by measuring signals sent back by protons, in particular hydrogen atoms, that have become excited in the magnetic field. For this reason, tissues containing water are best visualized using MRI. The main applications of MRI in dentistry include the investigation of temporomandibular joints (TMJ), soft-tissue lesions of the salivary glands, and tumor staging [25,26,27,28,29,30]. The high-contrast resolution in soft tissues is ideally suited for the detection of internal derangement of the TMJ. MRI is also used for the detection of cysts, tumors, joint effusions, synovitis, and neoplasia of the tongue, cheek, neck, and lymph nodes [19].

Ultrasound (US) is an emerging modality in dentistry that is promising because it is non-invasive, inexpensive, painless, and does not produce harmful, ionizing radiation. US refers to the oscillation sounds in the frequency of 2 to 20 MHz, which are in the upper limits of sound audible to humans. The technique is based on the propagation and reflection of ultrasound waves in tissues. Acoustic waves are coupled and transmitted into the human body and the waves are scattered back or reflected to the transmitter as they encounter the different tissues interfaces. The images are composed of grayscale pixels whose intensity is based on how much sound is reflected or scattered back. In dentistry, ultrasound frequencies in the range of 3 to 12 MHz are used. Recent applications of this technology include the diagnosis of carious lesions, tooth fractures, soft tissue lesions, periodontal bony defects, maxillofacial fractures, and TMJ disorders [31,32,33,34,35]. Because standardized views are not often used for US imaging, legends are particularly necessary to orient the viewer and identify anatomic landmarks.

Figures and their corresponding legends are an essential piece in the medical and dental literature. They are included in manuscripts to help the reader understand the clinical and scientific objectives of the research project or case report but are an often-overlooked component of a scientific paper. A well-written legend can help orient the reader to the specific anatomic features and changes that are described in the body of the paper, leading to a better visualization and understanding of the paper. Authors tend to focus on the body of the manuscript and neglect to properly describe figures. Although it may seem to be a very small aspect of an article, many readers will skim a journal and figures are often what will catch their eye drawing attention to the accompanying article. A well-written legend describing a figure may pique the interest of the reader and encourage them to read the entire article. With the many different imaging modalities available in medicine and dentistry, and the field constantly evolving, legends to images also offer an excellent opportunity for the author to educate medical and dental practitioners on recent diagnostic advances in the technology. In radiology, it is a chance for the radiologist to improve the knowledge base of the general practitioner. Figure 4 is an example of an ideal legend for a radiographic image that indicates the technique and appropriate parameters.

The present study sought to evaluate radiographs and legends in the dental radiology literature using a rating scale developed by a board-certified oral and maxillofacial radiologist. This is the first time to our knowledge that figure legends have been studied in the radiology literature.

From the final ratings data, the chosen journals received an average rating score close to 3. Ideally, a journal would consistently put forth radiographic legends that are descriptive enough to receive a score of 5, but each journal maintains lower scores for radiographic legends than the standard applied by the ratings survey. Since the average rating score between journals does not differ drastically, this could imply that the scientific community has come to a tacit consensus on what constitutes an acceptable radiographic legend. The rating criteria used to evaluate journal legends in this paper, however, are more rigorous. The criteria were chosen to be more demanding because advancements in dental imaging technology have resulted in a greater variety of images that can be produced, which without a proper descriptive legend could confuse readers. It could be argued that advancements in technology mandates an updated consensus on acceptable radiographic legends.

This pilot study used the new rating scale, IQRS, and a new program to evaluate the radiology literature. Future studies can include other dental journals as well as medical radiology journals. It would be interesting to evaluate how the medical radiology literature compares to the dental radiology literature using IQRS. Figures and their corresponding legends are an essential piece in the medical and dental literature. They are included in manuscripts to help the reader understand the clinical and diagnostic objectives of the research project or case report. Without a proper legend, the paper is less effective in conveying clinical information. Authors should not only focus on the written text, but also pay close attention to the legends of diagnostic images.