Readability Optimization of Layperson Summaries in Urological Oncology Clinical Trials: Outcomes from the BRIDGE-AI 8 Study

1. Introduction

Healthcare information is often too complex for the general public to read, leading to a major issue of understandability of medical content [1].Recognizing the importance of delivering accessible health literacy, organizations such as the American Medical Association (AMA) and the European Union (EU) have issued guidelines aimed at improving the readability and accessibility of medical content [2,3].

In the wake of the COVID-19 pandemic, the public’s reliance on digital sources for health information surged, leading to digitalization of the population [4]. These sources include the internet and social media, which pose significant risks as they can rapidly spread misinformation [5]. For gathering information ongoing or recruiting clinical trials, physicians and the public alike often turn to platforms such as Clinicaltrials.gov which plays a key role by providing brief layperson summaries intended for delivering lay people information about the trial. A “brief summary” is intended as a short description of the clinical trial, including core descriptive information about a clinical trial— structured to summarize its purpose, design, population, and main questions, written in language intended for the lay public [6]. However, evidence suggests that these lay summaries are frequently falling short of the recommended readability marks for lay audiences [7].

The rise of Generative Artificial Intelligence (GAI) has shown to be a promising tool that can transform the way health information is delivered to the public [8,9,10,11,12]. GAI can be utilized to create these brief summaries intended for lay audience that have an appropriate reading level without compromising necessary information [13]. The aim of this study is to evaluate the performance of a generative AI–powered tool in producing readable and comprehensive layperson-tailored summaries of clinical trials in urological oncology.

2. Materials and Methods

Clinical Trials Lay Summary Selection and Cohort

Urologic-oncology clinical trials were extracted from ClinicalTrials.gov using disease-specific search terms including “prostate cancer”, “bladder cancer”, “kidney cancer”, and “testis cancer”. Searches were performed independently for each disease. To identify currently active studies, the “recruiting” filter was applied to each search, isolating trials actively enrolling participants. The filtered results were then sorted from newest to oldest, and the top 50 most recent trials for each oncologic disease were extracted on 09/19/2025. From each disease-specific dataset, five trials were randomly selected using the RAND function in excel. Eligible trials were required to contain both a “Brief Summary” and “Detailed Description” section, and the respective oncologic disease term (prostate cancer, Bladder Cancer, Renal Cancer or Testis Cancer) had to be included in the trial’s “keywords” section. This process yielded 20 clinical trials (five per disease) for subsequent readability and content evaluation. This study is part of the Bridging Readable and Informative Dissemination with GenerativE Artificial Intelligence (BRIDGE-AI) initiative (https://osf.io/8yz6d/).

GAI-assisted Clinical Trial Lay Summary Generation

To generate the GAI-derived lay summaries, a newly implemented option within the Pub2People function of Pub2Post (www.pub2post.com) was utilized. Pub2Post is a GAI–powered tool that has been evaluated in prior research, demonstrating excellent performance in generating media content [14] and concise summaries [13] from existing medical literature. The system required no manual prompting and utilized dual-source trial data. For each of the 20 selected trials, the GAI model received two separate input files: one “Main View” file replicating the publicly accessible ClinicalTrials.gov trial page, and one “Researcher View” file containing expanded study details from backend trial documentation. Using these paired inputs, the GAI system autonomously produced a new brief summary for each clinical trial. This process was repeated five times per cancer type, resulting in 20 GAI-generated summaries across all disease cohorts.

Readability Scores, Grade Level Indicators and Text Metrics

Readability scores (RSs), grade-level indicators (GILs), and textual metrics (TMs) were calculated for both publicly available and GAI-generated brief summaries using a previously validated online tool (https://www.webfx.com/tools/read-able/). The validated readability measures (RSs and GILs) included the Flesch-Kincaid Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Gunning Fog Score (GFS), SMOG Index (SI), Coleman-Liau Index (CLI), and Automated Readability Index (ARI). Text metrics (TMs) such as total word count, sentence count, complex word count, percentage of complex words, average words per sentence, and average syllables per word were also collected.

Brief Summaries Completeness Assessment

To assess content inclusion, all 40 brief summaries (20 publicly available and 20 GAI-generated) were compiled into a single Excel database. Using the “RAND()” function (Microsoft Excel, Redmond, WA, 2024), summaries were randomly distributed between two independent raters, each receiving 20 summaries representing a randomized mixture of publicly-available and GAI-generated content. To reduce the possibility of any rater bias in the AI-generated brief summaries, the headings were removed and only the text was retained for the rater assessment. Raters were instructed to evaluate each summary against eight pre-specified content criteria corresponding to ClinicalTrials.gov structural sections. These included the Study Overview (Description), Study Overview (Conditions), Eligibility Criteria (Inclusion Criteria), Eligibility Criteria (Exclusion Criteria), Study Plan (Design Details), Study Plan (Arms and Interventions), Study Plan (Primary Outcome Measures), and Study Plan (Secondary Outcome Measures). Each domain was assessed dichotomously, with a score of 1 indicating presence and 0 indicating absence. Raters were also provided with direct links to each trial during the evaluation.

Statistical Analysis

Comparisons for all the readability indices (RSs, GILs, and TMs) between GAI-generated and publicly available brief summaries were treated as continuous variables and were compared with a Kruskal-Wallis Test. Fisher’s Exact Test was used for the categorical content inclusion metrics representing the presence or absence of specific study components. All analyses were performed using SAS, with statistical significance defined as p < 0.05.

3. Results

Readability and Text Metrics Comparison

Readability indices and structural text metrics demonstrated statistically significant improvements in favor of GAI-generated brief summaries across all measures (Table 1 Figure 1). GAI-generated summaries achieved a substantially higher FRES (mean 73.3 ± 3.5; median 74.2 [72.1–75.1]) compared with publicly-available summaries (mean 17.0 ± 13.1; median 17.2 [5.4–23.9]; p < 0.0001), indicating markedly greater readability. Corresponding grade-level mean indices showed significant reductions for GAI-generated summaries: Flesch-Kincaid Grade Level (7.0 ± 0.5 vs. 18.2 ± 3.8), Gunning Fog Score (10.1 ± 0.8 vs. 15.7 ± 1.7), SMOG Index (8.3 ± 0.6 vs. 20.8 ± 3.5), Coleman-Liau Index (5.8 ± 0.7 vs. 14.8 ± 2.4), and Automated Readability Index (7.4 ± 0.7 vs. 18.4 ± 5.0), all p < 0.0001. These results indicate that GAI-generated texts were consistently written at a middle school reading level – as per AMA and EU requirements- whereas publicly available summaries required a much more complex reading ability. GAI-generated summaries were substantially longer, averaging 34.4 ± 5.8 sentences and 551.1 ± 76.2 words, compared with 5.1 ± 4.3 sentences and 120.3 ± 81.9 words in publicly available summaries (both p < 0.0001). Despite increased length, GAI outputs exhibited simpler structure, with shorter average sentence lengths (16.2 ± 1.4 vs. 29.3 ± 9.8 words per sentence, p < 0.0001) and lower syllabic density (1.4 ± 0.0 vs. 1.9 ± 0.1 syllables per word, p < 0.0001). Absolute counts of complex words were slightly higher in GAI-generated summaries (32.8 ± 9.7) compared with public summaries (28.8 ± 21.0; p = 0.0903, not significant). However, the percentage of complex words was significantly lower in GAI outputs (6.0% ± 1.8%) relative to publicly available summaries (23.9% ± 5.5%; p < 0.0001), reflecting simpler lexical composition.

Content Coverage of Publicly Available vs. GAI-Generated Brief Summaries

Comparison of content elements between publicly available brief summaries and GAI-generated brief summaries revealed significant differences across multiple study components (Table 2 – Figure 2). While both formats consistently included conditions (95% vs. 100%, p = 1.00), GAI-generated summaries achieved complete coverage (100%) for all other sections, whereas public summaries showed variable inclusion rates. Within the Study Overview, description fields were present in 75% of public summaries compared with 100% of GAI-generated summaries (p = 0.05). Under Eligibility Criteria, inclusion criteria were reported in 80% of public summaries and 100% of GAI summaries (p = 0.11), whereas exclusion criteria were completely absent in the public summaries (0%) but present in all GAI-generated ones (100%), representing a highly significant difference (p < 0.0001). Within the Study Plan category, all design elements—design details, arms and interventions, and secondary outcome measures—were reported in 100% of GAI-generated summaries but only in 60% of public summaries (all p = 0.0033). Similarly, primary outcome measures were noted in 80% of public summaries and 100% of GAI summaries (p = 0.11).

4. Discussion

The present study evaluated the readability and content inclusion of publicly available brief summaries from clinical trials in urologic oncology compared with those generated by and GAI powered tool- Pub2Post (www.pub2post.com ). Using validated readability indices, GAI-generated summaries demonstrated significantly superior readability (all p-values < 0.001) while achieving complete inclusion of the elements outlined in the ClinicalTrials.gov guidelines. These findings highlight the potential of GAI to generate brief summaries that are both more accessible and more comprehensive, thereby improving public understanding and trust in clinical trial information. The internet has become an immense reservoir of medical information. In the wake of widespread digitalization—accelerated by the COVID-19 pandemic—it has become imperative to ensure that the health information consumed by the public is both verifiable and readable [4]. Health-related searches account for a substantial portion of online activity. However, prior studies have shown that such content is often either verifiable but not readable, or readable but not verifiable [15]. Within this context, misinformation—especially via “Dr. Google” and social media—remains pervasive and can serve as a patient’s first encounter with their disease, including urologic malignancies. This underscores the importance of directing patients and the public to validated, trustworthy sources such as ClinicalTrials.gov [16,17]. When individuals visit ClinicalTrials.gov, the brief summary is the first section encountered. According to the ClinicalTrials.gov “Plain Language Guide,” this section should be written in simple, accessible language that allows the general public to easily understand a study’s goals, research questions, and design [6]. Our analysis revealed that the original brief summaries showed a collegiate reading level (18–24 years), far exceeding the recommended 6th–8th grade level. Such complexity poses a substantial barrier for patients seeking to comprehend clinical trial information, potentially resulting in confusion, anxiety, or reluctance to participate in research. Furthermore, lower readability may inadvertently increase physician workload, as patients turn to clinicians for clarification. Even for healthcare providers, overly complex text can hinder efficient communication, limiting their ability to translate trial information into patient-centered discussions. Beyond readability, our findings indicate that publicly available summaries frequently fail to meet content inclusion guidelines [7]. Considerable heterogeneity was observed in length—ranging from a single sentence to seventeen—yet many omitted key elements such as eligibility criteria, study design details, interventions, and outcome measures. Notably, none included exclusion criteria. These omissions compromise the completeness and transparency of trial information, reducing the utility of ClinicalTrials.gov as a comprehensive patient-facing resource.

By contrast, GAI-generated summaries addressed both readability and completeness shortcomings. The model consistently produced outputs at the recommended middle-school reading level (FRES 73.3; FKGL 7.0; GFS 10.1; SMOG 8.3; CLI 5.8; ARI 7.4), while fully incorporating all guideline-specified content elements. Although GAI summaries were longer (mean 34.4 sentences; 551 words), they exhibited simpler sentence structure, reduced lexical complexity, and lower syllabic density—features conducive to improved comprehension. Collectively, these characteristics underscore GAI’s ability to synthesize clinical information into clear, accurate, and lay-friendly summaries [13].

The broader implication of these findings extends beyond urologic oncology. Integrating GAI-based tools such as Pub2Post into clinical trial registries could enhance health literacy, promote patient engagement, and foster trust in evidence-based medicine. By providing complete, readable, and verified trial information in a single platform, GAI may also help mitigate the spread of misinformation and support equitable access to research participation opportunities.

This study has several limitations. First, the sample size was modest (n = 40), comprising 20 publicly available and 20 GAI-generated summaries, which may limit generalizability. Future research should include larger and more diverse disease cohorts to validate these findings. Second, while readability was measured using a validated tool (WebFX), the accuracy, clarity, and contextual appropriateness of each section’s content were not assessed. Establishing standardized tools or expert review frameworks to assess these dimensions will be crucial in future evaluations. Finally, the study did not examine user-centered outcomes—such as patient comprehension, satisfaction, or trust—which require a further dedicated investigation.

5. Conclusions

GAI-powered approach can significantly improve both the readability and completeness of clinical trial brief summaries. Implementing such tools within ClinicalTrials.gov and similar registries could represent a critical step toward improving public access to comprehensible and accurate health information, thereby improving the accessibility of the lay public to health literacy and supporting informed participation in clinical research.