Evaluating Expert-Informed AI-based e-Training and Live Expert Training for Continuing Medical Education (CME): A Pilot Study

Continuing Medical Education (CME), or continuous professional development (CPD),plays a vital role in ensuring that more than 100 million medical professionals globally maintain up-to-date knowledge and clinical competencies[1]. When implemented effectively, CME can positively influence clinicians’ attitudes, enhance their knowledge and skills, and ultimately improves patients’outcomes[2]. Accelerated by the COVID-19 pandemic, healthcare professionals are increasingly choosing CME online platforms due to their flexibility and accessibility[3]; a trend that most likely will continue to expand..

Whilst various e-learning approaches have demonstrated improvements in knowledge acquisition, evidence on their effectiveness especially within CME remains limited[4,5]. As artificial intelligence (AI) gains traction in medical education[6], blended learning models that combine self-paced, AI-guided platforms with traditional synchronous methods may offer a promising route to improving learning outcomes and participant engagement. The Epistudia platform introduces such an approach, delivering a structured, evidence-based CME curriculum that integrates an expert input with an AI-driven personalization. It is designed to facilitate the development of CME content focused on evidence-based topics. Through the integration of AI, Epistudia enables the conversion of expert knowledge into personalized CME e-courses with minimal manual input. This process allows for rapid CME-course creation and facilitates a timely updates of the medical content, addressing the need for an up-to-date content in the context of rapidly evolving medical knowledge.

This study evaluates the effectiveness and acceptability of this blended, expert-informed AI-based training (EIAT) in CME-accredited training. Whilst CME is primarily aimed at licensed healthcare providers, selected advanced medical students may also be eligible to participate for educational purposes.

A total of 39 participants were registered for the training, either through the Epistudia website or by contacting the organizing committee directly. Of these, 17 participants provided informed consent and started the EIAT course on the Epistudia platform, forming the final study population (Figure 2). Among the 17 participants, 10 were clinicians, 2 were medical students, 2 were researchers, and 3 reported other professional backgrounds. Geographically, 12 participants were based in the United Kingdom, 1 in the United States, and 4 in Albania and North Macedonia.

Our pilot study demonstrated that EIAT is well-received by both healthcare professionals and medical students, with high levels of satisfaction, engagement, and perceived applicability of the knowledge to clinical practice. Additionally, participants showed improvement in knowledge acquisition, as measured both subjectively (self-perceived) and objectively (exam performance). While the EIAT yielded outcomes comparable to those of LET, participants reported slightly higher engagement and involvement with the the LET format, although the differences were not statistically significant.

To our knowledge, this is the first study to evaluate the effectiveness of EIAT specifically within the context of CME and professional development. Previous studies on e-learning have investigated a range of modalities including video modules, computer-assisted learning, gamified approaches, and live webinars and have generally found them to be comparable in effectiveness to traditional instructional methods[7,8, 9]. Consistent with our findings, a randomized pre-post study in Italy involving 39 family medicine residents showed that video module instruction was as effective as traditional teaching in improving knowledge and satisfaction[10]. A study conducted in two Welsh medical schools found that one hour of e-learning led to greater knowledge acquisition than one hour of lecture-based teaching[11]. Unlike Welsh study, our study included a broader sample of healthcare professionals and a smaller number of medical students, following a research skill training. The higher receptiveness to e-learning among younger learners may lead to a favorable effect of e-learning modalities. Moreover, in our study, the content of the training differed between the two formats, which may have influenced learning experiences and challenges. Nonetheless, our study builds on existing research by introducing EIAT, a novel approach that, unlike many traditional e-learning methods, requires less effort to develop and allows for more flexible and timely updates to course content. This adaptability is particularly valuable in biomedical education, where knowledge evolves rapidly.

Live expert-led training demonstrated higher levels of engagement and involvement compared to EIAT training, although the difference was not statistically significant. In our study, the AI-based training consisted of video instruction and self-guided exercises, but did not include interactive elements or feedback for participants. The absence of real-time interaction, gamification, or dynamic exercise features that were integrated into the live expert-led sessions may account for the observed differences in engagement. Supporting this, a study conducted among undergraduate students in Groningen found that incorporating game-based learning led to significantly higher engagement and involvement compared to passive video-based instruction[12]. These findings suggest that the addition of interactive components, such as gamification or real-time feedback, could enhance the effectiveness and appeal of AI-based training formats.

While our study has several strengths, including the training was CME-accredited and resembling a real-world scenario, it also has several limitations. Firstly, both participants and assessors were not blinded to the intervention. However, outcome assessments were conducted independently of the assessors, which likely minimized potential bias. Secondly, the level of difficulty in the learning content may have differed between the two training modalities. This could partly explain why participants in the expert-informed AI-based group scored higher on the pre-test compared to those in the live expert-led group, suggesting that the learning difficulty in LET was higher. The extent to which knowledge and learning outcomes were influenced by content difficulty remains unclear. Thirdly, the study may have been underpowered to detect statistically significant differences between the two training formats, and as such, the comparative findings and respective p-values should be interpreted with caution. Fourthly, participation in the LET was defined as being present at both the start and end of the session, which may have overestimated actual engagement, as it did not account for periods of inactivity during the training. Completion in the EIAT may have been overestimated, as it was primarily based on video module tracking and did not verify engagement with optional self-paced exercises or active interaction with reading materials. Additionally, because only participants who had completed EIAT were included in LET, those in LET may have been more motivated, potentially contributing to higher completion rates. Given differences in difficulty in learning activities between the two modalities may have been present, comparisons of completion rates should be interpreted with caution. For example, if learning activities in LET could be considered more difficult, similar completion rate or knowledge gain could also indicate greater efficacy. Fifth, because EIAT always preceded LET, a carryover/practice effect may be present; without counterbalancing or an order term, we cannot disentangle this from modality effects. Future studies should incorporate randomization or counterbalancing to better isolate modality effects. Lastly, the generalizability of our results may be limited due to the specific participant population. Extending these findings to other clinical specialties or to broader applications beyond CME requires further investigation.

In conclusion, our pilot study highlights the potential of EIAT as a tool to enhance CME. Future studies with larger sample sizes and more rigorous designs are needed to determine whether this approach could be effectively and formally integrated into professional training programs.