Interactive Conversational Agents for Cigarette Smoking and Vaping Cessation: A Mixed Methods Systematic Review

1. Introduction

Smoking is a major public health problem, responsible for the deaths of more than eight million people worldwide each year [1]. Lung cancer caused by direct exposure to cigarette smoke is responsible for these deaths [2,3], but up to 1.2 million of them are caused by passive smoking [1]. Smoking cessation is therefore a first-line intervention to reduce lung cancer and smoking-related mortality [4]. The use of electronic cigarettes (e-cigarettes) and vaping devices for nicotine consumption is growing rapidly, especially among young people, raising new public health concerns [5,6]. The number of e-cigarette users worldwide is estimated to be 82 million in 2021 [7]. And according to the World Health Organization (WHO), e-cigarette use among adolescents exceeds that of adults [8]. Vaping has been linked to mental health problems such as attention deficit disorder, personality changes, learning disabilities and sleep disorders [9,10], and young people are more vulnerable to the neurological effects of nicotine in e-cigarettes [11]. Vaping may also increase the risk of developing conditions such as recurrent spontaneous pneumothorax, chronic obstructive pulmonary disease, and asthma [12,13,14].

The evolution of digital technologies offers new perspectives for health promotion and preventive health interventions [15]. Interventions aimed at preventing smoking and/or vaping are benefiting from this evolution due to new generations’ familiarity with technology [16,17]. Interactive conversational agents, also known as chatbots, are computer programs that use natural language processing to engage in conversations with humans to provide or gather information [18,19,20]. Chatbots use algorithms to simulate a human conversation via text or voice messages. They are easy to use and require minimal computer knowledge and skills [21,22]. Chatbots can use simple, pre-determined conversational algorithms to simulate a conversation, or more complex systems based on neural networks and deep learning to understand speech, produce a vocal response, and simulate social interaction [23]. Interactive chatbots enable automated interventions with customizable, accessible, and inexpensive software [24]. A recent review examined the acceptability and effectiveness of conversational agents to help people quit smoking [25], but did not consider vaping cessation. In addition, this review did not focus on interactive chatbots, which are likely to be more effective than traditional chatbots [24]. The present review focuses on chatbots that can adapt and simulate interactive conversations, as opposed to conversational agents that offer users limited multiple-choice options, thus reducing the impression of real, meaningful interactions.

The aim of this mixed methods systematic review is to assess the effectiveness and characteristics of interactive conversational agents in promoting smoking and vaping cessation.

2. Materials and Methods

This mixed methods systematic review grew out of a broader scoping review of interactive conversational agents used for health promotion, prevention, and care [26]. This review focuses on a sub-analysis of studies that included the use of an interactive conversational agent or chatbot to promote smoking cessation or vaping cessation. The protocol was prospectively registered in the Open Science Framework (OSF) [27]. According to the registered protocol, a scoping review was initially planned before conducting this systematic review. The scoping part aimed to assess the scope of available studies on chatbots for vaping cessation, as this is an emerging field. However, due to the limited number of studies on chatbot interventions for vaping cessation, the scoping review phase was integrated into the final systematic review. Our review questions were: (1) what is the effectiveness of interactive conversational agents for smoking and vaping cessation compared with other alternatives? and (2) what is the benefit of integrating these technologies into smoking and vaping cessation interventions? A mixed methods systematic review was conducted according to the framework of Stern et al. [28]. The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) criteria were used to report the results [29].

We considered all types of evidence that met the PICOS criteria (population, intervention, comparison, outcomes, setting or context):

• Participants or population: We included all studies with lay participants (e.g., patients, students, citizens). Studies targeting only healthcare providers were excluded.
• Interventions or phenomena of interest: We included all interactive conversational agents, i.e., those that involve a two-way exchange (written, oral or visual dialogue), aimed at laypeople for smoking or vaping cessation.
• Comparator: We did not apply any restrictions.
• Outcomes: We considered all outcomes reported in the trials. We looked for outcomes related to cigarette and e-cigarette smokers. These outcomes included barriers, facilitators, acceptability, feasibility, adoption, compliance, motivation to quit, abstinence rates, quit rates, smoking/ vaping-related disease morbidity, quality of life, satisfaction, cost and cost-effectiveness.
• Setting: We included trials conducted in all settings. All study types were included (qualitative, quantitative and mixed methods).

The search strategy of the broader scoping review was reused, and terms related to smoking and vaping cessation were added. The original and modified versions of the search strategy were developed in collaboration with a university librarian (FB) experienced in systematic reviews. We conducted an iterative review process with the research team, and all relevant comments were incorporated into the final version of the search strategy and approved by all team members. We developed a specific search strategy for each of the following databases: MEDLINE (Ovid), Embase, CINAHL, Web of Science, and Inspec (Engineering Village). The search strategy is presented in Appendix A. The search was limited to studies from the last 20 years due to the recent nature of conversational agents, and no linguistic restrictions were applied to the search. We used terms such as chatbot, conversational agent, virtual embodied agent, and their spelling variants. In addition, as innovation implementations are often driven by the private sector, we conducted an Internet search of the following sources and databases: Google, Google Scholar, Institut national d’excellence en santé et en services sociaux, Canadian Evaluation Society, EuroScan, OpenGrey, Grey Literature Report, GreyNet, and Grey Matters.We conducted backward handsearching for other relevant references.

A search of the various databases included in the search strategy identified 148 publications. We exported all citations resulting from the application of the updated search strategy to Covidence, where duplicates were removed by its automated function. Pairs of reviewers (AL, AY, JP, SD, FN, FY) performed an independent blinded assessment of inclusion criteria based on titles and abstracts. The reviewers searched for and obtained all full text of the selected references and imported the files into Covidence [30]. The same reviewers performed an independent blinded assessment of the full texts. Reasons for exclusion were recorded in Covidence and reported in the flowchart. We used the PRISMA diagram to describe the search strategy, selection process and reasons for exclusion [31]. The PRISMA diagram is shown in Figure 1.

We developed an extraction grid and tested it on an article during a team meeting. Team members completed the extraction, and the data quality was validated by an experienced researcher (MPG). We extracted descriptive data (title, year of publication, authors, country), study type (published or gray literature, study design), intervention data (technology name, technology language, delivery channel, and features such as audio, text, voice, avatar), setting data (target population, delivery site), sample data (comparators, number of participants, sample sizes), outcomes (processes, patients, providers, or systems), and type of outcome (qualitative and quantitative).

We presented the data synthesis with a narrative summary of the differences and similarities between interventions, highlighting strengths and weaknesses, key findings, key resources used, and trade-offs. We also compared the direction of effects on primary and secondary outcomes in the included studies. We used a convergent approach to consider both qualitative and quantitative evidence.

We assessed the quality of the included studies using the Mixed Methods Assessment Tool (MMAT) [32,33]. Pairs of reviewers performed a blinded methodological assessment. In case of disagreement, a third experienced reviewer (MPG) was consulted to reach a consensus.

3. Results

3.1. Overview

We identified 105 studies for review after excluding duplicates (see Figure 1). After the abstract review, 25 full-text articles were assessed for inclusion. After full-text review and conflict resolution, eight studies met our inclusion criteria [34,35,36,37,38,39,40,41]. The eight included studies were conducted in European (Netherlands, Spain, Switzerland, UK) and North American (Canada, USA) countries between 2018 and 2023. All used either a chatbot alone or another technology with a chatbot (e.g., an app) to promote smoking cessation. We found one study on the use of chatbots that promoted both smoking and vaping cessation [37]. This study targeted Swiss adolescents aged 16 to 19 years attending a vocational school. The remaining seven included studies [34,35,36,38,39,40,41] targeted all cigarette smokers aged 18 years and older for recruitment. The number of participants recruited ranged from 153 to 688 for the clinical trials [37,38,39,40] to over 50,000 for the community study [41]. One study [34] was a prospective cohort study targeting a specific population; therefore, recruitment was much lower (six participants). Another study [36] was a pre-post test study with recruitment of 349 participants. The last study [35] was qualitative and involved semi-structured interviews with a smaller population (14 participants). All chatbots in the included studies had chat (text) functionality, two of them also had audio [34,40] and/or voice functionality [40,41], only one [34] included an avatar and another [37] included a quiz. Furthermore, in three studies [34,35,36], the chatbot used in the intervention had no comparator. One study [37] compared the chatbot condition with the condition without a chatbot. One study [38] compared a motivational interviewing (MI) chatbot with a neutral chatbot, whereas another study [39] compared a motivational interviewing chatbot with a confrontational advice chatbot. On the other hand, one study [40] compared a chatbot with usual clinical practice for smoking cessation, and the last study [41] compared an app with an integrated chatbot with the same app without the integrated chatbot. The characteristics of the studies are shown in Table 1.

3.2. Outcomes Measured

The reported effects of interventions on patient outcomes are shown in Table 2. One study [36] evaluated the impact of several versions of a motivational interviewing (MI)-focused chatbot using generative thinking on smokers’ progress toward a decision to quit. The study assessed participants’ willingness to quit smoking through three attributes: confidence, importance, and willingness. Only the fourth version (v5.2), which in addition to the improved reflection generator included extended interaction on some of the questions, showed a positive result for the three variables assessing willingness to quit smoking. The same study also showed that the chatbot with deep reflection and extended conversations had a positive effect on perceived empathy, compared to the chatbot without generative reflection. Haug et al. [37] evaluated the reduction of addictive behaviors, specifically smoking and/or vaping. The use of the chatbot did not show a positive effect on reducing smoking and/or vaping. Of these eight studies, only one [39] assessed smoking cessation intentions by comparing the use of a MI-based chatbot with a confrontational counseling-based chatbot. Neither chatbot communication style showed a significant effect on the intention to quit smoking. However, exposure to the chatbot, regardless of communication style, had a positive effect on smokers’ intention to quit. Two studies [40,41] showed a positive effect of the intervention on abstinence. In addition, one of the eight included studies [38] assessed motivation to quit smoking. Differences in scores were not statistically significant between groups but increases in motivation to quit were significant in both control and intervention groups. Three studies [38,40,41] measured engagement, with Olano-Espinosa et al. [40] and Perski et al. [41] showing a positive effect and He et al. [38] showing no difference in engagement in the intervention group compared to the control group. One study [38] measured the effect of an MI-based chatbot on therapeutic alliance and perceived empathy compared to a neutral chatbot but found no significant difference. The same study [38] reported higher perceived communication competence of the chatbot in the intervention group compared to the control group. Quality of life was only measured in one study [40], where both intervention and comparison had a positive effect on quality of life, with the intervention having a smaller effect than the comparison.

Abdullah et al. [34] did not report patient outcomes but presented descriptive data from six participants on the feasibility and acceptability of smoking cessation promotion. This study showed that the conversational agent was acceptable to participants and helped them set a quit date. Five participants reduced the average number of cigarettes smoked, and three tried to quit after the intervention. In addition, five participants adopted stricter smoke-free rules in their homes during the intervention period [34]. The final study [35] did not report quantitative results. This study showed that many users attributed human-like characteristics, thoughts, and behaviors to the chatbot.

We originally planned to conduct meta-analyses in the review protocol, but outcome data were insufficient.

3.3. Methodological Quality Assessment

Of the five methodological quality criteria for RCTs included in the MMAT [31], only the criterion related to blinding of participants was not met in all five RCTs [37,38,39,40,41]. Overall, these studies received a score of 4/5, indicating high methodological quality. Of the three remaining studies, the pre-post test study [36] used a non-randomized quantitative design. Because confounding was not considered in the design and analysis, this study received an MMAT score of 4/5, indicating high methodological quality. The prospective cohort study [34] used a quantitative descriptive design and achieved an MMAT score of 4/5 due to the risk of selection bias in the sampling method and incentives given to participants. Therefore, the sample may not have been representative of the target population. For the semi-structured interview study [35] with a qualitative design, the five methodological quality criteria for qualitative studies included in the MMAT were considered. The study received a score of 5/5, indicating very high methodological quality.

4. Discussion

4.1. Key Findings

This review aimed to describe current interactive conversational agents used in smoking and vaping cessation interventions, and to evaluate their features and effectiveness. Seven of the eight included studies targeted smoking among adult smokers using interactive conversational agents that were comparable in terms of features. The eighth study targeted both smoking and vaping among teenagers [37]. These studies did not address smoking and vaping in the same way in terms of parameters, comparators, and study designs. As a result, the primary and secondary outcomes reported in these studies were heterogeneous. This review considered all outcomes that could be sensitive to an interactive chatbot intervention according to predefined criteria. Reported outcomes included continued abstinence, motivation to quit, engagement, therapeutic alliance, perceived empathy, quality of life, willingness to quit, reduction in smoking and vaping, intention to quit, and anthropomorphism. Two studies [40,41] showed a positive effect of the intervention on sustained abstinence. Perski et al. [36] measured self-reported abstinence at one month, while Olano-Espinosa et al. [40] measured continuous abstinence at six months using biochemically validated quit rates. Continuous abstinence has also been measured in previous systematic reviews of digital interventions for smoking cessation [42,43,44]. For example, Do et al. [42] reported that interactive, tailored, web-based programs showed increased cessation effects for interventions with 6-month follow-up and moderate increases in smoking cessation for interventions with less than 6-month follow-up. The results of our review tend to confirm these findings, but due to the limited number of included studies and the different measurement methods used, no definitive conclusions can be drawn regarding the effect of chatbots on abstinence. Motivation to quit smoking was documented in a study comparing an MI-based chatbot with a neutral chatbot [38]. Participants in both conditions reported increased motivation to quit smoking after interacting with the chatbot. However, a previous systematic review of combined pharmacotherapy and behavioral interventions for smoking cessation [43] reported that motivation to quit was not considered an effect modifier for quit rates, meaning that quit rates were not significantly higher in more motivated populations. Therefore, motivation to quit should be interpreted with caution in future smoking and vaping cessation studies.

Regarding engagement, Olano-Espinosa et al. [40] and Perski et al. [41] reported that the intervention chatbot led to better engagement in treatment compared to usual clinical practice and a regular smoking cessation app, respectively. The prospective cohort study by Abdullah et al. [34] also seems to suggest that participants were more engaged with their treatment when using the chatbot. On the other hand, He et al. [38] reported that participants who interacted with either the MI-based intervention chatbot or the neutral control chatbot perceived them as equally engaging and empathetic, and developed similar levels of the therapeutic alliance with them. In their study, He et al. [38] suggested that these results may be due to the interactions not being frequent or long enough for MI to have an impact. This hypothesis was also supported by a systematic review of the use of MI for smoking cessation by Lindson-Hawley et al. [44]. However, in the study by Brown et al. [36], the perceived empathy of the chatbot that integrated enhanced generative reflections and more extended interaction on specific topics was significantly higher than that of the chatbot that did not generate reflections. This difference appears to be related to the use of generative reflections as opposed to the scripted reflections and responses in He et al.‘s study [38]

In addition, while Brown et al. [36]) demonstrated that a chatbot that integrated generative reflections and extended interaction was associated with statistically significant quit intentions compared to a chatbot without generative reflections, they also emphasized that of the three attributes measuring quit intentions, only the self-efficacy related confidence attribute was the best predictor of successful quitting. This finding is supported by other studies showing that confidence is a significant predictor of smoking behavior, as the more confident an individual is, the more likely he or she is to attempt and succeed in quitting smoking [45,46]. Furthermore, He et al. [48] reported that the perceived communication competence of the conversational agent was positively associated with both the intervention and comparison chatbots. These findings suggest that chatbots, regardless of their conceptual framework, may provide sufficient support to be more effective smoking cessation tools than other common practices. The only study that measured reductions in smoking and/or vaping was conducted by Haug et al. [37]. This study found that adolescent use of a chatbot did not contribute to reductions in smoking or vaping. Regarding vaping, this finding contradicts that of Graham et al. [47,48]. These authors demonstrated in their studies that a tailored and interactive text messaging intervention was effective in promoting vaping cessation in both adolescents and young adults [47,48]. This difference in results may be related to the composite nature of the primary outcome measure in the study by Haug et al. [37], which may not account for confounding factors. As for Leeuwis and He [39], their study showed that interaction with a chatbot, regardless of whether it incorporated a motivational interviewing communication style or a confrontational counseling style, did not positively contribute to smoking cessation intention. However, it is noteworthy that in some digital interventions, particularly online smoking cessation interventions, the intention to quit was significantly higher among smokers who successfully quit and those who relapsed compared to persistent smokers [49]. This finding is consistent with the secondary outcome of the study by Leeuwis and He [39], which showed that exposure to the chatbot, regardless of communication style, had a positive effect on smokers’ intention to quit.

Quality of life (QoL) was only measured in the study by Olano-Espinosa et al. [40]. Although there was no statistically significant difference between the groups in terms of smoking cessation rates, QoL was further improved among participants assigned to the chatbot compared to usual clinical practice. However, according to this study [40], this result does not necessarily mean that the chatbot itself had a better effect on QoL, as it could be related to the higher abstinence rates observed in the intervention group. Alphonse et al. [35] indicated in their study results that the chatbot thought and behaved like a human. However, these results contradict the study by Bendotti et al. [50]. In their study, they reported that participants expressed that they felt like they were talking to a robot, and that they initially felt like they were talking to a human, but that errors in the chatbot’s responses diminished this effect, causing frustration that discouraged them from continuing to use it [50]. Their findings may be related to functionality issues that are not present in the chatbot in Alphonse et al.‘s study [35].

4.2. Main Contributions

The current scope of interactive conversational agents available for smoking cessation was previously reported in a scoping review [51], although an update was needed given the rapid development of chatbot technologies in recent years. A systematic review of chatbots for smoking cessation was also published during our review [25], but this review did not include vaping. Like He et al. in their review [25], we found some support for the use of chatbots for smoking cessation. Our review found that compared to other smoking cessation methods, chatbots can lead to increased willingness to quit smoking and vaping, better engagement in treatment, and eventually higher sustained abstinence rates and improved quality of life. Our review also found that chatbots may be perceived as empathetic and may establish an appropriate therapeutic alliance due to their communication skills, which may explain the higher engagement observed. Other systematic reviews of digital health interventions for smoking cessation have reported that participants were generally satisfied with the use of internet-based interventions [42,44]. However, given the small number of included studies and the heterogeneity of outcome measures, it is important to update this review as new studies become available.

4.3. Strengths and Limitations

Neither the previous scoping review [51] nor the systematic review [25] included studies on electronic cigarette vaping, which, like smoking, is a growing public health threat [5,6]. We considered it relevant to include both smoking and vaping behaviors in our review, given their similarities in terms of potential health effects and intervention design. Therefore, our review included an examination of the effectiveness of chatbots in reducing vaping, which is a strength that makes our study original. However, it also highlights a lack of knowledge about the effectiveness and feasibility of this technology in the context of vaping, and whether the effects differ from those of smoking cessation. One of the strengths of this review is that it only considered interventions that used interactive conversational agents that can adapt to the individual, rather than static conversational agents that use a predefined set of questions and answers.

This systematic review has certain limitations. First, the small number of studies found and the variability in their designs and outcome measures limit the ability to make a quantitative synthesis of the reported effects. Unlike the review by He et al. [25], our review considered interactive chatbots, which is why the number of studies included in our review (n=8) is lower than theirs (n=13). The fact that there is only one study targeting vaping cessation is another limitation of our review. However, given the similarities and nature of interventions targeting smoking and vaping cessation, it is likely that most of the findings could apply to both smoking and vaping. As e-cigarette vaping is now considered an important public health issue, it is likely that more chatbot interventions for vaping cessation will be implemented, and their results should be incorporated into an update of this review.

5. Conclusions

This systematic mixed methods review provides an overview of interactive conversational agents for smoking and vaping cessation. This knowledge could be useful for the development of interactive conversational agents better suited to the context of smoking and vaping cessation. While the number of cigarette smokers remains stable worldwide, the growing popularity of vaping among young people may have serious public health implications. Alternative intervention tools for younger generations, such as chatbots, may provide public health professionals with an additional means of reaching them. It is important to continue research on this topic and provide up-to-date evidence to inform policymakers and researchers about the effectiveness and success predictors of interactive conversational agents in smoking and e-cigarette vaping cessation.