User Perceptions of Virtual Consultations and Artificial Intelligence Assistance: A Mixed Methods Study

1. Introduction

Virtual consultations, which typically involve remote telephone or video communication between patients and healthcare professionals (HCPs) in real-time, have emerged as a crucial approach for continuity of care provision in recent years, particularly as a result of the COVID-19 pandemic [1]. This shift essentially served as a large-scale case study for the feasibility of remote care provision while highlighting its potential benefits of reducing patient burden, optimising resources and improving healthcare accessibility [2,3]. As such, remote care delivery modalities indicate a promising avenue for the future of smart healthcare systems.

However, in certain regions, the rise in virtual consultation adoption was not sustained following the acute phase of the pandemic and significant consultations are reverting to being in-person, despite the advantages of virtual visits. Market research firm Trilliant Health reports that, in the US, telehealth use in Q3 2023 was 54.7% below the Q2 2020 peak [4]. In the UK, remote appointments are tending to return to their pre-pandemic baselines [5]. Similar observations have been made in Northwest Europe [6]. Reasons attributed to such reversals include insufficient technical resources, lack of financial incentives and cultural barriers [6]. Therefore, there is a need to further investigate if such trends apply to other regions, to understand the reasons for the reversals, and to identify means to encourage adoption in view of the highlighted benefits of virtual consultation.

One technology that has been identified as being potentially beneficial for virtual visits is artificial intelligence (AI). Researchers acknowledge the opportunities for AI integration in telemedicine to enhance virtual interactions and assist users [7,8]. The technology has already been successfully employed across medical specialties [9]; and it would be interesting to investigate its potential support during virtual consultations from the users’ point of view in order to gather insights around the corresponding human factors.

Despite the potential benefits, barriers persist in the integration of AI into virtual consultations. Concerns over data bias, AI output reliability and trustworthiness of AI systems have been emphasised [10]. Cabitza et al. further highlight the risk of overreliance on automated systems, low performance of AI models in regards to ambiguous and uncertain medical presentations, and the lack of explainable outputs of “black box” AI models that hinder post hoc analysis [11]. Furthermore, AI’s effectiveness in healthcare is contingent upon its acceptance by both HCPs and patients [11,12], highlighting the importance of human-computer interaction (HCI) elements when aiming to integrate AI in virtual consultations. Thus, understanding the attitudes and perceptions of its potential adopters is crucial in a virtual care setting. This can be drawn from the Unified Theory of Acceptance and Use of Technology (UTAUT) [13], which factors the constructs of performance expectancy, effort expectancy, social influence and facilitating conditions as critical in user adoption of technology. Based on this theory, performance expectancy, effort expectancy and social influence directly determine usage intention and behaviour, and facilitating conditions is a determinant of user behaviour.

In Ireland, there remains continued interest in virtual consultations. The country’s Health Service Executive has published its ‘Telehealth Roadmap 2024–2027’ that outlines its telehealth development and adoption plan [14]. Virtual care delivery can be especially appropriate in rural areas by increasing accessibility [15]. In Ireland, the Northwest region includes the country’s most rural communities, according to the most recent census [16]. For instance, within this region, an ongoing pilot project for Chronic Obstructive Pulmonary Disease (COPD) virtual wards has shown positive outcomes based on an initial feasibility proof-of-concept [17].

Virtual consultations for patients with noncommunicable chronic conditions have been shown to be generally associated with beneficial health outcomes and positive reception by users [18]. However, the lived experiences of users of virtual consultation within this demographic and their perception of AI integration in virtual care have not been extensively studied empirically, particularly in the current post-pandemic context. Vodrahalli et al.’s clinical pilot study assessed the image quality of skin lesion photographs taken by patients for telemedicine use based on the feedback of an AI support tool, but did not explore the experiences and preferences of users [19]. The feasibility study undertaken by Tao et al. investigated the assistance of an AI software in nodule detection and risk stratification of CT scans for a telemedicine-enabled lung cancer screening programme, but the researchers did not assess the perception of users of such assistance [20]. Other researchers conducted important reviews focusing on artificial intelligence and telemedicine [21,22], but empirical findings appear to be scarce in the literature. Furthermore, there is an opportunity to better understand the reasons for former virtual consultation users during the pandemic to switch back to in-person visits [4,5,6]. The authors of this current study also did not identify empirical studies or protocols on Open Science Framework and Google Scholar aiming to investigate the user perception in regards to AI assistance and virtual consultations for patients with noncommunicable chronic conditions.

This represents a promising research avenue that further motivated this study. Through a mixed-methods approach, it aims to provide a comprehensive understanding of virtual consultation experiences for chronic care of both current users and those who have reverted to in-person visits as well as their perspectives on potential AI assistance; and draw practical insights from these findings. The research questions are as follows: i) What are the differences in experience between current and former virtual consultation users? ii) What are the user needs and suggested improvements to enhance virtual consultations? iii) How do users view a potential artificial intelligence tool to supplement virtual consultations?.

To the best of our knowledge, this is the first empirical, mixed-methods investigation that focuses on virtual consultations for patients with noncommunicable chronic conditions and AI assistance. It provides a deeper understanding of remote care nuances and practical insights on how AI technologies can potentially address the relevant limitations. This study has found that virtual consultations for chronic conditions are widely usable and acceptable, particularly through hybrid approaches. It has contributed to the evidence-base of HCI factors that contribute to current user experiences of remote care modalities, and has further identified key AI features that could potentially enhance virtual consultations. Such an investigation follows recommendations from fellow researchers to explore and better understand the challenges and reservations of users and nonusers of virtual consultations [6].

2. Materials and Methods

This study adopts a mixed methods design, and a protocol was registered on Open Science Framework [23]. The qualitative segment with current virtual consultation users involves online semi-structured interviews (SSIs) with patients with noncommunicable chronic conditions and online focus groups with HCPs treating such patients [24], both conducted using the GDPR-compliant Microsoft Teams. The quantitative segment involves online surveys, developed with the GDPR-compliant Microsoft Forms, sent to both current virtual consultation users and those who have returned to in-person visits.

Public and Patient Involvement (PPI) input was integrated in order to ensure optimal data collection. This follows current recommendations, as such involvement in healthcare research can enhance a study’s design [25]. For this study, four PPI members representative of the target population for data collection were involved. They assisted in improving the participant information sheet and participated in piloting SSIs, focus groups and surveys. Subsequently, they provided feedback for improving the clarity of the content before data collection was undertaken.

Data collection was undertaken between May 2024 and March 2025 by the first author. Participation was voluntary, and no financial incentives were provided to participants. 10 patients and 15 HCPs were recruited to participate in the qualitative segment; with data saturation having been reached after the participation of 7 patients and 10 HCPs as responses were consistent and no new themes emerged. A total of 83 valid survey responses were collected. All data were anonymised prior to analysis.

This study was designed to explore and report on the experiences and perceptions of both patients and HCPs. This is an important consideration when investigating novel aspects of healthcare in order to better understand how both segments experience the modality with the same setup [26]. However, studies on novel approaches to care delivery often focus on the perspective of either patients or HCP, rather than both, which can hinder drawing sufficient insights to improve user satisfaction for adequate implementation [26]. Furthermore, the integration of current and former virtual consultation users offers a holistic view of different perspectives, which can facilitate the identification of limitations in view of improvements.

A summary of the methods is depicted in Figure 1 and described further in the following sub-sections.

2.1. Ethics Statement

Ethical approval and the fully informed, written consent of participants were obtained prior to data collection. Ethical approval for the study was granted by the Atlantic Technological University Research Ethics Committee (IREC-ATUD-23-038), Letterkenny University Hospital Research Ethics Committee (24/ 221124) and Sligo University Hospital Research Ethics Committee (Research Ethics Application No. 994).

2.2. Qualitative Data Collection and Data Analysis

Considering the focus of the overall project that this study is part of [27], the qualitative component was deemed adequate to uncover nuanced factors related to virtual consultation engagement of current users, as such an approach can impart important findings on the virtual consultation experiences of participating patients and HCPs [28,29]. A purposeful, snowball approach was undertaken for participant recruitment in the Northwest region of Ireland, in order to gather rich information with limited resources [30]. Chronic care teams employing virtual consultations from Letterkenny University Hospital and Sligo University Hospital were initially contacted to assess their eligibility and interest in participating in this study. Eligible HCPs subsequently assisted in identifying eligible patients. PD contacted prospective participants via email and phone calls, and did not receive responses from five people. The eligibility criteria are listed in Table 1.

A sample size of 30 was aimed for the qualitative stage; more specifically, 15 patients with chronic conditions for SSI participation and 15 HCPs for three online focus groups. Such a selection was considered adequate as for investigations involving the identification of challenges, 3-20 participants are generally considered acceptable [31]. Furthermore, recent studies undertaken in the fields of remote care and AI in healthcare had similar participant sizes [32,33,34]. This number is also believed to be a realistic estimate for recruitment as well as for managing during the study period and to provide data saturation [35].

The choice of qualitative technique for each participant group (patients and HCPs) was carefully considered. Gathering insights through SSIs can generate a broader range of themes based on individual perceptions, which is suited for research exploring emerging concepts [36]. This is adequate for use among patients, considering their varied, personal experiences that can help better understand opportunities for improvements. Focus groups, where discussions thrive on group dynamics [37], provide rich data beyond individual perspectives. Thus, this approach is adequate for use among HCPs as a collective voice to collect insights from the care delivery aspect.

10 patients were recruited for one-on-one online SSI with the first author and 3 online focus groups, with a total of 15 HCPs, with the first author were conducted. No repeat interviews or focus group was carried out. The participants belonged to two secondary care teams, namely for respiratory and rheumatology, in remote areas in the Northwest region of Ireland (Counties Donegal, Sligo and Leitrim). The virtual consultation modalities employed were a mix of video and phone consultations.

Findings from the authors’ scoping review [18] have informed the SSIs schedule with patients with non-communicable chronic conditions currently using virtual consultations (Supplementary Materials, Appendix 1). The SSIs aimed to gain insights into the associated behaviours and engagement factors during remote consultations for patients within this demographic. Focus groups with HCPs currently providing remote care for patients with chronic conditions aim to extract similar insights. The focus group topic guides were also informed by the scoping review findings (Appendix 2). SSI and focus group discussions began with rapport building between the participants and PD. An overview of the research project was provided, and unfamiliar terms were clarified. Discussions then focused on three general areas, namely experiences and cyberpsychological elements around virtual consultation, technological improvements to enhance virtual consultations, and acceptance/views of an AI tool to supplement virtual consultations.

Inductive, reflexive thematic analysis (RTA), based on Braun and Clarke’s guidelines [38,39], was selected for analysing the data gathered about the participants’ lived experiences and perspectives from SSI and focus group transcripts. The transcripts were anonymised and edited for typographical errors by the first author. PD familiarised himself extensively with the transcripts by reading through them multiple times. He manually coded the data to find patterns and identify themes from the SSIs and focus groups; subsequently, overlapping themes were merged. Context was provided by referring to field notes. The qualitative data was methodically and independently analysed, sense-checked and reviewed by KM and VO to independently identify and agree on the themes. PPI member HC further contributed to reviewing the qualitative themes. Reporting of the qualitative findings followed the recommended Standards for Reporting Qualitative Research guidelines (Appendix 3) [40].

RTA was selected as the data analysis methodology for its flexibility in providing detailed accounts of data, for its ability to efficiently summarise key insights from large datasets, as well as for its accessibility for early-stage researchers [41]. Moreover, recent studies investigating aspects of virtual consultations, such as pitfalls of the technology encountered by HCPs, patient participation and HCP perspectives have employed similar methodologies to provide valuable insights [42,43,44]. Therefore, an RTA was deemed appropriate for this study.

2.3. Survey Data Collection and Analysis

The online survey (Appendix 4) was designed by the research team and piloted with the PPI team. It was then distributed to the same Northwest region of Ireland, with assistance from HCPs, aimed at gathering input from both current and former virtual consultation users for nonmalignant chronic conditions. This survey strengthens the quality of the study as the combination of qualitative and quantitative methods provides a more holistic understanding of the collected data [45]. In addition, it can address some of the limitations of having only a qualitative element, such as having a limited target population with the potential of selection bias [46]. The survey was also used to triangulate our findings from the qualitative segment as it reflected discussion areas from the SSIs and focus groups.

Section 1 of the survey gathered participant demographics data and included some filtering questions. Section 2 of the survey is based on the validated Telehealth Usability Questionnaire (TUQ) [47], and comprises 21 items grouped in 6 subscales: usefulness, ease of use, interface quality, interaction quality, reliability, satisfaction and ease of use. The TUQ was favoured based on this study’s aim as well as for its intended design for both patients and HCPs utilising various telehealth modalities. Other telehealth-related questionnaires are not as comprehensive, are not always aimed at both patients and HCPs, or are not as commonly employed [48]. The TUQ was slightly modified by changing “telehealth” to “virtual consultation” to better relate to this current study, and some statements were slightly edited to be inclusive of both patients and HCPs, following feedback from the PPI pilot. Section 3 of the survey, designed by the research team, focused on the AI assistance views to mirror the elements investigated in the qualitative segment of this study. Respondents who reported returning to in-person consultations in survey question 17 (Section 3) were presented with branching question 18, a 10-item measure to understand their reasons for doing so. This was designed by the research team to better understand the perspectives of this participant segment, in alignment with the study’s aims.

Statements relating to virtual consultation usability, openness to AI assistance and reasons for returning to in-person visits were measured using a 5-point Likert scale (1: strongly disagree; 2: disagree; 3: neutral; 4: agree; 5: strongly agree), where higher scores indicate higher virtual consultation usability and openness to AI assistance. In the TUQ section, an ‘N/A’ option was also made available as some statements, such as on interface quality, might not apply to every virtual consultation experience. The scores were computed by summing Likert items, and a reliability analysis was conducted with the findings from the validated TUQ items.

While 110 participants attempted the survey, 83 valid responses were collected, and 27 did not meet study inclusion criteria based on filtering questions. The valid data were analysed with IBM SPSS (version 29.0.2.0 (20)). The quantitative data were analysed using descriptive statistics, and the Kruskal-Wallis test was used to compare responses between current and former virtual consultation groups. The latter non-parametric test was chosen due to the non-normality of the distribution (all Kolmogorov-Smirnov tests resulted in sig. values < 0.05). Epsilon squared (ε2) was calculated to determine effect size as it is less biased [49]. The survey also included some open-ended questions, and these responses were analysed using an RTA approach to triangulate the SSI and focus group findings.

2.4. Research Group Characteristics and Reflexivity

The research team comprises an interdisciplinary group, with backgrounds in medicine and healthcare (PD, BM, KM), marketing (VO) and engineering (MM). PD, MD, MSc (male) is a PhD researcher who received qualitative research training. BM, MD, PhD (male; Director of the Medical Futurist Institute), KM, PhD (Senior Research Fellow) and VO, EdD (Senior Lecturer) have prior experience in conducting qualitative research. MM, PhD (Senior Lecturer) is the lead project supervisor. PD contacted the study participants to explain the study aims.

The authors acknowledge that their individual experiences and perceptions might influence the interpretation of the data. While there were no prior relationships with the participants, the recruitment process could have influenced their perception of the study and their subsequent responses. The adoption of grounded theory as this research’s epistemological stance enabled the participants’ own experiences to guide the analyses and the formulation of codes and themes with an inductive approach. Furthermore, by triangulating the qualitative findings with the quantitative survey data, the validity and reliability of the findings were enhanced. There is also robustness in adopting Braun and Clarke’s guidelines [38,39], where the qualitative data was independently analysed by the first author and two other members of the research team, and further cross-checked by a PPI member with appropriate expertise.

3. Results

3.1. Qualitative Results

3.1.1. Participant Characteristics

For SSIs, 10 patients (n=10) in the 50-75 age bracket participated in one-on-one interviews with the first author, which lasted for an average of 41 minutes. Among these, 6 were male (n=6) and 4 were female (n=4). While some participants reported comorbidities (n=3), they engaged in virtual consultations for secondary care of either chronic respiratory or chronic rheumatology conditions. More specifically, 6 were COPD patients (n=6), 3 were rheumatoid arthritis patients (n=3), and 1 had psoriatic arthritis (n=1). The majority had experience with remote phone consultations only (n=6), and the rest had experienced both remote phone and video consultations (n=4).

Online focus group discussions between HCPs and the first author lasted for an average of 44 minutes. Three focus groups were conducted with a total of 15 HCPs (n=4, n=4, n=7) who were all female and aged between 30 and 60. They conducted virtual consultations for secondary care of either chronic respiratory conditions (n=8) or chronic rheumatologic conditions (n=7). There was a mix of professions among the participating HCPs, including clinical nurse specialists (n=7), advanced nurse practitioners (n=3), nurse practitioners (n=2), physiotherapists (n=2) and occupational therapist (n=1). The majority employed remote phone consultations only (n=10), and the remaining HCPs (n=5) employed both remote audio and video consultations.

3.1.2. Qualitative Findings

After comparing and merging themes from the SSIs and focus groups (Appendices 5-6), seven final themes were identified and agreed upon (Appendix 7). Five of them were relevant to both patients and HCPs, one theme applied to HCPs only, and one theme applied to patients only. These were triangulated with the five themes identified in the open-ended sections of the survey (Appendix 8). Each theme is individually reported in this section, along with illustrative quotes that exemplify each description.

Theme 1: Convenient and continuous care through video consultations

While there was a mix of participants using phone- and video-based consultations, there was a preference for the latter modality; in particular, if it includes self-management support and does not disrupt care pathways. Users appreciated video consultations for the visual feedback that they provide, which can improve communication and the quality of medical assessments.

“I think it would take an awful lot of way, of the stress away. Because if you’re trying to explain on the phone and you know the person can’t see it, and if you are in pain, it’s hard to even think… You can’t. It’s hard to communicate, like. So I think that the fact that you could, you could actually show would be... Would take a lot of the stress away, to be honest.” (Patient 4, aged 50-55, female, rheumatoid arthritis)

Virtual consultations were favoured for the convenient access to care that they provide, while eliminating the need to travel. The inclusion of remote prescriptions for medications such as antibiotics and steroids in the event of flare-ups provided a sense of confidence for patients to self-manage their condition while also having the option to remotely consult a HCP if needed. HCPs appreciated the flexibility of remote consultations in assisting in managing their workload, and that they also provide a structured management plan for patients, even if they are not hospitalised.

“Whereas if you call somebody and they don’t answer, you know you can always call the next person. Uhm, whereas if you’re waiting for scheduled visits and they don’t show up, that can be an hour and a half of your day. Uhm, and that can be very frustrating. So at least with the virtual call, it’s more flexible.” (HCP-1d, senior respiratory physiotherapist, aged 35-40, female)

“They’re following a pathway, so they’re not just dropped.” (HCP-2b, clinical nurse specialist, aged 45-50, female)

Theme 2: Optional, hybrid models with familiar, empathetic HCPs

Participants generally did not view virtual consultations as a replacement for in-person visits, but rather as a supplement. They favoured a hybrid virtual care model, which involves occasional face-to-face consultations on top of remote visits.

“So it is great as little as possible not to have to attend the hospital, but on the on the other point of view, sometimes I think it is necessary for the healthcare professional to actually see how is the patient doing and to actually eyeball them to actually see them physically.” (Patient 5, aged 65-70, female, rheumatoid arthritis)

In order to meet diverse needs and preferences, hybrid models should be optional. This would cater for users of differing technological literacy and those who have a sole preference for in-person visits.

“People don’t watch, engage with technology and, uh, they they, you come up with a bit of resistance, even though you’re explaining the benefits.” (HCP-2c, clinical nurse specialist, aged 40-45, female)

Furthermore, the need for empathetic and familiar HCPs was highlighted as a means to improve the interaction quality. This could be addressed by having the first visit in person to build rapport, and subsequent online consultations to involve the same HCPs from the first visit.

“Yes, the first time was in person. And which was good because I got to meet them. You got to know them. You build a sort of rapport with them and everything and like and it was all good. And then everything else was virtual [...]” (Patient 1, aged 55-60, female, COPD)

“The girl who called me for the four or five days, I’ve never met her before. It was a phone call. Just you know, and I was thinking to myself: “It’d be nice to have met her”, you know, to know who I’m talking to.” (Patient 8, aged 65-70, male, COPD)

This theme is triangulated by the survey’s qualitative findings, as it overlaps with the survey theme of “Virtual care access, practicality and equity” which indicated that patient circumstances and limited assessment are factors that do not encourage continued virtual consultation use.

“Fear of inaccurate findings just placed on visual consultation. Face to face consultation more reassuring.” (Survey participant ID 1, aged 65-74, female patient, rheumatoid arthritis)

“discharged from virtual ward” (Survey participant ID 82, aged 55-64, male patient, COPD)

The survey theme “Reduced engagement and interpersonal connection” is also relevant here. Participants noted that patient retention for sustained virtual care use is negatively impacted by poor engagement over virtual means and hurdles in developing meaningful rapport.

“I believe face to face creates a better therapeutic relationship with your medical provider.” (Survey participant ID 18, aged 55-64, female patient, chronic gynaecological conditions)

From the survey respondents, there was also a preference for hybrid models. For future use, users indicate a preference for a mix of in-person and virtual visits to adapt to individual needs and preferences.

“A hybrid model of virtual and Face to Face appointments is probably the future” (Survey participant ID 11, aged 45-54, female HCP)

Theme 3: Technological ease and optimisation for holistic virtual care

Participants considered virtual consultations to be effective if they could provide a comprehensive outlook on patients’ health. Some healthcare aspects that might be tangential to their conditions might be overlooked, such as mental health and well-being, which could provide insights into individual outcomes.

“Say, some kind of a diary that they could keep as regards the influence or the effect of their medication on their mood, and that would be helpful, then in further follow up, virtual follow-up with their healthcare worker.” (Patient 5, aged 65-70, female, rheumatoid arthritis)

“And that whole mental health thing needs to be looked at as much as giving you the tablet to get rid of that, uh, chest infection.” (Patient 3, aged 70-75, male, COPD)

There was also an emphasis on the importance of technology that is intuitive to use and features dedicated technical support. This could facilitate the adoption of more advanced tools, which was the case in one cohort where training was provided to employ the virtual consultation platform.

“Yeah, because if they didn’t come to show me [...] I wouldn’t be able to manage it, like.” (Patient 9, aged 65-70, male, COPD)

“they do all the teaching of the technology and they’re there as a support when things don’t go just to plan” (HCP-2a, advanced nurse practitioner, aged 55-60, female)

Theme 4: Cautious and transparent AI integration in virtual consultations

Virtual consultation users supported a cautious adoption of AI assistance during their remote interactions, as there is concern over the integrity of their data and the validity of the automated output. There is, therefore, a need to maintain human contact and transparency with robust safeguards and control over the AI output.

“I’m very, very… uh… Nervous about invasion and privacy. Uhm, invasion of people’s privacy and just the way the way intelligence, AI can be used.” (Patient 7, aged 55-60, male, psoriatic arthritis)

“It may kind of frustrate rather when you wouldn’t get talking to a person at themselves. You know a person, a real person.” (Patient 4, aged 50-55, female, rheumatoid arthritis)

“I suppose you’ll be concerned about errors” (HCP-3a, occupational therapist, aged 40-45, female)

Such a theme was reflected in the survey as well. Users were open to versatile types of AI assistance, such as in communication, diagnostics, paperwork automation and providing deeper insights into patients’ status. This openness is subject to the tool being transparent, user-friendly, abiding by regulatory standards and does not reduce the human element.

“I don’t want it to replace my interactions with my nurses” (Survey participant ID 92, aged 65-74, female patient, COPD)

“Possibly useful as a diagnostic tool where diagnosis is unclear. Could input all investigations and signs/symptoms so far, to aid in considering other potential rare causes when common are outruled.” (Survey participant ID 7, aged 35-44, male HCP)

Theme 5: Potential AI roles in virtual consultations

While the type of AI assistance varied according to individual needs, users generally recognised AI’s potential to enhance virtual consultations with tailored support and improved efficiency. For patients, consultation highlights, mental health and well-being resources, condition-specific updates and lifestyle advice would be empowering.

“And particularly the the ability to get information and save it and, uh, record it and save it. Uh… And you know, summarise it and then… It has it, it helps the patient to focus on on their problem and and their own responsibility for helping themselves in the problem in, in the whole disease, chronic disease.” (Patient 5, aged 65-70, female, rheumatoid arthritis)

“Some sort of software that if you were having a flare up in the middle of the night that would help you calm down.” (Patient 1, aged 55-60, female, COPD)

“Sometimes when you’re put on a new drug, maybe if an AI education on that drug rather than the, you know, the doctor giving you the lowdown on it, or whatever it might be, you know, good to be able to ask questions and relate like that from, you know, just just find out the background of the drugs and the, you know, what they hope to achieve, and the side, well and the side effects too.” (Patient 6, aged 70-75, female, rheumatoid arthritis)

For HCPs, automated consultation summaries, clarification of medical terms and troubleshooting assistance are favoured.

“If they were mentioning like a condition to you that you didn’t know or haven’t heard about before, that it would be, uh, helpful that it would be able to generate information on that condition for you instantaneously, I suppose.” (HCP-1d, senior respiratory physiotherapist, aged 35-40, female)

“Some sort of dictation, you know, where our consultation could be, maybe all presented back to us so that we could sign off on it.” (HCP-2a, advanced nurse practitioner, aged 55-60, female)

Theme 6: Quality of virtual interactions and assessments are not satisfactory

This theme stood out from the focus groups conducted with HCPs as being relevant to this participant demographic. HCPs highlighted a lack of rigor in virtual clinical assessments and a diminished sense of personal connection with patients.

“face to face definitely gets a more accurate, from a clinical assessment, you get a better, a better picture of what’s going on.” (HCP-3e, advanced nurse practitioner, aged 50-55, female)

“And I suppose the other thing with the the kind of frustrations of the virtual is when you don’t see them walking on the door, like you can tell an awful lot about a patient even watching them as they walk from the chair and to your consultation room.” (HCP-1d, senior respiratory physiotherapist, aged 35-40, female)

Theme 7: Nuanced and paradoxical perceptions of AI

A theme that predominantly emerged from interviewing patients is the contradictory - but multifaceted - feelings that they expressed about AI assistance in virtual consultations. There was an initial sense of anxiety around the technology as well as uncertainty around its potential in healthcare, as exemplified by the participant quotes below:

“You know, it’s more advanced than maybe some some humans are, like. Yeah, that’s scary, right? That’s scary. You don’t know, you know. Is that the whole world gonna be looking at me doing this, doing that, you don’t know. You can’t trust it at the minute, can you?” (Patient 2, male, aged 70-75, COPD)

“My biggest concern, Prans, is, uhm, safety. People’s safety of the information that the AI has on you and it has access to. Uhm, and you know, that is my big concern that anyone would come to harm like via using AI.” (Patient 7, aged 55-60, male, psoriatic arthritis)

“I just find it hard to get my head around AI.” (Patient 6, aged female, 70-75, rheumatoid arthritis)

These could be attributed, to some extent, to a general lack of familiarity with AI’s uses in virtual consultations, as the first author had to provide more prompts to stimulate the conversation further in comparison to other discussion areas. When provided examples of AI assistance to enhance their virtual care experience, the patients were cautiously open to its use.

“I don’t have any concerns about what you’re talking about [...] I would have, I would have concerns about maybe the processes of developing the the modern robot type thing that’s going to take over the world “ (Patient 3, male, aged 70-75, COPD)

“This wasn’t too invasive, I would definitely go for it, like” (Patient 2, male, aged 70-75, COPD)

The need for additional prompts and clarifications highlights the potential need for additional guidance to be provided to patients in order to understand AI’s potential. The paradoxical perceptions of the technology could indicate barriers to adoption.

In the survey, respondents also indicated a lack of familiarity with the technology’s potentials, despite their openess to AI assitance.

“The virtual ward is great but am not sure how we would use AI, but I dont know much about AI” (Survey participant ID 93, aged 75-84, female patient, COPD)

“AI - this is all new to me, I think the nurses would know more” (Survey participant ID 87, aged 75-84, male patient, COPD)

3.2. Quantitative Results

3.2.1. Participant Characteristics

Among the 83 survey respondents, 56 were females (67.5%) and 27 were males (32.5%). The majority of respondents (36.1%) belonged to the 75-84 bracket, followed by the 45-54 bracket (18.1%) and the 65-74 bracket (16.9%). 51 responses (61.4%) were from patients, and 32 (38.6%) were from HCPs. The most common primary conditions for which virtual consultations were employed include COPD (63.9%), rheumatoid arthritis (8.4%) and diabetes (4.8%).

The majority (73.5%) employed phone consultations only, while 24.1% employed both phone and video consultations, and 2.4% used video consultations only. Most respondents (54.2%) reported changing to a hybrid model (which includes a mix of in-person and virtual visits), 22.9% have continued using virtual models, 20.5% stopped using remote care and returned to in-person visits, while 2.4% have switched from one virtual mode to another.

3.2.2. TUQ analyses

The TUQ subscale scores were computed by summing Likert items. The range of scores of each TUQ subscale are as follows, with higher scores indicative of greater agreement with the subscale’s statements: 3-18 for usefulness, 3-18 for ease of use and learnability, 4-24 for interface quality, 4-24 for interaction quality, 3-18 for reliability, and 4-24 for satisfaction and future use. Reliability testing of responses for the 21 items resulted in a Cronbach’s alpha coefficient of 0.96, indicating good internal consistency.

The overall TUQ mean score in this research was 90.6 (SD=15.0). The average scores for each subscale based on the responses obtained are as follows: 13.7 (SD=2.1) for usefulness, 13.2 (SD=2.1) for ease of use and learnability, 17.1 (SD=3.4) for interface quality, 17.0 (SD=3.4) for interaction quality, 12.0 (SD=3.4) for reliability, and 17.5 (SD=2.8) for satisfaction and future use. These indicate a higher level of agreement with the statements and, thus, a generally high satisfaction with the users’ virtual consultation experience. Figure 2 summarises the frequency of responses for each TUQ item.

The means of the TUQ scores for current and former virtual consultation users were compared to identify any potential differences between the groups. The non-parametric Kruskal-Wallis test was performed due to the non-normality of the distribution. This resulted in a p-value greater than 0.05 for each TUQ subscale, which does not indicate a statistically significant difference across the different user groups. The ε² effect sizes ranged from 0.002 to 0.032, indicating negligible to small group--related differences across the TUQ measures. The results of this analysis are shown in Table 2.

3.2.3. Views on AI assistance

For the items on AI assistance, the scores were computed by summing Likert items. The majority of survey respondents (43.4%) strongly disagreed with having an understanding of the potential of AI software in healthcare settings. Similarly, the majority (42.2%) strongly disagreed with having an understanding of the limitations of AI software in healthcare settings. However, 45.8% agreed to being open to trying an AI software that assists in the remote care process. This is concordant with the qualitative Theme 7, which identified a lack of familiarity with the technology, despite being open to its assistance.

As 54.2% and 50.6% of respondents expressed neutral sentiments towards concerns about AI accuracy and safety, respectively, in a healthcare setting, this also indicates a potential lack of familiarity with the technology. Figure 3 summarises the responses for each survey item on the views of AI.

For each individual statement relating to views on AI assistance, the score can range from 1-5, with higher scores indicative of higher levels of agreement with the corresponding statement. The mean scores of the AI views measures were compared across current and former virtual consultation user groups to identify any potential differences, and a Kruskal-Wallis test, selected due to the non-normality of the distribution, was conducted to analyse differences between the groups. The latter resulted in a p-value greater than 0.05 for each statement, which does not indicate a statistically significant difference across the different user groups. The ε² effect sizes ranged from 0.005 to 0.065, indicating negligible to small group--related differences. The results of this analysis are shown in Table 3.

3.2.4. Perspectives of former virtual consultation users

Respondents who reported returning to in-person consultations in survey question 17 (Section 3) were presented with branching question 18, which includes 10 individual, Likert-scaled items to understand their reasons for doing so. The scores for each statement range from 1-5, and higher scores are indicative of higher levels of agreement with each.

Regarding whether returning to in-person visits improved the quality of care experience, the mean score was 3.5 (SD=1.0). There were lower levels of agreement in finding the use of technology during virtual consultations discouraging in seeking/delivering healthcare, with a mean score of 2.9 (SD=1.0). In regard to finding in-person consultations more convenient for user needs, the mean score was 3.1 (SD=1.1). A mean score of 3.6 (SD=0.8) was obtained in finding virtual consultations more accessible and flexible for users’ schedules. There were higher levels of agreement that a hybrid option would be more convenient, with a mean score of 4.0 (SD=0.8). Respondents scored a mean of 3.5 (SD=1.1) towards finding that face-to-face interactions led to better communication. The statement that virtual consultations allow for better communication scored a mean of 2.9 (SD=0.8). The average score for finding in-person visits more comfortable and secure was 3.5 (SD=1.1). A mean score of 2.9 (SD=0.7) was obtained for feeling less anxious about attending appointments with virtual consultations. Respondents provided an average score of 3.2 (SD=0.5) towards considering using remote visits again with the availability of an assistive AI tool to effectively address some of their needs and challenges.

The distributions of the scores for each item are displayed in Figure 4.

4. Discussion

With a focus on real-time virtual consultations for noncommunicable chronic conditions, this mixed-methods investigation aimed to gain insights into: i) the experiences of current and former users, ii) the needs and improvements identified by users, and iii) the user views of an assistive AI tool.

4.1. User experience of virtual consultations

This study found that there is a general satisfaction with virtual consultations among its current and former users, identified in Theme 1 and supported by the generally positive sentiments in the responses across each TUQ subclass (Figure 2). Our sample yielded an overall TUQ mean score of 90.6 (SD=15.0), indicating high usability, which aligns with ratings from recent studies. In Kasim et al.’s cross-sectional survey, the overall TUQ mean (on a 5--point scale) among HCPs in Iraq was 4.8 (SD=0.88) [50], and Sayed et al.’s study on perceived telehealth usability among adults from the general population in Saudi Arabia identified that nearly two-thirds (60.4%) of the participants have a high telehealth usability score [51]. Similar findings are echoed in Noceda et al.’s study on patient satisfaction with telemedicine in the Philippines [52], and a national survey in the USA on patient perspectives of telehealth for fertility care [53]. The convergence of our findings with these diverse cohorts supports the conclusion that, from an HCI perspective, virtual consultations deliver usable, acceptable, and satisfying experiences across different settings.

Based on the findings of this study, users appreciated the modality’s convenient and continuous care provision, especially with adjunct services such as remote medication prescription, as they empower patients to better self-manage their condition. This finding aligns with similar recent studies investigating user experience and satisfaction of virtual consultation [52,54,55], and reinforces the notion that virtual consultations are likely to persist as smart healthcare models in the near future, especially among rural communities.

In particular, there is a clear preference for hybrid models of virtual care delivery (Theme 2). This would cater for diverse user needs and preferences. Even those who have switched to in-person visits would find a hybrid option more convenient, as they did not find face-to-face consultations more or less convenient for their needs while they tended to appreciate the accessibility and flexibility of virtual consultations (Figure 3); therefore they could potentially adopt virtual models if a hybrid option would be at their disposal. However, this should be interpreted with caution as the Kruskal-Wallis test did not indicate a statistically significant difference across the different virtual consultation user groups (all p > 0.05), and the ε² effect sizes ranged from 0.002 to 0.032, indicating negligible to small group--related differences across the TUQ measures (Table 2) [56]. Nevertheless, Aldakhil et al. suggest that a hybrid approach for remote chronic care management might be necessary to ensure comprehensive patient safety [57].

For improved communications and assessments, video consultations are favoured; although HCPs are not satisfied with the quality of virtual assessments (Theme 6) due to the inherent limitations of the modality when it comes to physical and laboratory tests, which can lead to diminished rigour. These are widely recognised limitations of remote modalities [18,58], and could potentially be motivations to reverting to in-person visits, as former virtual consultation users indicated higher levels of agreement in the survey in regards to improved quality of care experience by returning to in-person visits (Figure 3). There is some indication that self-assessed measures at home can be reliable [59,60], and remote assessments could be supplemented with portable diagnostic devices to improve remote assessments, although work still needs to be undertaken in regards to standardisation and facilitation [59]. It could therefore be recommended for future smart healthcare systems to consider the integration of portable diagnostic devices in the provision of remote care.

4.2. User needs and suggested improvements to enhance virtual interactions

While themes 1 and 2 of this study reveal user preference for video consultations and a hybrid approach, Theme 2 highlights the need for a strong empathy element during such interactions. The latter aspect seems to be limited over virtual visits as HCPs are also dissatisfied with the level of personal connection with patients, based on Theme 6. This might be a factor in determining sustained virtual consultation use, as those who returned to in-person visits showed higher levels of agreement towards better communication during face-to-face interactions (mean=3.5, SD=1.1) than with virtual means (mean=2.9, SD=0.8) (Figure 4). This can be considered as unexpected, as empathy would be a necessity in patient-HCP interactions, considering the positive impact on patient outcomes [61], but appears to be diminished in virtual interactions based on this study’s findings. While Budd et al.’s review found that empathy can be effectively conveyed over telecommunication means [62], HCPs might not be confident in the format and could benefit from adequate training. The lack of physical presence in virtual consultations reduces the available cues [63], making it challenging to display and develop empathy and understanding. This current study found that those who returned to in-person visits were in higher agreement that such consultations are more comfortable and secure (mean=3.5, SD=1.1), while they displayed lower levels of agreement towards feeling less anxious about attending appointments with virtual consultations (mean=2.9, SD=0.7). This could be attributed to the lower levels of rapport building, which might lead to poorer acceptance of remote interactions. As Duffy et al. posit that virtual visits require alternative means to develop the therapeutic relationship [64], the need to better equip HCPs to interact with patients virtually seems to persist in the post-pandemic context.

The qualitative element of this study unearthed a general satisfaction with virtual consultation technology, with even former users not particularly discouraged in seeking/delivering healthcare through technological means. However, the importance of intuitive, friction-free technology use was stressed, with training facilitated as needed (Theme 3). The provision of training in virtual consultation use, in particular when unfamiliar platforms and interfaces are involved, can be recommended for current and future practices, as addressing such needs has been shown to improve acceptance of remote care modalities [65].

Our findings also identified that additional technological assistance during virtual consultations could support the provision of a comprehensive outlook on patients’ health. HCPs would benefit from additional health data of their patients, such as through wearables and portable health monitors, while patients favour a comprehensive access to and delivery of care remotely to better understand and manage their condition. This can be addressed by integrating portable medical sensors with validated output to ensure their accuracy [66]. The aspect of mental health and wellbeing, which has a significant association with chronic conditions [67], was also stressed. Telehealth has been identified as a suitable means to deliver mental health services [68], and such services could be integrated in virtual consultation platforms via dedicated counsellors or automated means in preparation for a future with holistic smart healthcare systems. Figure 5 provides a visual depiction of the potential supports that this study has identified.

4.3. Views on Artificial Intelligence Supplements in Virtual Consultations

Virtual consultation users expressed interest in AI assistance, acknowledging its potential to improve the remote care experience as identified in Theme 5 and supported by the survey results. This sentiment was also present among those who returned to in-person visits, as they indicated higher levels of agreement towards considering using remote visits again, with the availability of an assistive AI tool to address some of their needs and challenges effectively (mean=3.2, SD=0.5) (Figure 4). This should be interpreted with caution, as the Kruskal-Wallis test did not indicate a statistically significant difference between the responses of current and former virtual consultation users (all p > 0.05), and the ε² effect sizes ranged from 0.005 to 0.065, indicating negligible to moderate group--related differences (Table 3) [56].

The variance in user views on AI assistance could be explained by UTAUT constructs of social influence and facilitating conditions that determine the usage intention and behaviour, and user behaviour, respectively [13]. Users identified versatile AI uses based on their individual needs, such as condition-specific updates and automated consultation summaries, reflecting the breadth of the technology’s potential in remote care contexts [8], while also indicating the need for further improvements in this space.

However, such positive attitudes are contingent on cautious and transparent integration of the technology, as Theme 4 stressed concerns over data integrity and accuracy. This echoes findings of other studies [69,70], and such apprehensions could potentially be addressed by maintaining human control and supervision over AI outputs. This finding emphasizes the importance of human factors in consideration of integrating AI technologies in virtual chronic care contexts.

Nevertheless, there is a marked indication that there is a lack of familiarity with AI’s potential in virtual consultations and healthcare settings, despite the cautious openness to its assistance. This is highlighted in Theme 7, as well as most survey respondents strongly disagreeing with having an understanding of the potential (43.4%) and limitations (42.2%) of the technology (Figure 3). Such a nuanced perception of AI in virtual consultations might result from the novelty of the technology and a lack of exposure in this particular context. Needs-adjusted training for the use of AI in healthcare can potentially facilitate its acceptance [69], and can be a recommended approach when integrating such software in virtual consultation programmes.

4.4. Future Research and Recommendations

Based on this study’s findings, several evidence-based recommendations can be derived for current and future virtual consultation for chronic care practices; particularly in rural and under-served contexts. Considering the general preference for hybrid models, virtual consultations should be offered with a mix of in-person and virtual video-based visits. The first interaction should be conducted in person for initial rapport building, and patients could be trained in employing virtual consultation software and provided with portable health monitoring tools to facilitate remote clinical assessments. To enhance the therapeutic relationship, the same clinical team with familiar HCPs is recommended to consult individual patients.

These findings have broader relevance for remote care policy as our analysis addresses some of the OECD’s recommended priorities for policymakers in regards to telemedicine use [71]. Due to prevalent dissatisfaction with the quality of interaction, interpersonal relationships, and clinical assessment rigour over virtual means, policymakers are recommended to invest in adequate infrastructure and training before deploying video-based remote care programmes to facilitate sustainable adoption of the technology. To cater for diverse user needs and preferences, they should ensure that hybrid virtual consultations for chronic conditions are offered as options rather than replacements to traditional encounters, provided that in-person visits are feasible.

Furthermore, as AI technology becomes increasingly integrated in virtual consultation pathways, regulators need to enforce standards to ensure their safety and reliability. The EU AI Act is considered a significant step in this direction, but it requires additional refinements to address its shortcomings regarding the interests of patients and sector-specific guidelines [72]. This study’s participants displayed a nuanced acceptance towards the technology, and improved understanding of its capabilities could address user concerns and build trust in AI-enabled systems.

Considering the measured openness to AI assistance, a range of AI assistance is proposed in Table 4 with output that is reviewed and editable by human supervisors while maintaining the patient-HCP interaction. These can guide the development of evidence-based AI tools to supplement virtual care delivery for chronic conditions. It can further be recommended that future work on AI integration in virtual consultations to adopt a participatory approach with end-users, such as co-design methodology [73]. This could integrate their valuable insights to address real-world needs and nuances. Their inclusion could further address limitations around flexibility, accessibility, and usability that digital health and AI-driven healthcare solutions often lack [74,75].

4.5. Limitations

Limitations of this study must be acknowledged. Firstly, as data was collected from the public health sector in the Northwest region of Ireland, caution is advised when generalising the findings. There is also the potential for some level of bias towards digitally literate participants in the qualitative stage, which was mitigated by the more widely-distributed survey. While the survey was administered for triangulation purposes and to gather additional insights from a broader set of participants, its sample size and the nature of self-reporting can limit its generalisability. In addition, the Kruskal-Wallis tests indicate association and not causation, and their low statistical power indicates the need for caution when interpreting the findings. Therefore, the survey findings can be considered more exploratory than confirmatory. Nevertheless, the rich qualitative data and the alignment of the themes with the quantitative findings strengthen the validity of the interpretations. Future work could focus on broadening the sample size and could consider validating or expanding the author-developed questionnaire items.

5. Conclusions

As healthcare evolves in response to technological progress, it is crucial to understand the underlying factors in view of improving outcomes within smart healthcare systems. This mixed-methods investigation has uncovered important HCI and usability insights from the perspectives of virtual consultation users in the post-pandemic context. It has found, in particular among rural areas, a general positive satisfaction with technology-mediated care delivery modalities, with a preference for video-based, hybrid models. Remote interactions might be lacking in terms of empathetic care delivery and the rigour of clinical assessments, indicating the need for improvements in terms of both technological and human factors. There is potential for AI assistance to address these limitations of virtual care delivery and this study has identified practical evidence-based AI features that could support the development of corresponding intelligent human-computer interfaces to encourage sustained use. However, acceptance of such AI support is nuanced, indicating a strong need to maintain patient-HCP interaction; thereby highlighting the importance of human factors in technological upgrades in remote chronic care contexts. This study’s findings provide practical insights for current and future virtual care delivery, policy development and evidence-based software development that can promote and support successful chronic care management. In line with the evidence base and recommendations from this study, future work could co-design AI prototypes with end-users and further pilot them in real-world settings.