What to measure? Development of a Core Outcome Set to assess remote technologies for cochlear implant users.

1. Introduction

Opportunities to access cochlear implant (CI) care via remote technologies, rather than the traditional method of accessing care through in-person appointments at specialised clinics, have rapidly expanded in recent years [1,2,3,4,5,6]. Various studies have shown that both synchronous and asynchronous options for remote CI services, such as intraoperative CI telemetry, implant programming, electrode-specific measures, and post-operative assessment of speech recognition, management, and review are possible and feasible [5,6,7,8,9]. The use of telehealth for CI service provision has the potential to significantly improve efficiency, effectiveness, and equity of care for CI users, in a personalised manner. However, it is vital that its implementation is well-considered [1,10,11].

Common barriers to integration of remote care into hearing services for either CIs or hearing aids include lack of adequate funding frameworks; poor integration with current clinical practices; mistrust of the accuracy and quality of the remote care service; measures and outcomes; and audiologists’ confidence in their clients’ ability to utilise the remote technology [1,2,12,13,14]. Nevertheless, most studies show that use of a hybrid system in which both remote and in-person care is provided is the preferred method of service delivery both by clients and audiologists [9,13,14,15].

In order to implement remote care effectively into clinical practice, it is essential to demonstrate that remote service provision provides equivalent, if not superior, care compared to the current standard of in-person clinical care [16]. This is necessary for regulatory purposes, as well as to ensure that CI service providers and users have sufficient trust and confidence in the outcomes of the remote service to consider using it.

Within audiology, a vast number of outcome measures exist for measuring the effectiveness of CIs and hearing aids [17,18,19]. Danermark et al. [20] suggested a concise set of outcome measures for assessment of hearing in general, and Allen et al. [21] identified a core outcome domain set for hearing rehabilitation, primarily for hearing aids. These sets of measures and domains, however, do not address some of the specific auditory issues associated with severe-profound hearing loss, or the technical issues associated with the use of CIs. More specific to CIs, Andries et al. [22] recommended a CI-specific outcome assessment protocol for adult CI users which an expert group of CI professionals selected based on the WHO international classification of Functioning, Disability and Health (ICF) framework. The ability to assess outcomes for CIs, such as speech perception, is particularly problematic in remote care given the difficulties determining presentation levels and establishing standardised test environments (e.g. a sound-treated booth), compared to in-person clinical measures. Currently, there is no set of measures for CI specific outcomes when also used in combination with remote technology.

The use of relevant and sensitive outcome measures to evaluate CI services delivered via remote technologies is vital to facilitate the provision of evidence-based health care services, allowing stakeholders to make informed decisions about how to best care for their patients. The current approach to audiological outcome measures is essentially non-standardised [23] both for in-person and remote services, making it difficult to compare and integrate results across different studies and services, for example in systematic reviews with meta-analyses.

To address these issues, over the last decade there has been an increase in the development and use of core outcome sets (COSs) [24]. A COS is an agreed standardised set of outcomes that should be measured and reported as a minimum dataset for a specific condition [25], ideally with input from end-users including patients, clinicians, industry and other key stakeholders. Outcome measures are identified as part of pre-specified outcome domains. A core outcome domain set (CODS) has been defined for hearing aids with separate, and significant, input from both patient and hearing care professional stakeholder groups [26] based on best practice guidelines [27], however there is nothing similar for CIs. Traditionally, CI outcomes have focused on the domains of speech perception and CI uptake, although there is growing evidence that self-reported measures offer a more functional real-world outcome, tapping into different mechanisms of benefit to speech perception outcomes [28,29]. More recently, there has been a growing number of self-reported measures specific to cochlear implants [30,31], although the extent of how they are used in clinical practice is unclear.

With the increase in use of remote technologies there is a need to consider the meaningful domains that are specific to these technologies, which may also be relevant to in-person services. For example, empowerment has recently emerged as a feature of remote technologies [32,33], but it likely also applies to in-person services. Furthermore, there are other considerations specific to service delivery of remote technologies that are often identified as benefits to both patients and services, such as reduced time, convenience and costs [1,2,12,34].

An essential tenet of this research is to be able to demonstrate the equivalence of remote care services, either as a stand-alone or hybrid model of care, to the ‘gold-standard’ in-person clinical model of care. Thus, the overall objective of this research was to develop a COS to evaluate remote technologies delivered within CI services to maximise the potential benefits of this model of care. Feedback was sought from relevant parties (e.g. CI users and their families service providers, including management and clinicians, CI manufacturers, CI advocacy groups) to ensure a broad range of perspectives were considered. Outcome domains encompassing measures specific to the delivery of remote technologies in terms of both (i) patient outcomes (i.e. benefits of remote technologies for CI patients), and (ii) service delivery, were included, ensuring the outcomes can be easily integrated into clinical care. This paper reports on the final phase of a broader 3-phase study (see Figure 1) that followed the COS development roadmap described by Hall et al (2015). The study was registered on the COMET (Core Outcome Measures in effectiveness Trials) website https://www.comet-initiative.org/Studies/Details/2586.

Phase one of the study included a systematic review of outcomes measures identified in studies documenting use of remote services for the provision of CI and hearing aid care [29]. A total of 250 different outcome measures were identified, with CI studies revealing significantly more outcomes in the ear and labyrinth domains compared to hearing aid studies (43% vs 10%), and hearing aid studies revealing significantly more outcomes in the cognitive (28% vs 5%) and emotional (35% vs 10%) domains than CI studies.

Phase two involved a combination of stakeholder workshops with CI users and their significant others, CI professionals, and hearing advocates, followed by a series of three parallel e-Delphi reviews conducted separately for CI professionals and CI users across Australia, the UK and USA. This utilised a methodology described by Allen et al [35] outlining development of a CODS for adult CI outcome domains. This phase aimed to identify by consensus, the most important outcome domains based on stakeholder input [35]. The Delphi review assessed 58 domains across three supradomains: Service, Clinical (assessment-based) and Patient (self-report). The top three domains in which consensus of ≥80% was achieved within each supradomain for both groups (i.e. CI users and CI professionals), are shown in Table 1. Agreement was good for the Service supradomain, however consensus was poorer for the Clinical supra-domain and there was no between-group agreement for the Patient supra-domain. Many domains ranked highly by CI users were ranked far less important by professionals.

The aim of the final phase of the study reported in this paper, Phase 3, was to identify a COS to evaluate remote technologies delivered within CI services based on the previously defined CODS. Due to the substantial mismatch in outcome domains for both the Clinical and Patient supra-domains between CI users and CI professionals noted in Phase 2, Allen et al [35] recommended inclusion of domains ranked most-highly by CI professionals for the Clinical supra-domain, and by CI users for the Patient supra-domain in an interim, pragmatic COS. This would facilitate an easy transition into a robust, pragmatic, and clinically acceptable COS utilising clinical measures used regularly and trusted by CI programs across Australia and New Zealand.

2. Materials and Methods

Outcome measures that mapped onto the CODS were selected from those identified in Phase 1, as well as measures identified by the research team as commonly used in clinical care in Australia and New Zealand were included in Phase 3 of the study. The list of identified outcome measures was appraised according to their content validity, and their development methodology to determine which outcome measures to include in Phase 3. The final list of included measures, consisting of 43 PROMs and 10 speech perception measures (see Supplementary Tables S1 and S2) was presented to experienced CI clinicians from Australia and New Zealand in a single round online survey. Each clinician was asked to rate the outcome measures for their use of the measure, and if used, the measures’ usefulness, trustworthiness, ease of use and likelihood of future use if it were recommended to them.

2.1. Single Round Online Survey

Individuals who agreed to participate completed a single round online survey utilizing Qualtrics software. The online survey consisted of questions about use of 10 speech perception tests and 43 patient-reported outcome measures (PROMs) using a 4-point categorical scale; (never heard of, never used, occasionally used, regularly used). A short description was provided for each PROM listed. For example; “The Social Participation Restrictions Questionnaire (SParQ) is a hearing-specific, patient-reported outcome measure that was originally developed through consultation with adults with hearing loss, clinicians, and researchers. It has 19 items, each assessed on an 11-point scale. Responses are averaged to form two subscales: Social Behaviours and Social Perceptions”. A copy of the survey is available in the Supplementary Digital Content.

Eligibility criteria for participants were: recent or current CI clinicians from a range of large CI clinics and research institutes and identified by the research team as having extensive knowledge of current CI clinical practices, and/or extensive knowledge of currently available adult-focused CI outcome measures used in Australia and New Zealand. Clinicians included experienced clinicians who participated in Phase 2 [35]. They were invited to participate via an email message.

When participants indicated that they had regularly, or occasionally used a measure, they were asked to provide a rating, using a 5-point Likert scale, ranging from “strongly disagree” to “strongly agree” for each of the following statements;

• This measure is easy to use in clinical practice (ease of use)
• This measure gives results that are trustworthy/believable
• This measure gives results that are useful in clinical practice
• I would use this measure in clinical practice if it were recommended to me

Respondents were also asked about other clinical measures they used as part of their standard protocol; their usual approach to testing asymmetrical hearing losses; and the factors that they considered when choosing a speech test to ensure that no measures were missed.

Statistical analysis was conducted in Python (v3.11.0) [36]using pandas (v2.2.1) [37], numpy (v1.23.5) [38], and scipy (v1.11.4) [39] and scikit-learn (v1.4.2) [40]. Graphics were generated using matplotlib (v3.8.4) [41]. Descriptive statistics were calculated for demographic variables and survey responses.

2.2. Online Final Recommendation Workshops

Two Final Recommendation workshops, approximately 90 minutes each in duration, were held online through Microsoft Teams videoconferencing software with CI users and CI professionals to finalise key recommendations for the interim pragmatic COS. The outline presentation of the study was provided, one for professionals, and a lay person version for CI users, summarising the key results from the CODS and the current study. A semi-structured interview guide (see final workshop agenda in supplementary material) was developed. Questions included:

What domains should be included in future iterations of the COS?

(CI professionals only) Which outcome measures or subdomains should be recommended as a minimum standard?

How should we prioritise outcome measures within each subdomain?

Participants consisted of two groups: 1) Adult CI users, and 2) CI professionals aged ≥18 years of age with sufficient self-reported English proficiency to participate in the workshop. Adult CI users were required to have at least 6 months experience using a CI. CI professionals were required to have at least 12 months experience providing CI services to adult CI users. Individuals with self-reported disability, other than hearing loss, that precluded full participation in the workshop were excluded.

3. Results

3.1. Single Round Online Survey

Twenty CI clinicians from Australia (n=18) and New Zealand (n=2) participated, 17 of whom responded to both the PROM and the speech perception test familiarity survey questions. Participants’ clinical and research experience is detailed in Table 2. Home and work postcodes for Australian participants were mapped to the Index of Relative Social Advantage and Disadvantage (IRSAD) decile with all Australian participants living in the top 30% of postcodes, and working in the top 50% of postcodes, suggesting that participants skewed toward relative social advantage. For participants from New Zealand, postcodes were mapped using the New Zealand Index of Deprivation for 2023 [42], with one participant living in the top 30% of postcodes bur working in an inner-city location in the bottom 30% of postcodes, and the other living in the bottom 30% of postcodes but working in the top 50%.

3.1.1. Familiarity Ratings (Speech Perception and PROM Measures)

A summary of familiarity ratings is shown in Table 3. Participants were most familiar with speech perception measures. Four speech perception measures (BKB-A, CUNY sentences, CNC words, and AB words) fell within the top five most-used outcome measures and were regularly used by 50% of participants. The DIN/DTT test, BKB-SIN and QuickSIN were occasionally used by > 50% of participants. The remaining three speech perception tests (HINT, Austin and AzBio) had either been “never heard of “ or “never been used” by 59%, 53% and 76% of participants respectively.

Familiarity with PROMs was lower than for speech perception measures. The SSQ (90% of participants) and the SSQ-12 (75% of participants) were the most used PROMs. Participants used the SSQ-12 (70%) slightly more regularly than the SSQ (65%). The Glasgow Hearing Aid Benefit Profile (GHABP), Hearing Handicap for the Elderley (HHIE), Abbreviated Profile for Hearing Aid Benefit (APHAB) were occasionally or regularly used by at least half the participants. No participant regularly used the short or revised version of the HHIE. The Nijmegen Cochlear Implant Questionnaire [55] (NCIQ), and the Cochlear Implant Quality of Life Questionnaire (CIQoL Profile and CIQoL-Global) [64], which have been recommended by the Adult Hearing Standards of Care; Living Guidelines [91] were not commonly used. The NCIQ was only used by 5% of participants regularly, and by 45% occasionally. The CIQoL was used only occasionally by 35% (Global version, 35 items) and 20% (Profile version, 10 items) of participants.

3.1.2. Ease of Use, Trustworthiness, Usefulness and Likely Recommendation to Use Ratings

Ratings were provided for ten speech perception measures (Figure 2), and 21 PROMs (Figure 3) which had been used by ≥3 participants. The CIQoL Profile ratings were also included although they had only been used by 2 participants. Measures with which participants were more familiar were, in general, considered easier to use (τ_B = 0.383, p < 0.001), more trustworthy (τ_B = 0.323, p < 0.001), and as providing more useful results (τ_B = 0.300, p < 0.001). Participants also reported that they would be more likely to use them in practice, if they were recommended (τ_B = 0.406, p < 0.001) (Figure 4). Ratings were generally high, with very few respondents disagreeing with any of the statements.

The correlation between ratings of ease of use, trustworthiness, clinical usefulness and willingness to use outcome measures was assessed using univariate and bivariate kernal density plots, and correlations between all rating scales were high (see Supplementary Figure S1).

3.1.3. Free Text Responses

Participants suggested several additional PROMS not listed in the survey (Supplementary Table S2) including the Australian Quality of Life Scale (AQoL; n=4), the Strengths and Difficulties Questionnaire (n=3) and the Listening Effort Questionnaire (LEQ-CI; n=3).). The most recommended physiological test was Neural Response/Auditory Response Telemetry (n=3). The Ling Sounds speech sounds identification test was also suggested (n=3).

The most important factors to consider when choosing a speech perception test (see Supplementary Table S3) were: Tests available in the primary language (n=11), and accent (n=7) of the CI user, Cognitive appropriateness (n=4), Speed of delivery (n=2) and measure length (n=2) (see Supplementary Table S3).

3.2. Final Recommendation Workshops

Transcriptions of the final workshops were reviewed and summarised by authors CS and MF, then reviewed by all other authors. Key findings from the workshops are shown in Table 4.

Clinicians felt that future outcome measures should include domains such as cognition, listening effort, listening fatigue, empowerment, social connectedness, relationships, and fatigue. There was a general consensus between both groups that more holistic measures of CI outcomes are needed to provide a more comprehensive understanding of the communication difficulties of CI users and their real-life impact. A combination of different types of measures and understanding their interactions was considered crucial for advancing the field and improving clinical outcomes. However, with the emergence of newly developed measures, interpretation of results may be challenging due to lack of clinician familiarity. Thus, when considering the introduction of new outcome measures, it is vital to consider training as part of implementation, to raise awareness, familiarity and ensure trust in the data obtained with the measure.

CI users felt interaction with the clinician in some aspect of the remote service was important to feel engaged in the process. They recommended that remote services be well-considered and designed to ensure a seamless process for all aspects of the service, from enrolment, validation of enrolment, and login to completion of the remote checks and appointments, payment etc. Whilst benefit was seen in the ability to adjust CI settings remotely, CI users reported it was essential that there was a built-in fail safe or reboot option at the CI user’s end if the service failed midway for some reason.

4. Discussion

Whilst remote care has been shown to be a feasible option for CI service provision, uptake and sustained use of such services has remained low. Research has shown that key barriers to it’s use include concerns about the accuracy and reliability of remote technology methods, limited confidence in the ability of client’s to access and use communication technologies, and the inability to demonstrate that remote services can provide equivalence of service compared to ‘gold standard’ in-person care [12]. In order to compare remote and in-person services effectively, as well as address the concerns about accessibility, usability of services, and accuracy of results, it is essential that the same set of outcomes measures that are sensitive and meaningful to both CI users and CI professionals, are compared across clinics, across modes of service provision, and clinical trials/studies.

While the COS recommended by Andries et al. [22] included CI-specific outcome measures, the measures and domains selected for inclusion were not selected with the input of CI users but rather by a core group of CI experts. While well-known, commonly used instruments and assessment methods were identified, several of the PROMs selected were not designed to current recommended best practice, e.g. using consumer input, considering the risks of bias, or following evidence-based criteria for good psychometric measurement properties [92]. Nor were they CI-specific. Finally, the measures recommended did not specifically consider implementation within a remote care service, and the issues associated with this. The COS [22] included a large number of measures including: PROMs (Work Rehabilitation Questionnaire [WORQ; 59 items], Abbreviated Profile of Hearing Aid Benefit [APHAB; 24 items], Audio Processor Satisfaction Questionnaire [APSQ; 15 items], Speech Spatial and Qualities of Hearing Questionnaire [SSQ-12; 12 items], Hearing Implant Sound Quality Index 19 [HISQUI19; 19 items], and Audiometric measures; Aided Pure tone audiometry, Speech perception [Monosyllabic words in quiet, Sentences in noise], and Sound localisation.

Several of the domains identified as most important by CI users and CI clinicians in the earlier stages of our study [35], were not included in the domains in the above-mentioned COS. Of particular note, device integrity and status, device use and hearing-related quality of life. Understandably remote service domains of reliability, accessibility and ease of use, were not included. In Phase 2, a complete lack of consensus between CI users and CI professionals was observed in the most important domains for self-report “Patient” measures and there was only limited consensus for objective “Clinical” measures. Thus outcomes recommended in the COS by Andreis et al’s [22] may not be important to CI users given the lack of CI user input. Further, our final recommendation workshop revealed a general consensus between CI users and CI clinicians for a minimalist approach to the number of core measures. One must consider the potential for overburdening CI users and clinicians with multiple outcome measures, particularly PROMs with large numbers of items, some of which ask similar questions. A large number of time-consuming outcome measures may result in poor completion compliance, and inefficient or limited clinical use of completed measures. Measures completed or received immediately prior to, or during an appointment, particularly PROMs, may be difficult and time-consuming for the clinician to analyse appropriately during the appointment.

Although the original aim of this study was to develop a COS for CI users utilising remote technology in Australia and New Zealand, this has proved difficult for several reasons:

1. Lack of consensus between CI users and CI professionals on the most important domains for the patient supra-domain [35] mean that implementation of a concise COS is problematic if one is to measure the most important domains within each supra-domain.

2. Lack of well-designed and/or well-validated outcome measures for some of the domains rated as most important to assess. Rigorous development and assessment of novel outcome measures is therefore required.

3. Current clinical practice trends in Australia and New Zealand, observed in our online survey of CI clinicians, indicate that CI services rely predominantly on a relatively small pool of specific speech perception measures as the primary measure of CI outcomes. Clinicians are far less familiar with most PROMs that align with the CODS.

4. Several of the outcome measures identified and explored in the outcomes survey have proprietary test materials, technical requirements, and licencing costs associated with them.

These issues present several problems when considering wide-spread implementation of a COS for remote technology into well-established CI clinics. Large CI clinics often perform retrospective analysis of CI outcomes over time as an important indicator of the success of both individual CI users and CI clinics as a whole [93]. Thus, implementation of a brand-new set of outcome measures must consider the impact on the ability to compare outcomes over time. It may be necessary to align the outcomes of new measures with pre-existing measures for a period of time, in order to retain the ability to compare outcomes over time. Significant support, resources and training around measures that are new to the field, or simply just new to the clinic will be required.

Clinicians must have confidence and trust in the recommended outcome measures, both in the methods of data collection and interpretation of results, as evidence by the high levels of correlation between the ease of use, trustworthiness, usefulness, and recommendation ratings provided by participants. Familiarity with an outcome measure engenders an understanding of effective methods of use and interpretation, and in turn outcome measures that are inherently easier to apply and interpret are more likely to become part of existing clinical practice [94,95,96]. Measures that provide trustworthy results are also more likely to be considered useful in clinical practice. Understanding which of these four factors of clinician experience, if any, are primarily responsible for positive clinician experience and uptake is essential to support implantation efforts. This is particularly salient given our findings in relation to some more commonly used surveys such as the CIQoL and the NCIQ, both of which were recommended outcome measures in the recently drafted Adult Hearing Standards of Care; Living Guidelines [91]. Our study revealed poor ratings for ease of use, likelihood to use if recommended and, to a lesser extent usefulness for the NCIQ which would indicate that it is unlikely that this measure would be readily adopted into CI clinical practice in Australia or New Zealand. Furthermore, the NCIQ contains 60 items and so fails to meet the recommendations of our CI users about shorter, more concise measures. The CIQoL, whilst receiving relatively good ratings for all four categories, had only been used by a maximum of four clinicians, thus training for its implementation is required.

In light of these considerations, it appears most appropriate to recommend a pragmatic, interim COS for remote technologies for CI users in order to facilitate uptake into current clinical practice with the recognition that CI outcomes are constantly evolving [91], and as such, so are the important outcome domains, and outcome measures with which to assess them are also evolving. Furthermore, it was decided to limit measures to those commonly used in English-speaking countries in the first instance, as Australia and New Zealand were the focus of this study, to further facilitate compliance with use of the COS as commonly used outcome measures differ substantially across countries. There was a strong focus in the workshops on the need to ensure that implementation of any new set of outcome measures did not overburden either CI users or CI clinicians. Whilst there was a push to utilise more meaningful, “real-life” outcome measures, this was not to be at the expense of additional time and effort for key stakeholders. In fact, the preference was for a reduction in time allocated towards assessment of outcomes. Similar findings have been noted in other allied health fields [97]. However, a reduction in the length of speech test lists, or the number of questions in surveys should not be at the expense of a reduction in their psychometric properties, such as test-retest reliability and validity. Any recommended outcome measures must have, and retain, good psychometric properties to ensure their usefulness.

Recommended Interim, Pragmatic COS

4.1.1. Service Outcomes

A single Likert-item for each of reliability, usability and acceptability, with an option for free text.

Wording needs to be further defined but a regularly mentioned example was a 5-response option based on agreement (e.g. the remote technology was reliable: strongly agree to strongly disagree)

4.1.2. Clinically-Measured Outcomes

CNC (or similar CVC) and optional digit triplet testing (DTT), testing device integrity/system check, device use, and adverse events

Recommended included outcome measures are:

An adaptive speech test that is presented in noise but could also be completed in quiet depending upon the CI user’s speech perception ability. Of the four most regularly used tests identified by CI professionals (BKB-A, CUNY, CNC and AB), the CNC test was included as this is a current requirement for CI candidacy determination.

The DTT is a speech in noise test that is often delivered remotely, thus having the appropriate underpinning architecture for delivery via remote technology systems. Other advantages of this test include that it is often delivered adaptively, the digit material is easily translatable into other languages with easily understandable stimuli, it can be delivered without the need for calibration equipment, and it is widely used across the world.

Device integrity and status, device use, and adverse events were the other three most highly rated clinical tests in addition to speech perception testing. Both groups felt it was vital to ensure that both internal and external components of the CI were functioning appropriately (device integrity) to ensure that any outcome measures additional to these measures are not impacted by device malfunction. Device use, via datalogging, has been included in the COS to ensure that limited CI outcomes are not the result of limited CI use. We acknowledge, however, that there are known discrepancies between reported use and logged data, and that this discrepancy could be due to either technology errors or the user’s decision not to report limited use.

Any potential worry about device usage should be discussed in a supportive and caring manner with the CI user.

Measures excluded include:

The BKB-A test due to fixed level presentation of sentences, the fact that it was originally developed for a low (kindergarten age) literacy level, so it is somewhat child-like, resultant ceiling effects, and its typical presentation mode in quiet.

The CUNY sentence test due to ceiling effects, and the relatively high language knowledge/literacy level required, in addition to the potential influence of auditory memory on outcomes.

The AB word test due to the limited number of lists available, which could lead to practice effects, and because the Australian version of the test materials is no longer available for purchase from the National Acoustic Laboratories.

Speech sound identification/discrimination assessment (e.g. the LING test), whilst rated highly by CI users, was perceived by CI professionals as a more diagnostic measure to indicate specific hearing difficulties, rather than an overall measure of CI outcome, thus was not included in the current interim COS.

4.1.3. Patient Reported Outcome Measures

Given the discrepancy in domain importance between CI users and CI professionals in this supra-domain, preference was given to CI users based on feedback provided in Phase 2, in which it was suggested that CI users’ everyday life experiences should be prioritised.

Recommended included outcome measures are:

The SSQ or short-form SSQ-12 as an interim PROM because it is the most regularly used, easy to use, trustworthy, and most likely to be used if recommended whilst other PROMs gain acceptance and are used. CI professionals also felt that in the context of remote services, given the potential impacts associated with variability in the home test environment at each test point, there was a necessity to have a reported measure of hearing ability to confirm the behavioural test measure. The SSQ-12 has been recommended in the ANZ adaption of the Adult Standards of Hearing Care; Living Guidelines ANZ adaption [98]. However, it should be noted that it does not address the most important domains, and although it is commonly used, this alone is not an appropriate criterion for its long-term inclusion in future COSs.

Hearing-related quality of life, satisfaction, and wellbeing are domains for consideration. Based on these domains, the CIQOL (Cochlear Implant Quality of Life) [31] would be a suitable PROM, augmented by a satisfaction measure. The CIQOL is a well-validated measures developed with stakeholder input, using modern psychometric Item Response Theory analysis. The CIQoL Profile (35 items) has sub-domains hearing, communication, social relationships, emotional well-being, independence and daily life, device satisfaction and use, cognitive and mental engagement and perception of self and identify. It is not a unidimensional measure (i.e. quality of life) but the authors suggest the broader sub-domains reflect quality of life. Alternatively, the Living with Cochlear Implants (LivCI) [99,100,101] , a recently developed CI-specific, 22-item PROM which includes four sub-domains addressing psychosocial and wellbeing, participation (i.e. HRQoL), aesthetics and visibility (a primary driver of satisfaction), and stigma could be considered. Like the CIQoL, the LivCI has been developed according to COSMIN best practice principles, including extensive stakeholder (e.g. CI professionals and CI users), content evaluation, contemporary Rasch analysis to ensure high-quality items that are independent, alongside Classical Test Theory analyses. Either of these measures could be a potential candidate to replace the SSQ in future. Both measures are recommended for use in the ANZ adaption of the Adult Standards of Hearing Care; Living Guidelines ANZ adaption [98].

In the absence of an appropriate PROM for CI user satisfaction of devices, it was suggested that, as for Service measures, a Likert single-item measure could be used in the interim.

4.1.4. Future Emerging Domains

Other domains that are emerging as important for remote technologies within audiology [21], but not widely considered in the CI field, such as empowerment, listening effort, auditory fatigue, should also be considered for a future COS. There are a number well-developed CI- or hearing-specific PROMs which address such domains. Additionally, an assessment of digital literacy, whilst not an outcome measure per se, prior to CI users using remote technologies should also be considered [102]. It is strongly recommended that ongoing monitoring of the clinical practices and opinions of CI clinicians is carried out to ensure advancement of clinical practice. A part of this process would be to update the interim COS recommended here over time, as well as to guide development of policy and ongoing implementation, training and de-implementation within clinical practice [103].

5. Conclusions

Development of a core outcome set (COS) to assess remote technologies used by CI users is vital given the increase in the use of remote technologies for CI care. It is important that such a COS is relevant across both remote and in-clinic services to enable comparison and seamless integration of the two modes of service. It must also incorporate meaningful, useful outcome measures for CI users, their families and CI clinicians alike using well-designed, trusted measures, that can be incorporated into clinical practice without unnecessarily over-burdening staff, CI users and their families. We present a pragmatic, interim COS for use in hybrid clinical practice, noting that ongoing monitoring of meaningful future outcomes and clinical practices may result in adaptions to the recommended COS in the future.