Psychometric Testing of the CEECCA Questionnaire to Assess Ability to Communicate among Individuals with Aphasia

1. Introduction

Aphasia is a multimodal cognitive disorder caused by acquired brain damage that impacts spoken language, listening comprehension, reading, writing, and other cognitive processes dependent on the activity of the left cerebral hemisphere, which are essential for the proper functioning of language [1,2]. Cerebrovascular accidents (CVAs) are the most common cause of aphasia [3]. Multicentre studies show an incidence of aphasia ranging from 18% to 38% [4,5,6,7].

The use of language is one of the most distinctive attributes of the human species, enabling us to become social beings and to partake in a particular culture [8]. Through language, we interact with those around us, and with ourselves, we learn and organise our thoughts. Language represents a basic feature of our personality. Given the various functions that language has for human beings, its impairment or loss has a significant impact on the quality of life of individuals with aphasia, making it difficult for them to carry out certain activities of daily living and negatively affecting their physical, mental, emotional, familial, and social wellbeing [9,10,11]. The consequences of aphasia can be exacerbated by inappropriate communication strategies in the healthcare setting [12]. It has been shown that when nurses interact with individuals with aphasia, patients are given little opportunity to start or maintain a conversation, and conversations revolve around the professional’s goals rather than the patient’s needs and demands [13]. This asymmetrical communication leads to a loss of autonomy and makes it difficult for the patient with aphasia to participate in their own care, resulting in feelings of frustration, helplessness, fear, anger and/or resignation [14].

The complexity of aphasia requires a multidisciplinary and interdisciplinary approach, in which nurses have a fundamental role to play as care providers. Assessment, the first step in the nursing process, is key to ensuring an accurate diagnosis and a care plan tailored to the needs of the individual with aphasia [15]. We believe that this assessment should also facilitate the search for strategies to achieve the most symmetrical and effective nurse-patient communicative interaction possible. The growing production of nursing studies in the past two decades reflects the interest of researchers in caring for patients with language and communication disorders [16,17]. However, surprisingly few studies have been devoted to the assessment of the communicative abilities of aphasia patients. Most aphasia assessment tests have emerged from disciplines such as neurology, psychology, linguistics, and speech therapy. Exploratory modalities include screening tests designed to determine the presence or absence of aphasia; assessment batteries, commonly extensive tests constructed from multiple subtests usually requiring specific knowledge for the interpretation of results; and tests to assess one specific component of language in particular, such as naming or language comprehension [18,19,20]. The characteristics of these assessment tools make them unsuitable for daily use by nurses [19].

The CEECCA, Cuestionario para la Evaluación Enfermera de las Capacidades Comunicativas en la Afasia (Nursing Assessment of Ability to Communicate among Patients with Aphasia questionnaire), is based on standardised nursing languages (SNLs) and the pictograms of the ARASAAC, Centro Aragonés para la Comunicación Aumentativa y Alternativa (Aragonese Centre for Alternative and Augmentative Communication). The CEECCA is intended to assess the communicative abilities among patients with aphasia based on areas of interest for care. It consists of 43 items assessing 14 specific areas of language, corresponding to five global dimensions likely to be affected in aphasia: “Verbal expression”, “Written expression”, “Expression through pictograms”, “Auditory comprehension”, and “Reading comprehension”. Its design includes the defining characteristics (DCs) of the two communication-related NANDA-I nursing diagnoses: “Impaired verbal communication” (00051) and “Readiness for enhanced communication” (00157), as well as the NOC outcome indicators “Communication” (0902), “Communication: expressive” (0903), “Communication: receptive” (0904), and “Information processing” (0907). In an initial study [21], the design and construction process of the CEECCA and the results of the content validity tests were published. In these tests, the CEECCA obtained high content validity and representativeness scores. This study demonstrated the utility of the NANDA-I and NOC SNLs for the construction of instruments to improve the accuracy of nursing diagnosis and the measurement of outcome indicators in specific care settings. After this preliminary study, it was necessary to determine the remaining psychometric properties regarding the validity and reliability of the CEECCA instrument in a sample of individuals with aphasia.

The research hypothesis of this study is that the CEECCA is a valid and reliable instrument for nurses to assess the ability to communicate among individuals with aphasia, including dimensions of interest for care.

The objectives of this study are to carry out the necessary validity and reliability tests to obtain the psychometric properties of the CEECCA (criterion validity, construct validity, internal consistency, and reliability) and to describe the profile of the patients with aphasia in the validation sample based on their sociodemographic and clinical variables.

2. Materials and Methods

2.1. Data Collection

Once the design, construction, and content validity phases of the CEECCA have been completed, the instrument is tested on a sample of individuals with aphasia to calculate the remaining psychometric properties. The protocol used is based on the proposals put forward by Carvajal et al. [22] and Ramada-Rodilla et al. [23] for validating health measurement instruments already used in other studies on the design and validation of assessment instruments based on SNLs [24,25]. This stage consists of four phases.

(A)

Firstly, the research team selected the nurses and speech therapists who would participate in the fieldwork phase. Three nurses were selected using a convenience sampling method: two from the Primary Care Management Board of the Tenerife Healthcare Area and one from the Management Board of the Nuestra Señora de La Candelaria University Hospital. The instruction phase for the professionals comprised five joint explanatory meetings of approximately 60 minutes each. Their content focused on explaining the objectives and methodology of the study, the instructions for administering the CEECCA and the proxy instruments, as well as other methodological and ethical considerations. Each collaborator was given a field notebook with all the documents required for the administration of the tests and data collection.

(B)

Secondly, participants were selected to form the study sample. Participants were selected using a convenience sampling method in various settings: The University Hospital of the Canary Islands (HUC), primary healthcare facilities in the Tenerife Healthcare Area, as well as private rehabilitation centres and associations. Inclusion criteria were persons of legal age; diagnosed with aphasia as a consequence of acquired brain damage; with Spanish as their mother tongue; who agreed to participate in the study. The exclusion criteria were: patients with a low level of consciousness (in a vegetative state and/or minimally conscious); a personal history of neurological or neurodegenerative disease prior to the brain injury that caused the aphasia; a psychiatric-psychological history of communication disorder prior to the brain damage; with a cognitive level preventing them from taking the test; pre-morbid reading and writing disability; severe visual or hearing impairment that hinders the correct administration of the instrument; behavioural problems that impede communication with the researchers; history of alcoholism and/or other drug abuse.

The necessary sample size was calculated by taking as a reference the sample sizes used to validate other instruments included in a 2017 systematic review aimed at identifying and evaluating the psychometric properties of screening-type tests for diagnosing post-stroke aphasia [26]. The samples of aphasia patients, with whom these tests were validated, had an average of 42 participants (Table 1). Taking this data as a reference and considering the difficulty in recruiting participants with this clinical and psychosocial profile, a sample of around 50 participants was deemed necessary to estimate the correlation coefficients for analysing the convergent criterion validity and reliability coefficients (Cohen’s κ) of the questionnaire through non-random concordance estimates of at least 0.30 while maintaining 95% confidence levels.

(C)

Subsequently, the proxy instruments used to carry out the validity tests of the questionnaire were selected and applied. The three proxy instruments used in the validation phase of the CEECCA questionnaire had previously been used in the design and construction phase of the questionnaire, enabling consistency to be maintained between the two processes: The Boston test for diagnosing aphasia. Spanish adaptation. Second edition [36]; the selected indicators of the four communication-related Nursing Outcomes Classification (NOC) outcomes [37]; the selected defining characteristics (DCs) of the 2012-2014 NANDA-I nursing diagnosis “Impaired verbal communication” [38]. The standardised nursing classifications used were the latest revisions available at the time of administration.

Firstly, a speech therapist administers the activities selected from the Boston test for the diagnosis of aphasia:

• Conversational speech.
• Descriptive speech.
• Visual confrontation naming.
• Writing mechanics.
• Written confrontation naming.
• Auditory discrimination of words.
• Auditory comprehension of commands.
• Reading comprehension. Matching pictures and words.
• Reading comprehension. Reading sentences and paragraphs.

During this first visit, clinical and sociodemographic data are collected and informed consent is obtained.

At an interval of one to three days, a nurse assesses the selected communication-related NOC outcome indicators and determines the presence or absence of the DCs of the NANDA-I diagnosis “Impaired verbal communication”.

(D)

A nurse administers the CEECCA to each subject in the sample. The CEECCA was administered at the primary healthcare facilities in the Tenerife Healthcare Area, at the HUC rehabilitation units, in the rehabilitation departments of the collaborating centres and associations, and in the participants’ own homes. The CEECCA is administered once again by another nurse under the same conditions at an interval of one to seven days. Alternatively, one of the nurses who previously administered the questionnaire repeats the process four weeks later.

2.2. Data Analysis

The results obtained from the administration of the CEECCA questionnaire and the proxy instruments, as well as the data on clinical and sociodemographic variables were entered, as they became available, into an SPSS v.25.0 database for further refinement and processing. The data processing plan involved four phases:

2.2.1. Sample Description

The sample is described by expressing nominal variables as absolute and relative frequencies and by expressing quantitative variables as the median (minimum-maximum).

2.2.2. Validity Tests

• Construct validity

Validity tests were carried out by means of a principal components analysis, following the Kaiser-Meyer-Olkin (KMO) sampling adequacy measure and Bartlett’s test of sphericity, confirming the dimensions that make up the questionnaire, using a varimax rotation, to check that the component items of the questionnaire load towards the areas that theoretically make up its dimensions. The analysis was performed using the scores obtained by each of the subjects in the sample in the first administration of the CEECCA.

• Convergent criterion validity

Concordances between the first administration of the CEECCA questionnaire and the proxy instruments were assessed. Each area of the CEECCA was compared with the selected areas of the Boston test as shown in Table 2.

To obtain the concordance between the two instruments, the results of each area of the CEECCA were used as dichotomous variables (i.e., Functional/Dysfunctional) in accordance with the qualitative rules designed for this purpose. It was agreed to select the 60th and 70th percentiles from the percentile table summarising the results of the Boston test. Two concordance tests are performed with this proxy instrument, with results equal to or above the selected percentiles being considered functional, while results below them are considered dysfunctional. These estimates are made using Cohen’s κ corrected for random chance effects.

Regarding the NANDA-I classification [38], Cohen’s κ concordance degrees are estimated between the results of each area of the CEECCA (in terms of functionality/dysfunctionality) and the presence of DCs of the NANDA-I diagnosis “Impaired verbal communication” relating to that particular area (Table 3).

The degree of concordance between the results of each area in the CEECCA and the results of the evaluation through the four communication-related NOC outcome indicators [37] is also calculated (Table 4). For this proxy instrument, two concordance tests are performed: an initial test, in which scores between 3 and 5 on the Likert scale, both inclusive, and assessing each NOC outcome indicator, are set as the criterion for functionality, and a second test setting scores 4 and 5 as functional. This conversion allowed the NOC indicator scores to be reformulated into dichotomous variables. Cohen’s κ statistic was again used.

Additionally, non-parametric correlations were calculated between the CEECCA total scores and the total scores of the selected subtests in the Boston test, the total number of DCs of the NANDA-I diagnostic label, and the total score obtained in the evaluation of the NOC outcome indicators for the sample. The Spearman-Brown rho statistic was used for this calculation.

2.2.3. Inter-Observer Reliability, Intra-Observer Reliability, and Internal Consistency

The degree of concordance between nurses (with an interval between one and seven days) and of individual nurses (with a four-week interval) was calculated for the results of each area in the CEECCA. These estimates were made using Cohen’s κ concordance statistic corrected for random chance effects. As a supplementary reliability analysis, internal consistency tests were carried out by calculating Cronbach’s alpha and the correlation between each CEECCA item with the other component items of the instrument. This was calculated using the Spearman-Brown rho statistic.

2.2.4. Responsiveness

The responsiveness of the questionnaire is tested on a sample subject who underwent a two-week, 20-hour intensive speech therapy rehabilitation programme. The two-hour sessions were held over a period of five days. The intervention consisted of conversation therapy supplemented with activities of increasing difficulty that focused on the subject’s affected processes. The intervention was conducted by a speech therapist with expertise in this type of intervention. The CEECCA was administered two days before the intervention and again the day after the end of the intervention. In addition, the selected subtests of the Boston test were administered before and after the intervention to check for changes using a benchmark instrument.

3. Results

3.1. Sample Description

The sample consisted of 47 subjects diagnosed with aphasia, 16 females (34%) and 31 males (66%) recruited from 20 May 2019 to 18 February 2020 (9 months and 5 days). Their median age was 65 years (41-94). All participants were recruited on the island of Tenerife, in the Canary Islands, Spain. Thirty-four percent of the subjects were recruited in primary care consultations, 44.7% in specialised care consultations, and 21.3% in other associations or rehabilitation centres. Regarding their level of education, 25.5% could read and write, 34% had completed primary education, 12.8% had completed secondary education, 17% had a technical or vocational training degree, and 10.6% had a university-level education. Chronic health problems were present in 93.6% of the sample. The most prevalent comorbidities were: high blood pressure (72.3%), dyslipidaemia (57.4%), depression (36.2%), urinary incontinence (27.7%), atrial fibrillation (23.4%), obesity (21.3%), constipation (21.3%), epilepsy (19.1%), Type 2 diabetes mellitus (17%), anxiety (12.8%), faecal incontinence (10.6%), dysphagia (10.6%), and insomnia (10.6%). Aetiological factors leading to aphasia included: ischaemic stroke (59.6%), haemorrhagic stroke (17%), neurodegenerative disease (10.6%), traumatic brain injury (TBI) (6.4%), central nervous system infection (2.1%), and brain tumour (2.1%). The types of aphasia in their clinical records included anomic aphasia (25.5%), afasia transcortical mixta (19.1%), global aphasia (17%), primary progressive aphasia (12.8%), motor aphasia (4.3%), transcortical motor aphasia (4.3%), anomic motor aphasia (4.3%), semantic variant primary progressive aphasia (4.3%), transcortical sensory aphasia (2.1%). One participant in the sample (2.1%) could not be assigned any of the established aphasic syndromes. Chronic aphasia with a course of more than 12 months was present in 76.5% of the sample. The majority of the sample was right-handed, at 97.9%, with only one participant being left-handed.

3.2. Administration of the CEECCA

The mean duration of the first administration of the questionnaire was 16 minutes (9-32), the second administration was 15 minutes (5-37), and the third administration was 15 minutes (8-37).

The percentages of dysfunctionality for each area of the questionnaire are shown in Table 5.

3.3. Construct Validity

The results of the Kaiser-Mayer Olkin test (KMO) and Bartlett’s test of sphericity are shown in Table 6. The total variance explained of the CEECCA instrument for five dimensions is shown below (Table 7).

The five theoretical dimensions explain 78.6% of the total variance. Table 8 shows the rotated component matrix describing the grouping of the items in the five dimensions.

Table 9 shows the theoretical location of the component items of the CEECCA dimensions, their location as per the factor analysis, and their final location.

3.4. Convergent Criterion Validity

Table 10 shows the results of the convergent criterion validity tests, comparing each area of the CEECCA with the selected areas of the Boston test [36], and taking the 70th and 60th percentiles as references.

Table 11 shows the results of the convergent criterion validity tests comparing each CEECCA area with the selected DCs of the NANDA-I diagnosis “Impaired verbal communication” [38].

Table 12 shows the results of the convergent criterion validity tests, comparing each area of the CEECCA with the selected indicators of the four NOC outcomes related to communication.

The non-parametric correlations between the CEECCA score and the scores of each of the proxy instruments for the whole sample are shown below (Table 13).

3.5. Reliability through Internal Consistency

The internal consistency value (Cronbach’s alpha) was 0.98. The intensity of the strength of the inter-item correlation is represented using different colours (Figure 1) [39].

3.6. Test-Retest Reliability

3.6.1. Inter-Nurse Reliability in Administering the CEECCA

Table 14 shows the inter-nurse reliability results in terms of functionality and dysfunctionality for each area of the questionnaire in the first two administrations, with a time interval of one to seven days.

3.6.2. Intra-Nurse Reliability when Administering the CEECCA

The reliability results in terms of functionality and dysfunctionality for each area of the CEECCA when the same nurse administers the questionnaire at baseline and at one month are shown below. Table 15 shows the results for nurse (a) and Table 16 shows the results for nurse (b).

3.5. Responsiveness

The CEECCA areas that exhibited the greatest changes after the intervention (i.e., from a dysfunctional to a functional outcome) were: Descriptive speech, Naming objects verbally, Writing name and surname(s), Naming objects in writing, Naming actions in writing, and Auditory comprehension of sentences. The areas that did not change in terms of functionality but obtained better scores after the intervention were: Conversational speech and Naming actions verbally. The areas that remained unchanged after the intervention were: Expressing actions through pictograms, Expressing emotions through pictograms, Auditory comprehension of words, Auditory comprehension of verbal commands, Reading comprehension of words, and Reading comprehension of sentences (Table 17).

No item in the CEECCA areas displayed poorer scores after the intervention.

The Boston test subtests obtained better scores, with only the scores on the subtest “Conversational speech” remaining unchanged (Table 18).

The resulting CEECCA questionnaire is available in as Figure S1 (Supplementary Materials).

4. Discussion

The psychometric tests carried out on the assessment instrument derived from this study have yielded satisfactory results, providing a valid, reliable tool for nurses to assess the main dimensions of language in individuals with aphasia in a simple way and adapted to their daily work. The CEECCA is an instrument whose design [21] and validation processes incorporate aspects of NANDA-I nursing diagnoses and NOC outcome criteria. This allows consistency to be maintained throughout the nursing process as applied in clinical practice [40].

A potential limitation of this study is the sample size used to validate this instrument. In our opinion, the time constraints and operational limitations of the study, together with the difficulty in recruiting participants with this clinical and neuropsychological profile, have prevented a larger sample size from being recruited. Other nursing assessment instruments based on SNLs have been validated using larger sample sizes [24,24,41]. However, sample sizes are notably smaller in several validation studies of screening-type instruments for the diagnosis of aphasia [27,28,29,30,33,34,35]. These studies do not discuss the reasons for using such a limited sample of subjects with aphasia, however, the frequency of this phenomenon suggests that other studies have also encountered difficulties in recruiting this profile of subjects. Similarly, limited samples of patients with aphasia are reported in other studies not devoted to the design and validation of language assessment instruments. For instance, a 2020 systematic review on the use of transcranial direct current stimulation (tDCS) and speech therapy intervention in patients with aphasia illustrates this point. This review includes 35 studies, with a mean sample size of 14 participants and only one study with more than 40 participants. Some studies mention the challenge of obtaining informed consent from individuals with language and communication disorders, resulting in a systematic exclusion of people with aphasia from the samples of many studies [42,43]. A Cochrane review assessing the effectiveness of different strategies in improving the care provided to post-stroke patients and their families [44] reveals that, of the 14 reviewed studies, only one included patients with aphasia, and ten studies considered the presence of aphasia as an exclusion criterion.

The calculation of the construct validity of the questionnaire begins by carrying out the sampling adequacy tests that warrant the performance of a factor analysis. Bartlett’s test of sphericity, with statistical significance being p<0.005, indicates that the variables that make up the test are intercorrelated and, therefore, a factor analysis can be performed. However, the Kaiser-Meyer-Olkin (KMO) statistic provides a result of 0.30, suggesting that the data fit a factor model poorly [45,46], mainly due to the limited sample size. As a result, the calculation of the total variance explained with five dimensions gives a result of 78.6%, a high value that points to the possibility of reducing the dimensions of the questionnaire, as three dimensions explain 72.0% of the total variance. However, it was decided not to reduce the number of dimensions and to explore the statistical locations of the items. The rationale for this decision lies in the interest of maintaining a questionnaire structure and design that would allow the diagnostic labels of dysfunctionality to be established for the dimensions derived from the selection process based on the SNLs and screening instruments for the diagnosis of aphasia. The rotated component matrix distributes the items into their different factors, maintaining a similar and coherent structure to the one proposed in theory. Despite this, most of the items assessing the Auditory comprehension dimension, especially the Auditory comprehension of words area, share a factor with the items assessing the Expression through pictograms dimension. In this regard, the CEECCA uses a multiple-choice auditory word recognition test to assess these areas. The two tasks necessarily involve the same processes; therefore, it is expected that a patient with dysfunctional auditory comprehension of words assessed using the CEECCA will perform more poorly in the areas of Expression through pictograms.

Convergent criterion validity tests using the Boston Diagnostic Aphasia Examination (Second Edition) as a proxy instrument show concordance percentages representing moderate to strong correlations for most of the areas compared using both the 60th and 70th percentile as references [47,48]. The areas in the CEECCA questionnaire with the strongest correlation were: Conversational speech, Descriptive speech, Naming objects verbally, Naming actions in writing, and Auditory comprehension of verbal commands, with total concordance percentages of up to 93.7%. The area with the lowest kappa value was Auditory comprehension of words, with a total percentage of less than 60% and a weak correlation for both the 60th percentile and the 70th percentile. These results may be explained by the different methods used by the two tools for assessing this area. While the CEECCA assesses this area using an auditory discrimination test with five very familiar words with high levels of agreement in terms of naming, the Boston test assesses this area using 36 words from six semantic categories, with different levels of phonemic complexity, lexical frequency, and imaginability. The Boston test will therefore be able to identify problems in the discrimination of less familiar words and will be more sensitive in detecting impairment in the recognition of words belonging to a particular semantic class. However, the comprehensive assessment of the patient’s performance in all language areas of the Boston test makes it an instrument that requires a long administration time (between one and a half to two hours) [32] and specific knowledge on the part of the assessor in order to make a proper evaluation of the patient [49]. In turn, the CEECCA seeks opportunities for communicative interaction in each language area through a simple assessment process that does not require a long administration time. To this end, it was necessary to limit the number of items and prioritise the interest in detecting functionality/dysfunctionality in patients with more severe communication disorders, even knowing the loss of sensitivity that the tool would experience in identifying dysfunctionality in milder or more selective communicative disorders. Another aspect to consider is that the Boston test was administered by speech therapists with experience in the care of individuals with aphasia, while the CEECCA was administered by nurses without specific knowledge in speech rehabilitation. Even so, the percentages of total concordance between the two tests ranged between 93.7% (p<0.001) and 55.3% (p<0.001) for the 70th percentile, and between 91.5% (p<0.001) and 57.4% (p<0.001) for the 60th percentile. These data are supported by the degree of correlation between the Boston test subtest total scores and total performance on the CEECCA questionnaire as measured using the Spearman-Brown correlation coefficient, with a coefficient of 0.96 (p<0.001) indicating a strong positive association [50].

Convergent criterion validity tests that take the presence or absence of the DCs of the NANDA-I diagnosis “Impaired verbal communication” as correlation variables indicate κ values suggesting weak to moderate concordance strengths [47,48]. On this point, it is important to mention that the NANDA-I classification is not a diagnostic tool, and therefore, it may be questionable to perform a criterion validity test using it. However, we believe that it is interesting to consider the possibility that the diagnostic labels proposed by the CEECCA serve as sub-levels of specificity of the diagnostic labelling proposed by the NANDA-I. In this test, it is observed that the DCs that are more specifically related to the area of language assessed have higher concordance strengths. For example, the language area “Naming objects verbally” relates to seven DCs of the NANDA-I diagnosis. In this case, the percentages of total concordance with the DCs “Difficulty forming words” and “Slurred speech” are higher than with the DC “Difficulty expressing thoughts verbally”, which refers to a manifestation not necessarily related to a verbal naming problem. When calculating the strength of the correlation between the CEECCA total scores and the total scores for the DCs of the NANDA-I diagnosis present in the sample, the results indicate a strong negative correlation of -0.85 (p<0.001). This negative correlation is due to the assignment of a value between zero (the poorest possible response) and four (or three) (the best possible CEECCA response), so that a lower CEECCA score for the whole sample is correlated with a higher number of DCs present in the sample.

The κ values obtained in the convergent criterion validity tests using the selected indicators of the four communication-related NOC outcomes indicate moderate to strong correlations [47,48]. Considering as dysfunctional a score of 1 or 2 on the Likert scale of the NOC taxonomy, 82% of moderate to very strong concordances is observed; and taking a score of 1, 2, or 3 as dysfunctional, the percentage of moderate to very strong concordances drops to 60%. The Spearman-Brown correlation coefficient shows a strong positive correlation (0.91) between the CEECCA scores and the NOC indicator assessment scores for the whole sample. A study on the psychometric properties of an instrument (CoNOCidietDiabetes) [25] whose design and validation uses the NOC classification, also obtained better levels of correlation between the total scores of the two instruments (r_s = 0.72; p-value = 0.001) than between the individual correlations for each NOC indicator, where 41% of correlations were rated as weak and only 9.1% as moderate. Other studies on the design and validation of instruments based on the NOC indicators obtain values similar to those obtained in this study when comparing the results for the whole sample using conceptually similar instruments as a reference. For instance, a 2015 study [51] evaluating the psychometric properties of an instrument reported that the Spanish version of a pain level scale based on the NOC outcome “Pain level” showed a strong correlation (r_s = -0.81; p-value<0.001) with the Numerical Pain Rating Scale.

In general, convergent criterion validity tests for the CEECCA questionnaire appear to show adequate levels, even higher than those obtained in other design and validation studies based on SNLs with larger sample sizes [25,41,51].

The internal consistency of the questionnaire was high, with a Cronbach’s alpha value of 0.98 and a predominance of moderate and strong inter-item correlations. Although such a high value may suggest item redundancy, Cronbach’s alpha value does not increase when an item is removed from the questionnaire (Table 19). This, together with our interest in maintaining a structure that would allow diagnostic labels of dysfunctionality to be established for each area of the CEECCA, meant that a reduction in the number of items was not considered.

Inter-nurse reliability when administering the CEECCA (i.e., when two different nurses administer the instrument in an interval of one to seven days) shows concordance percentages above 90%, with κ values above 0.75 (p<0.001).

Intra-nurse reliability, both when it is the same nurse in the first and third administration (nurse a) and in the second and third administration (nurse b), shows concordance percentages above 80% in twelve of the fourteen areas of the questionnaire. The areas with the lowest levels of concordance were the areas corresponding to comprehension-related dimensions, especially the Auditory comprehension of sentences and verbal commands and Expression through pictograms dimensions. Goodglass and Kaplan [36] point out that test-retest reliability should be interpreted with caution due to the high fluctuations in the performance of patients with aphasia; however, they also note that, when aphasia becomes chronic, variability in language performance is markedly reduced. On the other hand, a time interval of four weeks does not seem to be long enough to explain this change as an effect of the progress of the disorder itself, or to explain a significant change because of rehabilitation if the patient was receiving it. In the reviewed literature, there is no clear consensus on the most appropriate time interval for conducting an intra-rater reliability test on subjects with aphasia. An interval of 20 to 40 days has been used to calculate intra-rater reliability in other similar studies with subject samples without aphasia [52,53,54]. For the Community Integration Questionnaire Adjusted for People with Aphasia only inter-rater reliability testing was performed, not intra-rater reliability testing [55]. None of these test-retest reliability calculations were performed for other instruments such as the Frenchay Aphasia Screening Test [27] or the Ullevaal Aphasia Screening (UAS) test [29].

The fact that a high percentage of the sample (66.2%) was receiving speech therapy rehabilitation at the time of assessment could be considered as a change variable for the results at one month; however, in more than 76% of the sample, aphasia had been present for more than 15 months. This reduces the likelihood of relevant changes caused by rehabilitation in a 4-week interval. In addition, none of the rehabilitation interventions received during this stage underwent changes in their characteristics or intensity.

To test the responsiveness of the CEECCA, a specific intervention is introduced that modifies the intensity and characteristics of the speech therapy rehabilitation. The intervention consisted of an intensive therapy based on conversation therapy, supplemented with activities of increasing difficulty focusing on the affected processes, lasting 20 hours and spread over ten sessions. Although the available evidence is not yet sufficient to determine, categorically, at what time intervals and intensity levels positive results occur, some authors suggest that a minimum of two hours per day, for a period of two to three weeks, can be considered intensive treatment [56]. The intensity of rehabilitation treatment is considered a relevant variable for its success [20]. A number of systematic reviews assessing the effects of speech and communication therapy in patients with post-stroke aphasia [57,58] highlight the positive relationship between high-intensity rehabilitation treatments and improved outcomes in functional communication and writing. In recent years, a growing number of studies point in the same direction, concluding that high-intensity rehabilitative interventions improve learning and brain plasticity and strengthens synaptic contacts between neurons [59,60,61], even among patients with chronic aphasia [62,63]. The subject assessed in this test was diagnosed with motor aphasia with a course of more than four years. According to the pre-intervention assessment, the areas relating to the verbal and written expression dimensions exhibited the greatest degree of dysfunctionality. After the intervention, these same areas showed the greatest improvements when compared to their baseline levels, according to both the CEECCA and the Boston test.

When analysing the psychometric properties of the CEECCA, the data are comparable to those obtained in other validation studies of SNL-based instruments (Table 20).

Using the GRAQoL Index, which assesses the psychometric properties of health measurement instruments through the fulfilment of set criteria, the CEECCA questionnaire obtains a final score of 75%, with an A grade of recommendation and above average when compared to other health instruments [66].

5. Conclusions

The results obtained suggest that the CEECCA is a valid, reliable instrument for the nursing assessment of the ability of individuals with aphasia to communicate, including dimensions of interest for their care. The CEECCA can be administered at any stage of aphasia and in any healthcare setting. This instrument favours nurse-patient communication by indicating which dimensions and areas of language are functional in order to maintain a communicative exchange.