Polish Adaptation and Psychometric Validation of the PREM-C9 Questionnaire Dedicated for Patients with Chronic Obstructive Pulmonary Disease

1. Introduction

COPD is a common, preventable, and treatable disease. About 384 million people worldwide are affected by COPD. According to the latest GOLD report, COPD is “a heterogeneous lung condition characterized by chronic respiratory symptoms (dyspnea, cough, expectoration, exacerbations) due to abnormalities of the airways (bronchitis, bronchiolitis) and/or alveoli (emphysema) that cause persistent, often progressive, airflow obstruction” [1]. Causes and risk factors of COPD have been recently described with the GETomic acronym, which stands for dynamic, cumulative and repeated gene (G) - environment (E) interactions over the lifetime (T) that damage lungs or change their normal ageing processes [2]. The most important environmental risk factor is cigarette smoking, but this group also includes occupational exposure [3] and air pollution [4].

Commonly, the assessment of clinical effectiveness of interventions uses well-defined endpoints. According to the guidelines of the Polish Agency for Health Technology Assessment, these are endpoints related to mortality, morbidity (endpoints related to the course/exacerbation of the disease), health-related quality of life (HRQoL), unwanted events and adverse effects (divided into severe and other) [5]. However, the same guideline underlines that endpoints relevant from the patient's perspective should also be considered. These endpoints include Patient-Reported Outcome Measures (PROMs) and Patient-Reported Experience Measures (PREMs). The patient's perspective is paramount. Principle 5 of the COPD Patients Charter refers to the patient's right to appropriate specialist care whenever the patient needs it [6]. Improving patient care requires using appropriate tools to assess care for patients with COPD.

According to the Food and Drug Administration (FDA), PROMs can be defined as "any reports of the status of a patient's health condition that comes directly from the patient (without interpretation of the patient's response by a clinician or anyone else)"[7]. Such information is collected through standardized questionnaires. PROMs were originally developed and implemented for use in clinical trials as a way to include the patient's voice in the evaluation of clinical interventions [8]. However, their use has gradually become more common, and today, they are considered an integral part of assessing clinical effectiveness and patient-centred care [9].

PREMs allow to determine what is the patient's perception of their experience with received care. Typically, they are structured questionnaires, but unlike PROMs, they do not assess care outcomes but determine how patients perceive their experience of using care. They can be used as indicators of quality of care (determined from the patients’ perspective) at any level of healthcare system [10] and used by entitled agencies to assess medical technologies in the reimbursement process [11].

Like PROMs, PREMs also deliver information on the quality of care during the intervention, which allows for immediate response and adjustments. Several publications have shown that taking into account the experience of patients is a key factor in strengthening health systems. This data may be used as a basis to identify quality, performance or security issues [12,13].

The first PREM-type questionnaire that has been dedicated to asses experience of care in COPD is the PREM-C9 [14]. It is a simple questionnaire validated for use in the population of patients with mild to severe COPD. However, this tool has not been validated into Polish conditions yet. Therefore, the study aims are:

-

To create a Polish adaptation of the PREM-C9.

-

To determine the psychometric characteristics of the PREM-C9 questionnaire used in Polish conditions.

-

To provide knowledge regarding the characteristics and the role of the PREM-C9 in non-English speaking countries, allowing for cross-cultural comparisons and determining further possibilities of using PREM-C9 and generalizing the results obtained to other populations.

We believe that the Polish validation of the PREM-C9 test will stimulate Polish research on the role of the patient's perspective in the treatment of COPD.

Determining the usefulness of this questionnaire requires an examination of its validity and reliability. Thus, these parameters were assessed in a study on the Polish version of the PREM-C9. Questionnaire validity is understood as the level of agreement with which the assessed questionnaire measures what it was designed for. Reliability is defined as a questionnaire's ability to reflect the true value of the characteristic that has been evaluated [15].

2. Materials and Methods

2.1. The adaptation procedure

The PREM-C9 questionnaire was developed by Hodson et al. [14]. The procedure involved the use of both forward and back translation methods in accordance with ISPOR guidelines [16] and Breslin [17].

• Forward translation of the PREM-C9 into Polish by a professional bilingual translator with Polish as a native language.
• Reviewing the translated version of the PREM-C9 by the team, including pneumonologists and the research team members. This stage did not provide any significant changes in the Polish version of the questionnaire.
• Back translation made a professional bilingual translator with English as a native language.
• Comparison of the back-translated version with the original text. This stage did not provide any significant changes; only minor revisions were introduced.
• Evaluation of the comprehensibility of the translated version in a group of 20 healthy volunteers who assessed whether each of the questions was understandable to them.
• The first use of the PREM-C9 questionnaire in the group of 42 COPD patients was to validate the tool and test its psychometric features. Material was collected from patients by a trained interviewer, or patients answered by themselves by filling out a paper version of the questionnaires. The questionnaire was made available both in paper form and digital version. The use of the electronic version of the questionnaire was caused by the COVID-19 pandemic and resulted from the need to limit interpersonal contacts.

The result of conducting the adaptation procedure was the preparation of the Polish version of the PREM-C9 questionnaire, which was used for further testing of the adapted version. The adaptation procedure is presented in Figure 1, and the translated questionnaire is in Figure 2.

2.1. Participants

The study was approved by the Bioethics Committee of the Medical University of Gdańsk, and consent was obtained from the first author of the validated questionnaire for its use. In addition, permission to use the other questionnaires in the study was obtained as well.

A total of 42 Polish-speaking participants were tested. The inclusion criteria were:

-

a patient diagnosed with COPD according to GOLD criteria from 2022;

-

a patient expressing informed consent to participate in the study;

-

a patient not hospitalized at the time of the study due to COPD exacerbation or other reasons;

-

a patient who is able to answer questions.

Most of the patients participating in the study (n=31; 73.81%) confirmed that dyspnea makes them walk slower than their peers or makes them stop to catch a breath when they walk at their own pace. According to mMRC (Modified British Medical Research Council) [18] this is equivalent to at least grade 2 dyspnoea. The basic characteristics of the study group with their socio-demographic characteristics are presented below - (Table 1.)

2.2. Measures

The questionnaire

PREM-C9 (hereinafter also abbreviated as PC-9) is a simple, 9-item tool of relational PREMs type that has been divided into three sections:

-

my everyday life with COPD (4 questions),

-

usual care in COPD (3 questions),

-

COPD exacerbation/Flare up (2 questions).

The patient evaluates the statements using a graphical Likert scale with the two extreme labels marked - 0 (the lowest score), which describes the best situation and 5 (the highest score) dedicated to the worst situation. In the Polish adaptation and validation of the PREM-C9 questionnaire, we used the same questionnaires as in the original validation process (CATTM and HADS) with one additional general PROM – EuroQol EQ-5D 5-level version (EQ-5D-5L). They are briefly described below.

COPD Assessment Test (CAT^TM) [19] (Polish version) is a disease-specific PROM type questionnaire dedicated to people with COPD that assesses how the disease impacts a person's life and whether it changes over time. It consists of 8 items, and the answering scale is from 0 to 5, where 5 is the worst rating. CAT considers presence and severity of cough, phlegm, sense of tightness in the chest, breathlessness, limitations in performing everyday activities, self-confidence when leaving home, quality of sleep, and life energy level. Five points mean the upper limit of normal in healthy non-smokers, <10 points means that COPD has a minor influence on the patient's life, 10-20 points average impact, >20 severe impact, >30 very severe impact of COPD on the patient's life - the disease makes it impossible to perform any everyday activities, and the patient never has good days. It is worth noting that the construction and graphic layout of PREM-C9 and CATTM are very similar.

Hospital Anxiety and Depression Scale modified version (HADS-m) [20] (Polish version) is a PROM type questionnaire measuring anxiety and depression in patients who are somatically ill. It consists of three subscales - depression (HADS-D), anxiety (HADS-A) and irritability (two questions from the last subscale were not used in the original version and subsequently neither in this study). The whole questionnaire consists of 16 questions, which are evaluated on a 0-3 scale. Obtained results can be interpreted as follows: ≤7 = Normal, from 8 to 10 Borderline abnormal (borderline case), from 11 Abnormal (case). Since in the study, only two subscales (A and D) were used, hence hereafter, the questionnaire will be referred to as HADS (as the modification was the addition of the third unused subscale).

EuroQol EQ-5D 5-level version (EQ-5D-5L) [21] (Polish version) is a general PROM-type questionnaire consisting of 5 questions (considering: mobility, self-care, usual activities, pain and discomfort, anxiety and depression) that can be evaluated in 5 levels: no problems (1 point), slight problems, moderate problems, severe problems and extreme problems (5 points) and visual analogue scale (EQ-VAS) to describe the general health status on the day of attending the study. EQ-5D-5L is available in more than 150 languages and is one of the most frequently used generic questionnaire assessing health-related quality of life (HRQoL) [22].

2.3. Statistical methods and calculations

All calculations were performed using the Microsoft Excel spreadsheet questionnaire, the StatSoft Inc. statistical package Statistica and the SPSS program. In the statistical description of quantitative data, classical measures of position such as arithmetic mean, median, and standard deviation were used as measures of variability. To evaluate the psychometric properties of the PREM-C9 questionnaire, Cronbach's alpha, Bartlett's test of sphericity, Kaiser-Meyer-Olkin coefficient and Spearman correlation coefficient were used. In all statistical tests, p<0.05 was taken as the level of statistical significance of differences.

The validity of the questionnaire was assessed using PCA (Principal Component Analysis) for the extracted principal components. The validity of the factor analysis was demonstrated by the Bartlett's sphericity test and the Kaiser-Meyer-Olkin coefficient (KMO), with p<0.05 and KMO> 0.6 as the cut-off level, respectively. Factor analysis was performed using the Oblimin and Varimax rotation method, assuming 0.4 as the threshold value. The reliability of the questionnaire was assessed using the Cronbach's alpha index, assuming 0.70 as the cut-off value.

3. Results

The Polish language version of the questionnaire was named identically to the original version - PREM-C9. Like the original version, the Polish questionnaire consists of 9 questions divided into three sections. Developing a Polish version was proceeded in accordance with the principle of facade equivalence (test graphics, instructions).

This questionnaire and the Polish versions of the CAT, HADS and EQ-5D-5L questionnaires were given to the patients participating in the study for completion. The results obtained in the study group using the individual tests are presented in Table 2.

In the overall assessment of current health status using the EQ-VAS, a mean score of 60.56% of the ideal value was obtained (Me=60; SD=22.32). The mean scores obtained for the whole group on the CAT scale should be interpreted as the average impact of COPD on patients' lives, which was higher than in the well-controlled patients (CAT>10 points). The results of the HADS scale indicated that, overall, patients did not seem to suffer from depression or anxiety (mean score for the whole group <8 points). However, four respondents had a score greater than or equal to 8 for the HADS-D scale and six patients on the HADS-D scale, which should be interpreted as borderline abnormal or even abnormal values.

• Assessment of psychometric characteristics
• Validity of the PREM-C9 questionnaire

In order to examine the theoretical validity of the test, each part of the PREM-C9 was subjected to factor analysis. The validity of the factor analysis was proven by Bartlett's sphericity test, with a significant result (p=0.000), which proved the presence of a correlation between the components. Also, the Kaiser-Meyer-Olkin (KMO) test results of >0.5 proved that there were relationships between the components and that they were selected appropriately, so that factor analysis was justified (Table 3).

The PCA (Principal Component Analysis) factor extraction method using the Kaiser normalisation criterion, with both Varimax and Oblimin rotation, identified three components (the same as in the English-language version). In all questions, a correlation value of at least 0.6 was obtained between the individual questions and the questionnaire as a whole, which means that each question in the Polish-language version of the PREM-C9 is valid (contributes relevant content) and should not be removed. The lowest correlation value was obtained for question 4 (I am happy to talk about the future), but this was not below the accepted criterion and was noticeably higher when Oblimin was used as the rotation method.

The next stage of the validity analysis was the convergent relevance analysis, for which Spearman's correlation was used between the PREM-C9 questionnaire assessed and the other questionnaires, i.e. HADS, CAT, EQ-5D-5L and EQ-VAS (Table 4). In each case, a statistically significant (p<0.05) low or moderate correlation (a correlation of at least pronounced strength) was obtained.

The reliability of PREM-C9 questionnaire

The assessment of the reliability of the Polish version of the PREM-C9 questionnaire showed that the Cronbach's Alpha index was 0.743, which confirms that the assessed questionnaire is reliable (value >0.70).

In order to analyse the correlation of individual items with the remaining considered as "a total", the reliability assessment was repeated, but after removing individual questions from the questionnaire. It was shown that the elimination of individual statements did not significantly affect the Cronbach's alpha coefficient value for the rest of the questionnaire. The values obtained ranged from 0.690 to 0.767. The greatest reduction in the index was found when question 6 (I have enough information about my condition) was removed, which should be understood as the weakest correlation of this question with the others, but the value was still close to 0.70. The constructed correlation matrix of each question (Table 5) also showed that question 6, after removal of which the lowest value of Cronbach's Alpha was obtained, did not correlate significantly with the other questions. Question 2 was also the question that did not correlate with the others, but after removing it, Cronbach's Alpha only decreased by 0.032. However, as the aim of the study was not to modify the original tool, questions 6 and 2 were left in their Polish version, as in the original structure of the PREM-C9.

Also, a reliability analysis using Klein's criterion that individual variables should correlate with the total score for the whole scale at a level of at least 0.4 confirmed the reliability of the whole questionnaire, with all nine questions retained (Table 6). A comparison of the Spearman rank order correlation results between PREM-C9, CAT and HADS with the results of the original version (Hodson 2019) and the first use in the original language version (Jones 2020) is presented in Table 7.

4. Discussion

The Polish version of the PREM-C9 questionnaire met all validation criteria, i.e. face equivalence (test graphics, instructions), translation equivalence (question content), psychometric equivalence (similar correlation with questionnaires also used in psychometric analysis of the English language version of PREM-C9)), functional equivalence (suitability for the same purposes), reconstruction equivalence (checking reliability and validity). Validation of the PREM-C9 into Polish will allow the use of the questionnaire in the assessment of care for patients with COPD in Poland. The use of PREM indicators is still not a common practice, which previous studies have proven [23]. and which limits the possibility of comparing the results obtained in terms of accuracy and reliability of the Polish version of the PREM-C9 questionnaire with other works apart from the original assessment in the English language version. There are known studies using the PREM-C9 questionnaire in its original version. For example, a study by Jones et al. [24] measured patients' experiences of living with COPD and the medical care they received, and the results were compared with other scales - some of which were also used in the validation study of the Polish-language version (CAT and HADS). The correlation rho of the PREM-C9 (for all domains combined) with the CAT questionnaire score was 0.27 (thus less than in the present study), but it was a statistically significant relationship (p=0.03). However, only a result measuring towards significance was obtained for the correlation with the HADS-D subscale (p=0.09). The correlation level obtained in the present study between the PREM-C9 test and the CAT was 0.44, which is similar to the values obtained in the original validation of the tool (0.42). Similar results were also obtained for the HADS-A, slightly lower for the HADS-D [14]. In addition to the English language version, the PREM-C9 was translated into Catalan and Spanish [25]. Ten patients participated in this study, however, this study did not determine the questionnaire's psychometric values but the authors only translated it. Chaplin et al. define a meaningful change in PREM-9 following pulmonary rehabilitation. The change in PREM-9 for responders (defined as HADS anxiety MID ≥ -1.5) was -5.26 (SD 8.33). Sensitivity and specificity analysis using ROC using HADS anxiety anchor yielded a change of -7.5 units. The minimum important difference for the PREM-9 is calculated between -3.67 and -7.5 units [26].

The PCA analysis using the Kaiser normalisation criterion with Varimax and Oblimin rotation obtained the lowest correlation value for question 4 (I am happy to talk about the future). It should be acknowledged that the nature of the question itself may have influenced the result, as it is of a general type, going beyond the context of the disease itself, hence it can be assumed that the results obtained for this question differed slightly from the others.

An assessment of the reliability of the Polish version of the PREM-C9 questionnaire confirmed that it is a reliable tool and, therefore, correctly (accurately) reflects the actual condition. The Cronbach's alpha index was 0.743. According to various sources, even 0.6 can be considered a satisfactory level, although more often, it is 0.7 [27,28]. Slightly higher values were obtained in the validation of PREM-type tools dedicated to other conditions, e.g. rheumatic conditions [29], oncological conditions [30] or hypertension [31]. However, this was not a study conducted in a Polish setting; such studies, apart from this one, have not yet been produced. Also, a literature review performed in Germany in 2022 showed that none of the PREM-type questionnaires analysed were fully evaluated in German conditions [32].

5. Implications and limitations of the study

The benefits of using patient perspective in shaping effective health care are well accepted. For example, the addition of these data in the care of patients with metastatic cancer to those collected in a standard way resulted in increased survival compared to traditionally managed care [33], and in a group of patients with arthritis, it improved self-perceived health [34]. Using PROM and PREM type data also reduces the utilisation of healthcare system resources by improving symptom control, increasing patient satisfaction and ultimately - improving HRQOL [35].

Much attention has been paid over the years to the reliability of questionnaires of the PROM and PREM type while at the same time pointing to the need to develop new validated research tools [36]. This study has just resulted in the development, in accordance with current principles, of a PREM-type questionnaire, the psychometric properties of which have been meticulously examined and confirmed. This study may inspire and encourage other researchers to develop and use this type of questionnaire in patient care. What may be of note in the study is the small study group (42 patients). However, it should be clear that there are no specific guidelines on the minimum number of respondents for this type of study [37]. In the publication by Tsang et al. presenting Guidelines for developing, translating, and validating questionnaires, guidance can be found that it should be between 30 and 50 people, similar guidance is found in a broader study by Aithal [38].

6. Conclusions

The developed Polish version of the PREM-C9 questionnaire is a reliable and valid tool that assesses Polish patients' experiences of their disease (COPD) and the care they receive.

The questionnaire can be used to conduct follow-up among Polish patients and for comparative studies in non-English speaking countries, allowing for cross-cultural comparisons and determining further possibilities of using PREM-C9 and generalising the results to other populations.