Initial Validation and Psychometric Properties of the Croatian Version of the Pieper-Zulkowski Pressure Ulcer Knowledge Test

1. Introduction

The treatment of pressure ulcers/injuries (PU/PI) remains a challenge worldwide [1] and is considered an indicator of the quality of care [2]. Despite the development of medicine, PU/PI still represent a ubiquitous health problem with multiple consequences on the health and treatment outcomes of patients, but also a large financial burden for health systems [3]. PU/PI developed in hospitals are financially debilitating [4], and are associated with prolonged hospitalization [5].

The clinical process of deciding how to treat PU/PI is delicate [6] and is dependent on the professional knowledge of the nurse as well as the current available scientific evidence. The process of optimal decision-making, in this process, includes applied practice that is based on evidence with the use of health information technology [7]. The recommendations of the guidelines emphasize the need for regular assessment of the knowledge and attitudes of healthcare professionals ​​to support clinical recommendations as well as to detect challenges [8] with the need to implement organizational and ergonomic strategies [9]. Forms of formal and lifelong training programs for nurses on PU/PI prevention represent the key links of effective prevention . Interventions need to consist of evidence-based "portfolios" tailored to the needs of patients [12]. In order to prevent PU/PI, nurses must be competent and well educated [13], ensuring routine implementation of key strategies to improve mobility and reduce pressure [8].

The results of previous studies indicate unsatisfactory levels of knowledge of nurses [14,15] and the need for regular knowledge checks for the purpose of defining educational needs and educational priorities[16].

The used Pieper-Zulkowski Test of knowledge about pressure ulcers (PZ-PUKT) was developed by the authors Barbara Pieper and Karen Zulkowski in 2014 and updated in 2021. This comprehensive tool assesses PU/PI knowledge in three domains: Prevention, Staging, and Wound Description [17].

This quantitative, psychometric, cross-sectional study was designed with the aim of translating and culturally adapting the PZ-PUKT and evaluating the psychometric properties of the Croatian version.

2. Materials and Methods

The aim of this cross-sectional psychometric study; performed in a state hospital in Šibenik, Croatia; was to conduct cultural adaptation, investigate internal consistency, and assess the psychometric properties of the PZ-PUKT Version 2 among clinical nurses in Croatia. The STROBE guidelines for reporting observational studies were followed.

2.1. Instrument

In accordance with ethical guidelines, permission to use the instrument was obtained from the questionnaire designer via electronic correspondence. The PZ-PUKT version 2 (2021) was used in this study. The instrument is an improved version of the original Pieper Pressure Ulcer Knowledge Test (PPUKT), one of the most commonly used knowledge assessment instruments [18] and recommended by Clinical Practice Guidelines [19].

It has been used in numerous studies (including cross-sectional, experimental, interventional) and has been translated and validated in several languages ​​including Chinese, Polish, Portuguese, Persian, Turkish [18,20,21,22,23], ensuring its applicability in different linguistic and cultural contexts. The instrument consists of 2 parts. The first part consists of 11 sociodemographic and educational data on the acquisition of knowledge about PU/PU. The second part of the questionnaire contains 72 items.

The claims are divided into 3 subscales (Prevention, Wound Description and Staging of PU/PI). The Prevention subscale contains 31 items, the Wound Description subscale contains 20 items, while the Staging subscale contains 21 items. There are a total of 35 correct answers in the questionnaire, while 37 incorrect ones. Answers are possible according to the items "True", "False" and " Do not know". The test is scored by adding up the correct answers, and the range of correct answers is possible from 0-72 points. Generally accepted scoring ranges for the original PZ-PUKT define less than 70% as unsatisfactory, 70% to 79.9% as satisfactory, 80% to 89.9% as good, and 90% and above as very good knowledge of pressure injury prevention [24].

2.2. Translation Procedure

The cultural adaptation of the psychometric instrument included procedures that enabled the adaptation of semantic, idiomatic, experiential and conceptual components. The process of translation and adaptation of the PZ-PUKT questionnaire was conducted in accordance with the guidelines of Beaton and associates [25] was conducted in two phases.

The first phase involved forward translation, meaning the translation of the questionnaire from English to Croatian, was performed by two translators- native Croatian speakers with excellent knowledge of English and an understanding of the goals and concepts of the study. This was followed by back translation, during which two independent translators, who did not have access to the original questionnaire, translated the Croatian version back into English. After that, an expert committee was assembled with the aim of developing the pre-final version of the questionnaire, which was then tested.

In the second phase of the study, a pilot test was conducted on a sample of 30 nurses. The aim of the procedure was to assess general understanding and acceptance without the burden of testing construct validity. The participants in the pilot study were nurses from a clinical setting who work with such patients on a daily basis.

After the pilot test, a final revision was carried out, where a final report with all versions was submitted to the authors of the questionnaire for review. This was followed by a test-retest procedure to assess the reliability of the instrument.

2.3. Assessment of Translation Equivalence

The method for assessing content validity developed by experts from The American Educational Research Association was used [26]. The study involved 12 experts who evaluated each item of the questionnaire using a Likert scale from 1 to 4 (where 1 indicated low relevance and 4 indicated high relevance). The experts provided feedback on the quality, clarity of each item, and its importance for measuring the targeted construct.

The content validity of the PZ-PUKT was assessed using the Item-level Content Validity Index (I-CVI) and Scale Content Validity Index (S-CVI). Items with an I-CVI ≥ 0.78 were considered acceptable [27].

2.4. Data Collection

Data was collected in printed form between March and May 2024. Each form contained information about the purpose and objectives of the study, as well as informed consent. The researcher supervised the completion of the questionnaire to ensure that the participants filled it out independently, without using any tools or external sources. The questionnaire was coded by the participants with a desired code, and responses were linked in the retest through this code. On average, completing the questionnaire took 20 minutes. No compensation was provided for participation in the study.

2.5. Sample

Determining the appropriateness of the sample size according to Bonett's formula for sample size calculation [28] in testing and evaluating the α coefficient for a 72-item scale, the acceptable value for Cronbach's α was 0.80. The confidence interval and test power were set at 95%. Based on these calculations, the minimum number of participants required for the study was 210.

A total of 300 questionnaires were distributed, and after excluding participants due to incomplete questionnaires or lack of response to retesting, 268 participants remained for analysis, representing a response rate of 89.33%. Participants from the pilot testing were not included in the total sample. The inclusion criteria are nurses employed in hospital departments with at least 1 year of experience, nurses of all education levels, and those who agree to participate in the study. To reduce selection bias, we applied strict inclusion criteria, and we also used random selection of participants in the groups to minimize bias.

2.6. Data Analysis

Descriptive statistics were calculated to summarize sample demographics and professional characteristics. Continuous variables were presented as means with standard deviations (SD) and medians with ranges, while categorical variables were expressed as frequencies and percentages. The content validity of the PZ-PUKT was assessed using the I-CVI and S-CVI.

The internal consistency of the PZ-PUKT and its subscales was evaluated using the Kuder-Richardson Formula 20 (KR-20). Additionally, split-half reliability was assessed using Guttman’s Lambda and the Spearman-Brown coefficient. Test-retest reliability was evaluated using the Intraclass Correlation Coefficient (ICC) with a two-way mixed-effects model to assess the consistency of test scores over time.

Item-level difficulty (DFI) and discrimination (DSI) indexes were calculated to assess the quality of test items. Statistical significance was set at p< 0.05, and confidence intervals were reported for key reliability measures.

All collected data were transferred to Microsoft Office Excel, where they were encoded for further processing, and statistical analyses were conducted using R software (version 4.3.1).

2.7. Research Ethics

Conducting the research was in accordance with the ethical standards of the Declaration of Helsinki (version 2013). The participants were guaranteed anonymity and confidentiality, and to ensure the questionnaire complied with ethical principles, they were informed at the beginning of the study about the purpose and goals of the research, how their data would be used, that their participation involved no risks, before giving their informed consent to participate. Participants could withdraw from the study at any time. The research was conducted in the paper version of the questionnaire. Permission to conduct the research was given by the Institution's Ethics Committee prior to conducting the research (Class: 007-10/24-01/3, File Number: 2182-1-50-01-01-24-1).

3. Results

3.1. Basic Descriptive Sample Parameters

Table 1 summarizes the demographic and professional characteristics of the sample

(N=268), consisting primarily of female participants (97.4%). The average age was 38.4 years, with Master-level nurses being the oldest (42.8 years) and Bachelor-level nurses the youngest (35.1 years). Most participants worked in the department of surgery (28.7%), internal medicine (21.6%), or neurology (20.9%). The average professional experience was 15.5 years, with technicians and Master-level nurses having the longest experience.

3.2. Item-Level Content Validity Index

Table 2 presents the I-CVI for each item in the test, categorized into three groups based on their I-CVI values. The assessment was conducted by 12 expert raters, with most items (59 out of 72) achieving perfect content validity (I-CVI = 1.00). Fourteen items had I-CVI values between 0.92 and 1.00, while one item (item 24) had a slightly lower I-CVI (0.83 ≤ I-CVI < 0.92). The overall S-CVI was 0.981.

3.3. Internal Consistency

Table 3 presents the internal consistency of the PZ-PUKT and its subscales using the KR-20. The overall scale reliability is 0.79. The Prevention subscale has a reliability of 0.68. Both the Wound Description and Staging subscales have a reliability of 0.57.

3.4. Split-Half Reliability

Table 4 shows the maximum split-half reliability is 0.87, Guttman Lambda 6 is 0.89, the average split-half reliability and Guttman Lambda 3 are both 0.80, Guttman Lambda 2 is 0.81, and the minimum split-half reliability is 0.60.

3.5. Test-Retest Reliability by Subscale and Overall Results

Table 5 presents descriptive statistics and the test-retest reliability with its confidence interval for the PZ-PUKT subscales and overall score. The overall test-retest reliability is ICC = 0.91. The Prevention subscale has a reliability of ICC = 0.88, followed by the Staging subscale at ICC = 0.86, and the Wound Description subscale at ICC = 0.84. The percentage of correct answers remained stable.

3.6. Categorized Difficulty Index

Table 6 categorizes the DFI of test items into easy, moderate, and difficult based on the proportion of correct responses.

The category with a DFI index > 0.70 includes 33 items, the category with a DFI index 0.30 ≤ DFI ≤ 0.70 includes 25 items, and the group with a DFI index < 0.30 includes 14 items.

3.7. Categorized Discrimination Index

Table 7 categorizes the DSI of items into 4 categories good, moderate, marginal, and poor based on their ability to differentiate between high- and low-performing respondents. The category with a DSI ≥ 0.30 includes 28 items, the category with a DSI between 0.20 and 0.30 contains 12 items, the category with a DSI between 0.10 and 0.20 includes 17 items, and the category with a DSI < 0.10 comprises 15 items.

4. Discussion

The methodological process adopted in the cultural adaptation of the Croatian version of PZ-PUKT, described in this study, was conducted in accordance with the scientific literature [25].

The content validity of the PZ-PUKT was assessed using the Item-level Content Validity Index (I-CVI) and Scale Content Validity Index (S-CVI). For this purpose, we engaged twelve experts from the field who assessed the appropriateness and relevance of each questionnaire formulation. The I-CVI was between 0.83 and 1.00. We calculated the S-CVI based on the average I-CVI calculated for each item of the instrument. The result of 0.981 indicated excellent content validity for the entire instrument [29]. Results from other validation studies indicate that the total S-CVI of the Turkish version was 0.960 (with an item range of 0.778 to 1), the Persian study indicates a total CVI index of 0.94, and the Chinese version describes that all items were corrected until they reached the maximum score [18,22,23].

This high S-CVI supports the clarity and objectivity of the statements of the instrument's items in order to measure the construct. Analyzing the results of previous psychometric studies in an international context, this instrument achieves the highest index values ​​of the overall scale, which proves that experts assess that the items are relevant for measuring the target construct.

The internal consistency of the instrument and subscales was assessed using the Kuder-Richardson formula 20 (KR-20) for scales with dichotomous variables. The overall reliability of the instrument of 0.79 indicates acceptable internal consistency [30].

The Prevention subscale has a reliability of 0.68, Wound Description and Staging each have 0.57. The values ​​of the Prevention subscale are at an acceptable level, while the remaining two show moderate reliability and possible areas for improvement.

Comparing the results of other studies, it is evident that in the Australian version of the KR-20 for the entire instrument was 0.86, while for the Prevention subscale 0.67, Wound Description 0.76 and Staging 0.65 [24], while other studies used Cronbach's alpha to assess reliability. The Brazilian version reports the lowest reliability coefficients of the subscales with 0.42, 0.37 and 0.42 with a total Cronbach's alpha of 0.82 [21]. Also in the Polish version, the subscales are below the acceptable level of 0.50, 0.38 and 0.47 with a total Crombach alpha of 0.72 [20]. The Philippine version reports subscale values ​​of 0.56, 0.64, and 0.67 with an overall Crombach alpha of 0.80 [31]. The best results were achieved by the Chinese version with subscale values ​​of 0.83, 0.84 and 0.82 with a total Crombach alpha of 0.92 [18].

Analyzing the results of the internal consistency coefficient in different studies, it is evident that in general the translated versions show acceptable reliability, while certain subscales could benefit from further refinement to improve internal consistency.

Internal consistency is calculated to determine the degree to which items within a test are correlated and consistently measure the same construct [32]. The maximum split-half reliability is 0.87, and Guttman Lambda 6 reaches 0.89, indicating excellent internal consistency. The average split-half reliability and Guttman Lambda 3 are both 0.80, suggesting good overall reliability. Guttman Lambda 2 provides a slightly more conservative estimate at 0.81, while the minimum split-half reliability, at 0.60, represents a lower bound for reliability. These results confirm the scale’s consistency, with some variation across different reliability estimates. Among the available translated and validated versions, only the Turkish version used the Spearman-Brown Split-Half analysis, dividing the entire scale into two halves, which indicated satisfactory internal consistency with a test value of 0.889 [23].

Regarding the test-retest reliability of the PZ-PUKT, the overall reliability is excellent (ICC = 0.91), indicating strong consistency between test and retest scores. The Prevention subscale demonstrated the highest reliability among the subscales (ICC = 0.88), followed by Staging (ICC = 0.86) and Wound Description (ICC = 0.84). The percentage of correct answers remained stable between the test and retest for all subscales, with minor variations. These results confirm the high reproducibility of the PZ-PUKT across time, with narrow confidence intervals for all reliability estimates. Compared to our study, the Turkish study shows similar but slightly weaker reliability results, with high correlations ranging from 0.746 to 0.871 [23].

The analysis of the instrument items was carried out using the difficulty index and the discrimination index. The aim was to clarify the reliability and potential dilemmas and represent a key step in the analysis by assessing the validity of individual test items [33]. The difficulty index (DFI) is calculated based on the percentage of correct answers in relation to the total number of responses, indicating the ease or difficulty of individual items [34]. It represents the percentage of participants who answered a specific test item correctly. If a difficulty factor is less than 0.30 it is considered difficult, higher than 0.70 is considered easy, and the desired range falling between 0.30 and 0.70 [35].

The difficulty index (DFI) of test items into easy, moderate, and difficult based on the proportion of correct responses. Easy items (DFI> 0.70) comprise the largest group, with 33 items, indicating they were answered correctly by most respondents. Moderate items (0.30 ≤ DFI ≤ 0.70) account for 25 items, representing an optimal range of difficulty. Difficult items (DFI < 0.30) include 14 items, suggesting these were the most challenging for participants. The distribution of difficulty is reasonably balanced, with a number of items across different difficulty levels.

The discrimination index of an item shows how well the item distinguishes between high and low scoring participants [36]. To determine the discrimination index of items, participants were divided into two groups: the top 27% of performers and the bottom 27% of performers [37].

The discrimination index (DSI) of items into good, moderate, reasonable, and poor categories is based on their ability to differentiate between high- and low-performing respondents. Analyzing the results of the discrimination index, it can be concluded that 40 items indicate effective and acceptable discrimination in distinguishing performance levels, with DSI values ranging from 0.20 to ≥ 0.30. A reasonable group of items (0.10 ≤ DSI <0.20) includes 17 items, suggesting these items may require smaller review or revision. Finally, poor items (DSI < 0.10), comprising 15 items, are prime candidates for review. Some questions (such as "Dragging the patient up in bed increases friction" or "It is the nurse’s responsibility to be sure a specialty bed is working") had a lower discrimination index; however, instead of reflecting a flaw in the validity of the questions, this may be due to a lack of knowledge among nurses and weaknesses in the education of the surveyed population.

4.1. Strengths of the Study

This cross-sectional study provided the first validated questionnaire for assessing healthcare professionals' knowledge of pressure ulcers in Croatia, covering three factors: Prevention, Wound Description, and Staging. Nurses are in a unique position to provide healthcare, as they are authorized through their competencies to implement preventive measures, classify, and treat pressure ulcers/injuries. This study has yielded positive findings regarding the PZ-PUKT instrument as a standardized tool for assessing nurses' knowledge, contributing to the delivery of effective and purposeful care.

4.2. Limitations of the Study

A limitation of this study is the presence of items with low discrimination power, the need for potential revision of certain test items, and challenges related to item difficulty. Despite a generally balanced distribution of difficulty levels, these factors highlight areas for improvement. Limitations of this study include the fact that it was conducted in only one public hospital, and since it is a cross-sectional study, the results can only be applied to the moment of the study, as the level of knowledge changes through formal and lifelong learning.

5. Conclusions

The aim of this study was to conduct the adaptation, translation, and validation of the questionnaire, as well as to analyze its internal consistency, making it a valuable tool for measuring knowledge about pressure ulcers, suitable for use in Croatian society.

Analytical procedures such as content validity, internal consistency, test-retest reliability, item-level difficulty, and discrimination indexes, conducted to assess the quality and reliability of the instrument, indicate satisfactory results.

The instrument can be used to assess knowledge in clinical settings of healthcare, social, and educational institutions and evaluate the effectiveness of educational programs, providing valuable insights into healthcare professionals' competencies. The data obtained from this instrument can be used to develop formal and lifelong educational programs, targeted guidelines, educational initiatives, and training on the topic, thereby further enhancing the quality of care and the professional development of healthcare workers.