Process of Reliability of the Ventilatory Workload Kinetic Index and Prioritization in the Intrahospital Clinical Setting

1. Introduction

In the clinical setting, the evaluation of respiratory compromise is a substantive concern since by weighing this concept of severity, the conduct to be followed by the health team is planned [1]. In everyday practice, therapies are guided using instruments that primarily rely on applying hierarchical ratings to categorize the severity of respiratory problems through scales or scores based on biometric characteristics [2]. Therefore, to establish whether a hierarchical system is consistent, it must be subjected to a process of analysis of its psychometric properties, particularly its reliability [3]

Scales have been developed through consensus, protocols, or conveniences that respond to real clinical problems. Subsequently, applying a methodological and scientific basis to any instrument is essential to evaluate its performance. Using a tool without having subjected it to a control, validation, or comparison system compromises its reliability. Therefore, these tools must be supported by analyses derived from these biometric processes [2,3]. In this context, there is a legitimate issue over the actual impact of scales in the field of physiotherapy; however, their usage is on the rise. Most of these scores consist of assigning values to a set of variables that categorize the severity of a disorder, which, in this particular case, is to assess respiratory compromise [1].

In this context, the Ventilatory Workload Kinetic Index (VWKI) has been proposed as a clinical instrument to evaluate ventilatory balance-imbalance and standardize the assessment of the respiratory system [4,5] based on a theoretical model that recapitulates the imbalance between ventilatory needs and neuro-cardio-respiratory capacities [6]. In such a condition, it is important to determine whether instruments like scales can effectively evaluate the complexity of the symptomatology and clinical status in a wide variety of instances [7]. Although technological development with its contributions has managed to reduce the variability of the results, reliable clinical measurement in its “bedside” execution as a learning phenomenon.

One of the tasks of respiratory specialists, in addition to providing professional care and staying up-to-date with their knowledge through continuous training, is to establish competencies that demonstrate the reliability and importance of their clinical measurements. This allows them to effectively control and use specialized instruments in their field of expertise [7]. Thus, the regulated application of scales that allow an integral evaluation of the object of study and their subsequent reliability evaluation should be a continuous, systematic, and situated process [8]. Therefore, this research sought to determine the reliability of the VWKI between two evaluators and its applicability in the clinical field of physiotherapy.

2. Materials and Methods

This cross-sectional exploratory study was conducted in January 2024 at the Hospital El Carmen de Maipú (HEC). The following services were considered: medicine, intensive care unit (ICU), and others (composed of Surgery and Geriatrics). This research was approved by the Scientific Ethics Committee (letter N° 39/2024) of the Central Metropolitan Health Service.

2.1. Study Design

It was divided into three stages: i) induction period, ii) application of the VWKI, and iii) reliability process (Figure 1). In parallel to the reliability process, the following information was retrieved from each participant's file on the assessment day: i) medical diagnosis, treatment, history of morbidities, and days of hospitalization. Finally, the physiotherapy coordinator reported the prioritization of each participant's care, which were category 1 (≥2 visits) and category 2 (≤1 visit).

2.1.1. Induction Period

To optimize the clinimetry of the scale, an induction period was developed, which was divided into two stages; i) presentation and orientation: 2 expository sessions of one hour each were held, where the VWKI was presented, and the material was handed out for subsequent discussion; ii) feedback: 2 sessions of one hour each where doubts were resolved, and behavior was unified for the evaluation of the different variables of the VWKI.

2.1.2. Application of the VWKI

The implementation of the VWKI in clinical practice consisted of evaluating 10 patients guided by the expert. It should be noted that the construction of the scale used is given under a theoretical model of loads and supports [1,6]. The VWKI was measured as follows: i) additional oxygen contribution (O₂) was determined, ii) respiratory rate (RR) was measured, iii) use of inspiratory and/or expiratory accessory musculature (UAM), iv) pulmonary murmur (PM) and airway resistance (AR), v) cough and attempts to permeabilize the airway (APA) [1].

2.1.3. Reliability Process

The required sample was calculated using the G*Power software version 3.1.9.7 with a power of 80%, a significance level of 0.05, and an effect size (f) of 0.4, based on the reduction in the VWKI reported by Muñoz-Cofré et al. (2024). According to the sample size calculation, a minimum of 81. measurement is required to detect a difference in VWKI; it must be an a priori calculation of a measure in the hospitalized population; a dropout percentage was not considered. Evaluator A (EA) was a physiotherapist from the Physical Medicine and Rehabilitation Service of the HEC, and Evaluator B (EB) was a physiotherapist expert in respiratory assessment (Chilean Ministry of Health) with 20 years of experience. The participants were randomly selected on the day of the assessment, and the application of the VWKI was isotemporal, so the variables were not modified. After applying the VWKI, the EA and EB recorded the values of each variable separately to avoid bias. Inclusion criteria were: i) 18 years of age or older, ii) hospitalized in Medicine, ICU, Surgery, or Geriatrics, iii) oriented in time and space and able to cooperate in the respiratory assessment. Exclusion criteria were: i) being connected to non-invasive mechanical ventilation, ii) patients with an indication not to mobilize, and iii) patients discharged from the hospital on the day of the assessment.

2.2. Statistical Analysis

SPSS software version 25.0 was used for all analyses. Median, minimum, and maximum scores were calculated for each variable, as well as the total score of the VWKI. Inter-rater reliability (EA and EB measurements) was analyzed using the intraclass correlation coefficient (ICC) with a 95% confidence interval (CI). The following interpretation of ICC was used: <0.5, poor reliability; 0.51 to 0.75, moderate reliability; 0.76 to 0.90, good reliability; and >0.90, excellent reliability. For comparisons of VWKI by user prioritization and by services, the normality of the data was first determined using the Kolmogorov-Smirnov test. Then, the Student's t-test or Mann-Whitney U test and an ANOVA or Kruskal-Wallis test, respectively, were used. The level of significance was set at p < 0.05.

3. Results

In relation to the stability of the VWKI variables (Table 1). Loadings such as the PM obtained good inter-rater reliability (ICC = 0.770, p < 0.05). In Translators, the RR and APA showed good inter-rater reliability (ICC = 0.831, p < 0.05; ICC = 0.889, p < 0.05, respectively), the SO₂ obtained excellent inter-rater reliability (ICC = 0.986, p < 0.05), while in Supports, the O₂ showed excellent reliability (ICC = 0.982, p < 0.05) and the cough showed good reliability (ICC = 0.820, p < 0.05). Finally, the VWKI total score obtained excellent inter-rater reliability (ICC = 0.913, p < 0.05) (Table 1). On the other hand, the AR showed moderate inter-rater reliability (ICC = 0.528, p < 0.05), while the UAM showed poor inter-rater reliability (ICC = 0.483, p < 0.05) (Table 1).

Concerning the physiotherapy care prioritization system, for both EA and EB, the total VWKI score was significantly higher in priority 1 (p = 0.001; p < 0.05, respectively) (Figure 2). On the other hand, when comparing the total VWKI score by services, it was observed that both EA and EB rated the VWKI significantly higher in the ICU than in Medicine and the other services (p = 0.001, p = 0.0001 and p < 0.05, p < 0.05, respectively) (Figure 3).

Table 1. Reliability of the VWKI variables and the total score.

	Evaluator A	Evaluator B	ICC (CI95%)	p Value
LOADS
Pulmonary murmur	2 (0-3)	2 (0-3)	0.770 (0.538-0.819)	<0.05
Airway Resistance	2 (0-3)	1 (0-3)	0.528 (0.247-0.702)	<0.05
TRANSLATORS
Respiratory Rate	1 (0-2)	1 (0-3)	0.831 (0.797-0.891)	<0.05
Oxigen Saturation	2 (0-3)	2 (0-3)	0.986 (0.978-0.991)	<0.05
Use of Accessory Musculature	1 (0-3)	2 (0-3)	0.483 (-0.043-0.724)	<0.05
Attempts to Permeabilize the Airway	1 (0-3)	1 (0-3)	0.889 (0.828-0.928)	<0.05
SUPPORTS
additional oxygen contribution	1 (0-3)	1 (0-3)	0.982 (0.972-0.988)	<0.05
Cough	2 (0-3)	2 (0-3)	0.820 (0.720-0.884)	<0.05
VWKI TOTAL	11 (2-21)	12 (2-20)	0.913 (0.864-0.944)	<0.05

The values are presented as median (minimum-maximum). ICC: intraclass correlation coefficient; CI: 95% confidence interval.

Table 1. Reliability of the VWKI variables and the total score.

	Evaluator A	Evaluator B	ICC (CI95%)	p Value
LOADS
Pulmonary murmur	2 (0-3)	2 (0-3)	0.770 (0.538-0.819)	<0.05
Airway Resistance	2 (0-3)	1 (0-3)	0.528 (0.247-0.702)	<0.05
TRANSLATORS
Respiratory Rate	1 (0-2)	1 (0-3)	0.831 (0.797-0.891)	<0.05
Oxigen Saturation	2 (0-3)	2 (0-3)	0.986 (0.978-0.991)	<0.05
Use of Accessory Musculature	1 (0-3)	2 (0-3)	0.483 (-0.043-0.724)	<0.05
Attempts to Permeabilize the Airway	1 (0-3)	1 (0-3)	0.889 (0.828-0.928)	<0.05
SUPPORTS
additional oxygen contribution	1 (0-3)	1 (0-3)	0.982 (0.972-0.988)	<0.05
Cough	2 (0-3)	2 (0-3)	0.820 (0.720-0.884)	<0.05
VWKI TOTAL	11 (2-21)	12 (2-20)	0.913 (0.864-0.944)	<0.05

The values are presented as median (minimum-maximum). ICC: intraclass correlation coefficient; CI: 95% confidence interval.

4. Discussion

This study aimed to determine the inter-rater reliability and applicability of the VWKI in the clinical physiotherapy setting. The main results were: i) excellent inter-rater reliability of the VWKI total score, ii) low reliability of the AR and UAM variables, iii) the VWKI was consistent with the physiotherapeutic care prioritization system, i.e., the VWKI was significantly higher in priority 1 patients, and iv) the VWKI was able to determine the complexity of the patients, thus, the score of the patients in the ICU was significantly higher compared to the other units. In terms of clinical intervention, both the stability of the data obtained by the assessment tool and the discrimination of the data could contribute to the prioritization of intervention and clinical decision-making [3]. In addition to this, the complementary analysis of Loads, Translators, and Supports offers the opportunity to differentiate praxis considering the behavioral patterns of the respiratory system [5,9].

One of the main findings of this study was the excellent inter-rater reliability of the VWKI total score (ICC = 0.913, p < 0.05). This agrees with Cabib et al. (2004), who assessed the inter-rater reliability of the VWKI between two physiotherapists in patients ventilated with pressure support (PVPS). Their results showed the same median VWKI for both physiotherapists (EA = 11 (6-18), EB = 11 (6-17); p = 0.9, K = 0.84). In conclusion, the VWKI, applied in PVPS, is a reliable inter-physiotherapist instrument [10]. It is important to note that the sample of the present investigation i) did not include patients in PVPS, and ii) the sample was recruited from three services with different complexity. Therefore, the reliability of the VWKI would be independent of the patient's condition and the service in which he/she is hospitalized. In this way, having a source of intra-hospital information with discriminatory characteristics from the hierarchy of commitments allows for the planning of human and material resources, which, in contexts of high demand, favors adequate decision-making associated with a standardized process [7].

An important point to consider is the low reliability obtained in the RVA (0.528 <0.05) and UMA (0.483 <0.05) variables, whose evaluative elements are more subjective in the assessment of the evaluators. This has been reported as one of the elements that should be handled with greater care given the clinical implications that derive from these variables, namely the balance between the muscular contraction needed to break the elastance of the thorax and the lungs [6]. In this regard, the experts recommend emphasizing the standardization of measurement methods, observer training, and measurement automation [11]. Therefore, it would be very useful to generate complementary material, in addition to the scoring table, that explains and clarifies point by point the situations that could generate false positives in the VWKI measurement.

The VWKI was consistent with the physiotherapy care prioritization system proposed by the HEC physiotherapy coordination; here, patients with two or more attendances had a significantly higher total VWKI score relative to patients with one attendance. This is consistent with the findings of Quinteros et al. (2014), who set out to explain, automate, and disseminate the usefulness of the VWKI. They further concluded that the systematization of the VWKI would allow them to stratify the risk of pulmonary failure of hospitalized patients and to address, in an objective and specific way, the respiratory physiotherapeutic treatment of each patient according to their characteristics [9]. In the context of clinical practice, the ability to prioritize care in a service and/or a group of assigned patients is a value associated with the orientation of technical efforts in terms of procedural urgency, which does not necessarily coincide with biomedical urgency. At the same time, stratification of respiratory compromise based on the total VWKI score would allow optimization of the therapeutic offer considering the severity and specific characteristics of the respiratory disorder under evaluation [3].

Finally, to emphasize the "sensitivity" of the VWKI in assessing the physiotherapic complexity of patients, it is important to note that while the ICU score was significantly higher compared to the Medicine Service and other units, this difference is not directly related to the amount of time or the level of difficulty involved in the procedures. In 2004, a study was conducted to examine the behavior of the VWKI in patients with varying degrees of medical complexity. The findings of the study revealed a substantial correlation between higher levels of attendance and an increase in VWKI (VWKI 17 (11-23) vs. 7 (4-14); P < 0.0001), which is a confounding factor when medical complexity is assimilated with kinesthetic severity [12]. In this regard, Muñoz et al., 2024 concluded that the VWKI serves to identify ventilatory problems in outpatients and can also distinguish where the greatest deterioration lies, considering the differentiated analysis of loads, translators, and assists, which does not necessarily correspond to the levels of intervention, which are often stereotyped in respiratory physiotherapy [1].

There are certain limitations in this study that must be acknowledged: i) The imbalance in the number of patients per service is due to the HEC's clinical reality, which necessitates adjusting the number of beds based on the epidemiological behavior of the in-patient population. This ethical consideration makes it impossible to selectively choose patients. ii) The duration of the theoretical training of the VWKI should be reassessed in light of the experience gained. iii) The evaluations were reduced to only three services, so it would be necessary to establish the reality of the ventilatory compromise of the entire hospital center.

5. Conclusions

The conclusions drawn in this study were the excellent reliability of the total score of the VWKI, its consistency with the system of prioritization of physiotherapy care, and the ability to determine the severity of respiratory clinical pictures.