Inter-rater reliability of an exercise adherence measurement tool: Measurement Of adherence Via Exercise Demonstration (MOVED)

Objective: To explore the inter-rater reliability of the Measurement Of adherence Via Exercise Demonstration (MOVED) adherence tool. Design: Reliability study of a patient adherence measurement tool. Setting: Simulated physiotherapist-patient consultations. Participants: Sixteen experienced physiotherapists rated patient adherence to exercise. Interventions: N/A Main Outcome Measure: Inter-rater reliability of MOVED. Methods: The MOVED tool consists of two parts. Part one asks patients to self-report their adherence to exercise dose (including number of completed sessions, sets and repetitions in the last seven days). Part two asks patients to demonstrate each exercise. Component scores are totalled to give an overall level of patient adherence to each exercise. Participants were provided with a copy of five physiotherapist-prescribed exercises. Participants were then shown five corresponding video vignettes of simulated physiotherapist-patient consultations where patients were asked to self-report exercise frequency, sets and repetitions and demonstrate each exercise within the context of a consultation. Participants were asked to rate the level of patient adherence to each of the five exercises using the MOVED tool. Inter-rater reliability of MOVED scores was assessed using Intraclass Correlation Coefficient (ICC) and 95% Confidence Intervals. Results: The ICC of part one (self-report) was .90 (95%CI .74-.98), part two (demonstration) was .98 (95%CI .94-.99) and total score was .96 (95%CI .88-.99), demonstrating excellent inter-rater reliability. Conclusion: The MOVED tool, which can highlight whether patients are adhering to exercise technique as well as exercise dose, may provide clinicians and researchers with a more robust measure of exercise adherence when compared with other measures currently available.


Introduction
Prescribed exercise has been shown to be beneficial in improving patient outcomes such as pain, mobility, range of movement and balance for a range of health conditions with many clinical guidelines actively advocating for the inclusion of exercise programs for low back pain, knee osteoarthritis (OA), hip OA (1,2). However, the effectiveness of an exercise program can only be established if patients actually do them. Adherence has been defined as the extent to which e person's behaviour coincides with agreed clinical recommendations (3). Levels of home-based exercise adherence are reportedly low for a range of conditions including low back pain, knee OA and pelvic floor weakness (4). The measurement of adherence to home-based exercise is complex with limited evidence to support the clinimetric properties of many adherence measures (5,6). Patient self-reported surveys and questionnaires are the most commonly used measure of adherence which are prone to reporting bias, often in the form of over-estimation of adherence (5,7,8). Objective measures of adherence such as smart phone applications and smart watches are often used to track activity such as step count or distance walked. The limitation of these measures is that they do not capture adherence to exercise technique, such as spinal posture during squatting, or that the exercise patients are doing at home accurately resembles the one that was prescribed. For this reason, it is recommended that adherence measurement tools of home-based or unsupervised exercise programs employ a combination of measures which include an observational component (5,7,9). One recent study reviewed patient adherence to exercise measured by patient self-report, physiotherapist-perception and an observational component where the exercise demonstrated by the patient was compared to the exercise prescribed by the physiotherapist (9). Agreement between patient-report and physiotherapistperception was 0.21, furthermore only eight of the 24 patients who self-reported being completely adherent to the exercise program were able to accurately recall and demonstrate their prescribed program to an independent observer (9). The findings of this study led to the development of an adherence measurement tool called the Measurement Of adherence Via Exercise Demonstration (MOVED). The MOVED tool combines patient self-report with an observational component to provide a composite exercise adherence score (Appendix 1). The aim of this study is to explore the inter-rater reliability of this measure.

Design and setting
A reliability study was conducted within a local health district in NSW. Ethics approval was granted through the Human Research Ethics Committee, the University of Newcastle, NSW.

Participants
Physiotherapists working within the local health district were invited to attend a workshop on strategies to facilitate patient adherence to exercise as part of their continuing professional development program. Following the presentation, attending physiotherapists were asked if they were willing to participate in a research study exploring the reliability of an exercise adherence measurement tool. Physiotherapists who did not wish to take part were able to leave at any time. Consenting physiotherapists were then provided with an overview of the MOVED measurement tool. Five video vignettes of simulated physiotherapist-patent consultations were shown with participants being asked to score each patient's level of adherence to the exercise contained within each video vignette using the MOVED tool. Copies of the physiotherapist-prescribed exercise instructions for each exercise were provided to participants to assist when completing MOVED.

Data Collection
Data were collected during September 2017.

Measures
Participants: Participant demographics including sex, number of years practicing as a physiotherapist, setting of main physiotherapy practice and possession of a post-graduate physiotherapy qualification were collected via a survey.
Adherence measurement tool: The MOVED measurement tool is divided into two parts: part one assesses self-reported adherence to dose and part two assesses adherence to technique using exercise demonstration. Scores for parts one and two are then added together to give an overall score for adherence. Appendix 1.
Part one of MOVED contains 3 questions. Patients are asked to self-report how often they completed each individual exercise contained within an exercise program in terms of 1) frequency of completed exercise sessions per day/week and 2) number of sets completed per session and 3) number of repetitions completed per set in the last seven days. The responses to each of the three questions are then compared to the physiotherapist prescription to give an adherence composite score for patient adherence to exercise dose. Physiotherapist participants used the patient responses given in each of the five vignettes to score part one for five different patients. Response options for part one included: Yespatient-report exactly matches physiotherapist prescription (2 points); No-patient reports doing more than prescribed (1 point); No -patient reports doing less (1 point) and No-patient reports being nonadherent (0 point).
Part two of MOVED then requires the patient to demonstrate each exercise with technique compared to the physiotherapist prescription of the essential components of the exercise. Physiotherapist participants scored each patient using the following question: Is the exercise demonstrated the same as the physiotherapist prescription? response options include: yes-exactly (3 points), >50% (2 points), <50% (1 point), not at all (0 points).

Data Analysis
Data analysis was completed using Stata 14® (USA).
Demographic data: Participant demographics were described using mean, range and percentages. Physiotherapist participant characteristics were compared with Australian physiotherapist workforce characteristics using the one-sample Pearson chi square test. Table 1.
Intraclass Correlation Coefficient (ICC) with 95% Confidence Interval using a two-way random-effects model were used to assess inter-rater reliability of MOVED scores for each question on the MOVED tool as well as the overall level of adherence score. Table 2.

Results
Participants: Almost two-thirds of physiotherapist participants were female. The main setting of physiotherapy practice included: hospital (n=6), community (n=5), domiciliary (n=4) and residential aged care (n=1). On average, participants worked 34 hours per week and had been practicing physiotherapy for 15 years. There were no significant differences between the physiotherapist participants and statistics from Health Workforce Australia. Table 1.

Discussion
Accurate measurement of adherence to home-based, unsupervised exercise poses many challenges.
Measures based on patient self-report are often subject to reporting bias, generally in the form of overestimation (5,7). Physiotherapist-perception of adherence may also be biased and may not accurately reflect what patients are doing at home (9). A limitation of objective measures are that they may not capture adherence to exercise technique (9). The MOVED adherence tool offers an opportunity to capture patient self-report to exercise dose as well as technique. It can also be used to highlight where the patient is having difficulty in adhering. If the patient self-report is different from the physiotherapist prescription, this could act as a prompt to the physiotherapist to further explore the reason for this. For example, if patient recall of exercise instructions is the issue this could be improved by giving printed information or using the patient's smartphone to film the patient doing the exercise as a resource to refer to once home. If the exercise is too hard or causes pain when demonstrated, then adherence may be improved through exercise modification.
The MOVED adherence tool demonstrated excellent inter-rater reliability for each component parts one and two as well as the final composite score and may provide clinicians and researchers with a more robust measure of exercise adherence when compared with other measures currently available.

Study limitations
The main limitation of this study was that simulated physiotherapist-patient consultations were used. Reliability of the MOVED tool may not translate to a real-world clinical context. In addition, the MOVED tool rates patients as adherent or non-adherent using a very narrow scoring system, with a much broader scoring of partial adherence. However, as the tool uses a composite score, the aim is that clinicians will be able to unpack the reasons patients are partially adherent and adopt targeted strategies to assist in supporting behaviour change.

Conclusions
The MOVED adherence measurement tool demonstrated excellent inter-rater reliability using simulated clinical scenarios. The MOVED tool is designed to highlight whether patients are adhering to exercise technique as well as exercise dose, and thus may provide clinicians with a prompt to further explore which particular aspect of exercise adherence patients are having issues with leading to the implementation of individualised adherence aiding support.