Exploring the Discriminant Validity of the Modified Arm Care Screen (MACS), Designed for Overhead Athletes, in Detecting Musculoskeletal Risk Factors in the General Population

1. Introduction

Shoulder pain ranks as the third most prevalent musculoskeletal issue in primary care, following low back pain and knee pain. The prognosis for individuals experiencing shoulder pain varies significantly, with about 50% still reporting symptoms six months after seeking medical attention [1]. Beyond the pain, many suffer from functional impairments that can disrupt their work, hobbies, social interactions, and sports activities. This often leads to psychological distress and a diminished quality of life [2]. The societal costs of shoulder pain are substantial. A cost-estimation study conducted in Sweden revealed that the financial burden per patient is approximately €4.139, with over 80% of these expenses attributed to sick leave [3].

Shoulder injuries are especially common in sports that require repetitive overhead movements at high speeds or in extreme positions, such as swimming, volleyball, and handball. Athletes participating in these sports, often referred to as ‘overhead athletes,’ are at a significant risk of shoulder injuries [4]. Numerous studies have attempted to identify risk factors and prevention strategies for these athletes [5]. The risk of shoulder injuries varies across different sports and depends on the definition used, such as time loss from sports, need for medical attention, or severity of the injury [5].

In swimming, between 23% and 38% of athletes experience shoulder injuries within a year [4]. Similarly, 23% of volleyball players report dominant shoulder pain during the ongoing season [6]. In a large study of elite handball players, 44% to 75% had experienced shoulder pain previously, with 20% to 52% reporting current shoulder pain [7]. The prevalence of weekly shoulder pain and substantial shoulder injuries in these athletes is 28% and 12%, respectively [4]. High rates of shoulder injuries are also observed in other overhead sports, such as baseball and water polo, with variations depending on factors like age, sex, and competition level [4].

Screening tools have been used in the past to identify athletes at risk of developing shoulder pain [8]. These tools are either related to sporting level, workload ratio, muscle strength and endurance [5,9] or special tests such as the Arm Care Screen Test (ACS) [10], the Closed Kinetic Chain Upper Extremity Stability Test (CKCUEST) [11] and others. Although these tests have shown promising results in overhead athletes, less is known about the usefulness of similar tools in the general population.

A screening tool known as the Arm Care Screen (ACS), based on Functional Movement Systems (FMS) principles, has been implemented to quickly and easily identify baseball players at a higher risk of shoulder injuries [10]. The ACS has demonstrated excellent reliability among high school baseball coaches, with intra-rater agreement (k = 0.76; 95% CI 0.54–0.95) and inter-rater agreement (k = 0.89; 95% CI 0.77–0.99)[12]. Previous studies have shown that the ACS can effectively differentiate between baseball players who have musculoskeletal risk factors for shoulder injuries and those who do not [10]. Additionally, recent examinations of a modified version of the ACS for its discriminant validity have yielded promising results in other overhead athletes, including those in volleyball, tennis, and basketball [8].

Although preliminary findings indicate that the modified ACS can effectively screen overhead athletes both on the field and during comprehensive clinical assessments, its discriminant validity in non-athletes remains less understood. No relevant studies have investigated the necessity of screening tools in evaluating musculoskeletal risk factors in the general population, particularly in those engaging in daily or occupational overhead activities. The prevalence of pathologies within these populations, such as shoulder impingement, rotator cuff tears, and tendonitis, is significant [1]. Recent data suggests that the percentage of individuals experiencing such conditions remains high [1], indicating a substantial need for effective screening tools to mitigate these risks.

Consequently, the objective of this study was to assess the discriminant validity of the Modified Arm Care Screen (MACS) in identifying common musculoskeletal risk factors in the general population. We hypothesized that suboptimal performance on the MACS subtests would exhibit high sensitivity in detecting the presence of at least one associated musculoskeletal risk factor among individuals in the general population who participate in daily overhead activities.

2. Materials and Methods

A prospective cross-sectional approach was used to assess the effectiveness of the MACS in distinguishing between the presence and absence of musculoskeletal risk factors among individuals in the general population. The study followed the Standards for Reporting Diagnostic Accuracy Studies (STARD) guidelines to ensure standardized and thorough reporting of the diagnostic accuracy study design. Ethical approval was obtained from the University of West Attica (approval number: Approved Proposal 25th of April 2024) prior to data collection. Informed consent was secured from all participating subjects, who were briefed about the study procedures and objectives.

2.1. Sample

To determine the appropriate sample size, we based our calculations on a prevalence rate of 70% for the risk factor of participant injury, as indicated by previous research studies [8,10]. To achieve a sensitivity of 0.90 for the Arm Care Screen (ACS) with a confidence interval (CI) width of 0.10, a minimum sample size of 50 participants was required [8,19]. Eligibility criteria for participation included being an active subject over the age of 18 and being asymptomatic with no history of shoulder surgery or shoulder-related treatments. Asymptomatic status was defined according to our previous research [20,21]. Moreover, subjects should have been regularly performing overhead activities in their daily lives. Exclusion criteria included any prior occurrence of shoulder pain that led to abstention from participation for more than three consecutive days from their activities of daily living, as well as a history of shoulder injury or physical therapy related to the chest, shoulder, or neck. A total of 30 subjects met the inclusion criteria and participated in the study.

2.2. Modified ACS

Data collection was conducted in weekends to minimize the risk of work-related injuries or symptoms from delayed muscle onset from activities of daily living. The data collection spanned 2 months and involved assessing subjects by using two independent assessors who were trained in administering the modified Arm Care Screen (ACS). Demographic information of the subjects was recorded prior to the assessment.

The original ACS comprises three subtests:

Reciprocal Shoulder Mobility: The subject stands with feet together and reaches one hand behind the head while extending the other hand behind and up the back, attempting to touch both fingertips. A positive result is recorded if the subject cannot achieve this on one or both sides while maintaining an upright posture.

90/90 Total Body Rotation: The subject stands with feet together and arms positioned at 90 degrees. They then rotate their entire body—including hips, shoulders, and head—while keeping their feet pointing straight ahead. A positive outcome is noted if the front of the opposite shoulder becomes invisible while maintaining proper posture.

Lower-Body Diagonal Reach: The subject stands two shoe lengths from a wall, balances on one leg, and reaches the opposite leg behind and across to touch the wall three times without touching the ground. The subject then repeats the test, aiming to touch the wall five times without losing balance. Failing to touch the wall five times consecutively without losing balance is considered a positive result. This test was adapted from key components of the Y-Balance test [13].

Figure 1. Modified ACS: (A) reciprocal shoulder mobility, (B) 90/90 total body rotation, and (C) lower body diagonal reach, (D) rotary stability.

Figure 1. Modified ACS: (A) reciprocal shoulder mobility, (B) 90/90 total body rotation, and (C) lower body diagonal reach, (D) rotary stability.

2.3. Risk Factors

To enable the ACS to detect musculoskeletal risk factors accurately, specific cut-off thresholds were established for each risk factor. Impairments were classified as either present or absent based on whether they fell below the defined cut-off values for physical impairments. These thresholds were determined from existing literature and values reported in previous research, as detailed in Table 1. These same values were utilized in the study of Matsel, et al. [14]. Two physical therapists independently assessed these ten musculoskeletal risk factors, separate from the ACS scoring process. Nine of these factors are detailed in the study of Matsel, Hoch, Butler, Westgate, Malone and Uhl [14]. The added risk factor was the Closed Kinetic Chain Upper Extremity Stability test (CKCUES) [15]. The procedures for the CKCUES have been described previously [16]. If a subject’s score fell below the threshold values reported in the literature (18.5 touches for males and 20.5 touches for females from a modified position), this was considered indicative of an existing risk factor. Although a recent study found that subjects scoring less than 21 touches are 18 times more likely to sustain an injury [17], we opted to use threshold values from an earlier study [18] which established different cut-off scores for men and women (18.5 touches for males and 20.5 touches for females). All risk factors, measurement procedures, and ACS tests used for comparison are detailed in Table 1.

To evaluate the validity of the ACS, we compared the results of each of the four ACS assessments (pass/fail) with the corresponding risk factor measurements (pass: above the cut-off values, fail: below the cut-off values). We assessed the diagnostic performance of the ACS by calculating sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratios (LRs).

Here’s how each result was classified:

• True Positive (TP): This was recorded when a participant had at least one shoulder or thoracic mobility risk factor, which was correctly identified by a positive result in the ACS reciprocal shoulder mobility test on either side.
• True Negative (TN): This occurred when the participant passed all goniometric shoulder and thoracic range of motion (ROM) tests and received a negative result on the reciprocal shoulder mobility test for both sides.
• False Positive (FP): This was noted when no shoulder or thoracic risk factors were present, yet the participant tested positive on either side of the reciprocal shoulder mobility test.
• False Negative (FN): This was observed when a participant had at least one shoulder or thoracic risk factor but tested negative on the reciprocal shoulder mobility test for both sides.

The same process was applied to evaluate the remaining three tests within the ACS: Total Body Rotation, Lower Body Diagonal Reach, and Rotary Stability, as detailed in Table 1.

2.4. Statistical Analysis

Demographic data were analyzed using descriptive statistics. Each musculoskeletal risk factor was categorized as previously described and recorded in separate 2x2 tables for each of the four components of the modified ACS. Chi-square tests were conducted to determine associations between each ACS component and the corresponding risk factors. When one or more expected cell frequencies were less than five, Fisher’s Exact test was applied [31]. The strength of the association was assessed using the Phi (Φ) coefficient, as previously recommended [32]. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), likelihood ratios (LRs), and odds ratios were calculated according to the methods outlined in previous research [33,34]. All statistical analyses were conducted using SPSS (IBM, version 25). A significance level of p < 0.05 was set for determining statistical significance in all tests performed.

3. Results

Demographic data for all 30 subjects are summarized in Table 2. Chi-square tests revealed non-significant associations between each component of the modified ACS and the related risk factors. The strength of these associations, indicated by Phi (Φ) values, ranged from 0.089 to 0.905, signifying a small to medium association (Table 3). Table 3 displays the precision of each component in the modified ACS in identifying individuals with musculoskeletal risk factors.

The sensitivity ranged from 47 to 0 among the four components of the modified ACS demonstrating a relatively low true positive rate, whereas a broader range of specificity values were found that ranged from 100 to 55, demonstrating a moderate to strong positive rate. Positive LRs, indicate the fold increase in the odds of having a particular condition in a participant with a positive test result [35]. This value ranged from 1.07 to 3.47. Negative LRs, indicate the fold decrease in the odds of having a particular condition in a participant with a negative test result [35].This value ranged from 0.75 to 1.21. The Positive Predictive Value (PPV) indicates the probability of the pathology in subjects that test positive and this value ranged from 0 to 100%, whereas the NPV which is the opposite in subjects that test negative ranged from 65 to 75% (Table 4).

The accuracy value indicated the overall probability of correct classification of each subject. This value ranged from 53 to 74 in the four components of the modified ACS. Finally, athletes with pre-existing risk factors experienced significantly increased odds of failing various tests. Specifically, they had 3.88 times higher odds (95% CI 1.186 to 12.74) of failing the reciprocal shoulder mobility screen, 130.3 times higher odds (95% CI 20.171-842.128) of failing the total body rotation screen, 48.87 times higher odds (95% CI 11.086-215.461) of failing the lower body diagonal reach, and 35.53 times higher odds (95% CI 7.45-169.33) of failing the rotary stability test compared to athletes who did not have risk factors, as presented in Table 4.

4. Discussion

The primary objective of this study was to assess the discriminant validity of the modified Arm Care Screen (MACS) in identifying musculoskeletal risk factors among individuals in the general population who participate in daily overhead activities. Through comprehensive statistical analysis, we derived key diagnostic values such as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), likelihood ratios (LRs), and odds ratios. Contrary to our initial hypothesis, the findings indicated that the modified ACS did not effectively identify musculoskeletal risk factors among individuals in the general population.

4.1. Key Findings

The study demonstrated that the modified ACS, while effective in identifying risk factors in overhead athletes, did not translate well to the general population. This discrepancy may be attributed to several factors, including differences in physical demands, movement patterns, and baseline musculoskeletal health between athletes and non-athletes [36]. These variations necessitate the development of screening tools that are specifically tailored to the unique characteristics and needs of subjects in the general population [36].

4.2. Sensitivity and Specificity

The sensitivity values for the modified ACS components ranged from 0% to 47.62%, indicating a low true positive rate in detecting musculoskeletal risk factors among individuals in the general population. Conversely, the specificity values ranged from 55.56% to 100%, suggesting a moderate to high true negative rate. This disparity underscores the need for more targeted screening tools for individuals in the general population, as the MACS may not adequately capture the nuances of their musculoskeletal health [35,36]. Low sensitivity suggests that many individuals with risk factors were not flagged by the MACS, potentially missing early intervention opportunities [10].

4.3. Positive and Negative Predictive Values

The PPV and NPV values further highlighted the limitations of the modified ACS in the general population. With PPV values ranging from 0% to 100% and NPV values from 65% to 75%, the modified ACS showed inconsistent performance. The high PPV for certain components, such as the Rotary Stability test, indicates a strong probability of risk factors in those who test positive. However, the low NPV suggests a significant number of false negatives, potentially overlooking individuals with actual risk factors [35]. This inconsistency in predictive values indicates the need for enhanced diagnostic criteria and screening protocols tailored in the general population [8].

4.4. Likelihood Ratios and Odds Ratios

The likelihood ratios (LRs) and odds ratios provided additional insights into the diagnostic accuracy of the modified ACS. Positive LRs ranged from 1.07 to 3.47, while negative LRs ranged from 0.75 to 1.21. These values reflect a limited increase in the probability of having a risk factor given a positive test result, and a slight decrease in the probability given a negative test result. The odds ratios for failing the ACS components were significantly higher in subjects with pre-existing risk factors, particularly for the Total Body Rotation and Lower Body Diagonal Reach tests. This indicates that individuals with musculoskeletal risk factors are more likely to fail these specific tests, but the overall effectiveness of the ACS in the general population remains questionable [37]. This reinforces the importance of developing diagnostic tools that offer higher sensitivity and specificity for the general population [36,38].

4.5. Implications for Screening and Prevention

The findings suggest that while the MACS may serve as a useful tool for screening and prevention of shoulder injuries in overhead athletes, its application in the general population is limited. The discrepancies in diagnostic accuracy between athletes and individuals in the general population highlight the need for population-specific screening tools [37]. These individuals may require assessments that account for their unique physical activities and musculoskeletal profiles, rather than relying on tools developed for athletic populations [5]. This points to a broader implication for public health initiatives aimed at injury prevention, emphasizing the customization of screening methods to fit various lifestyle and physical activity levels [5].

4.6. Limitations and Future Research

This study had several limitations, including a small sample size and the exclusion of individuals with a history of shoulder pain or surgery. Future research should aim to include a larger, more diverse sample and explore the development of screening tools tailored to non-athletes. Additionally, longitudinal studies could provide valuable insights into the effectiveness of such tools in preventing musculoskeletal injuries over time. There is a clear need for the refinement of MACS components and the exploration of alternative risk factors that may be more relevant to non-athletic populations [36]..

5. Conclusions

In conclusion, while the MACS is effective for overhead athletes, it does not adequately identify musculoskeletal risk factors among individuals in the general population. These findings underscore the importance of developing and validating population-specific screening tools to enhance the accuracy of musculoskeletal risk assessment and injury prevention strategies. Further research is necessary to refine these tools and improve their applicability across different populations. By advancing our understanding and methods of screening, we can better address the diverse needs of individuals and enhance preventative care in musculoskeletal health.