Methods and Tools for the Evaluation of Occupational Capabilities

1. Selection of Assessment Tool

The selection of the appropriate assessment tool by the researcher or educator is crucial for their work and research. They must consider all parameters and examine all issues before choosing the occupational therapy assessment tool (Alexopoulos, 2004).

2. Purpose of the Assessment

The purpose of the assessment can include detailed inquiry, determination of program content, evaluation of individual improvement, evaluation of the intervention program, as well as group allocation (Payne & Isaacs, 1998). The goal of the assessment dictates the choice of the appropriate test for the specific purpose and designated population (Alexopoulos, 2004). Some tests serve multiple purposes, necessitating validity checks for each purpose (Boyce et al., 1991).

Additionally, ensuring the adequacy and representativeness of the standardization sample on which the test norms are based is extremely important (Rudner, 1994).

3. Ease of Test Administration

The ease of test administration is significant in the selection process. Assessing a preschool-aged child needs to be a pleasant experience for both the child and the examiner. A simple scoring system maintains the examinee's interest and completes the measurement in a short time (Zittel, 1994).

4. Norm-Referenced and Criterion-Referenced Tests

4.1. Criterion-Referenced Assessment

Criterion-referenced assessment evaluates progress based on predefined criteria that set an improvement goal. The examinee can either meet or fail to meet these standards, without comparing their results to those of others. The "I Can" test (Wessel, 1976) is a well-known criterion-referenced test.

4.2. Norm-Referenced Assessment

In norm-referenced assessment, an individual's performance is compared to that of others to determine how the individual performs relative to the group. This allows these tests to be used for screening and diagnosing specific purposes (Zittel, 1994). The individual's performance on these tests is translated into relative scores (z-scores, standard scores, percentiles) based on norms from a sample group with similar characteristics. These norms need to be continually updated (Zittel, 1994). The BOTMP (Bruininks, 1978) is the most well-known norm-referenced test for children.

5. Bruininks-Oseretsky Test of Motor Proficiency (BOTMP)

In 1978, Robert Bruininks, using the Oseretsky Test of Motor Proficiency (Oseretsky, 1923) as a basis, developed an assessment battery for the motor performance of children aged 4-14 years. This battery is called the Bruininks-Oseretsky Test of Motor Proficiency (BOTMP) and comes in two forms: comprehensive and short. The comprehensive form includes 8 subtests (46 tests), providing a multifaceted view of motor performance (see Table 1 for motor performance metrics), and specific measurements in the area of motor skills, both gross and fine. The short form (14 tests) offers a brief overview of general motor performance. The standardization sample consisted of 765 typically developing children from the United States and Canada.

Each child is tested individually, with the comprehensive form taking 45 minutes to 1 hour and the short form taking 15-20 minutes. The test battery has a scoring system ranging from a 2-point (pass/fail) scale to a 16-point scale. First, the performance in each test is recorded (seconds, number of successful attempts, etc.) and then converted to the corresponding score. Scores from the tests of each subtest are summed, resulting in a subtest score. Subsequently, the subtest scores are converted to a standard score relative to the examinee's age. These scores are aggregated to yield the total score for gross or fine motor indices and the overall battery score. The battery allows for the comparison of fine and gross motor scores, as well as the total battery score, with percentiles and stanines.

According to the manual, the BOTMP is suitable for use in:

1. Screening

2. Placement within an educational framework (selecting the appropriate physical education

class)

3. Assessing gross and fine motor skills based on criterion-referenced scores (individual analysis

of subtest scores is not appropriate)

4. Designing and evaluating programs (scores from the battery enable diagnosis of motor status

by grouping students based on their motor progress and assessing the results of physical activity)

5. Diagnosing various developmental difficulties

6. Research

The BOTMP gained popularity, especially in the United States, shortly after its release. By 1985, a study in the U.S. identified 31 standardized motor assessment tests, ranking the BOTMP as the most widely used (90%) (Miles et al., 1988).

6. Test of Motor Impairment (TOMI) & Movement Assessment Battery for Children

(M-ABC) The TOMI (Stott, Moyes & Henderson, 1968) was designed to provide information about children's motor difficulties rather than their motor skills or age-appropriate abilities. The test was revised and standardized using a sample of nearly one thousand children aged 5-12 from England, Canada, and the U.S. The revised test, named the Test of Motor Impairment Henderson Revision (TOMI-H), included two key changes: simplifying the test structure reducing the number of tests, and creating a checklist to record the quality of skill performance during assessment. Later, another revision led to the development of the Movement Assessment Battery for Children (MABC) (Henderson & Sugden, 1992).

6.1. Purpose of M-ABC

The purpose of the M-ABC is to identify children with motor coordination difficulties. The battery includes the main test and the M-ABC checklist, designed to complement each other in assessing children aged 4-12. The test and checklist are used for screening, identification, and referral of children needing special programs (see Table 2, M-ABC-2 "Traffic Light System" questionnaire). Additionally, the M-ABC is available for research, and medical examination, and aids in planning interventions and evaluating programs, as both tools work together.

6.2. Structure and Scoring of M-ABC

The M-ABC test is a substantial revision of TOMI-H (Stott et al., 1984), differing in scoring criteria and task descriptions from the manual. Norms for children aged 4-12 allow documentation of normal motor performance (compared to 85% of peers), borderline performance (85-95%), and deviant performance (95-100%) for the lowest 5% of children. The battery consists of 8 tests grouped into 3 subtests. The test content varies by age, increasing in difficulty with age, and is organized into 4 age groups: 4-6, 7-8, 9-10, and 11-12 years. For example, the 4-6 age group includes tests such as:

Manual Dexterity:

1. Placing coins with preferred and non-preferred hand

2. Stringing beads (different numbers for each age)

3. Drawing lines within a path

Ball Skills:

1. Catching a bean bag with both hands

2. Rolling a tennis ball to a target

Dynamic and Static Balance:

1. Balancing on preferred and non-preferred foot

2. Jumping over a knee-height rope

3. Walking on tiptoes

Each test is scored between 0 and 5, with higher scores indicating poorer performance. The subtest scores are summed, and the total score ranges from 0 to 40. According to the manual, the testing time is 20-40 minutes.

6.3. M-ABC Checklist

The M-ABC checklist, derived from the Motor Competence Checklist (MCC) (Sugden & Sugden, 1991), includes information from the TOMI-H checklist (Stott et al., 1984). It is designed for children aged 5-11 and includes school-based tasks. It allows teachers and parents to assess children's motor behavior over a period. Standardized using a sample of 298 local children aged 6-10, the checklist is divided into 5 sections, with 12 tests in each. The initial 4 sections, based on Gentile's (1987) motor skill categorization, describe conditions:

a) stationary child-stable environment

b) moving child-stable environment

c) stationary child-changing environment

d) moving child-changing environment

The 48 tests of the 4 sections are scored on a 4-point scale (0 = very well, 3 = poorly) and interpreted in 3 levels. For children aged 6 and above, the total score is compared to the 5th and 15th percentiles, indicating problematic or borderline motor performance. The checklist identifies movement categories where the child struggles, providing specific information about challenging tasks. The 5th section addresses behavioral issues related to motor difficulties, scoring from 0 (rarely) to 2 (frequently), without summing the scores. The examiner rates overall behavior related to motor difficulties as high, medium, or low. The M-ABC test and checklist complement each other, though most research focuses on the test alone. By 1998, the M-ABC had not been used in the U.S., and it was not mentioned by Isaacs and Payne (1998) as one of the most used motor assessment tools. Similarly, the TOMI was never used in the U.S., and Miles et al. (1988) did not record it among the most popular tests.

7. Comparison Between BOTMP and M-ABC

Croce, Horvat, and McCarthy (2001) examined the concurrent validity of M-ABC by comparing the comprehensive and short forms of BOTMP with a sample of 106 children (67 boys and 39 girls) aged 5-12 years. They found Pearson coefficients of r = .77 for BOTMP-LF and r = .79 for BOTMP-SF, values that satisfied the researchers as the two batteries focus on different aspects (BOTMP on assessing motor performance and M-ABC on identifying impairment). Despite verifying the concurrent validity of M-ABC compared to BOTMP, caution is required because the scores of BOTMP-SF were derived from BOTMP-LF. Crawford and colleagues (2001) assessed the agreement between BOTMP, M-ABC, and the Developmental Coordination Disorder Questionnaire (DCDQ) (Wilson et al., 2000b) to identify children with motor problems in a sample of 202 children aged 8-17 years. They divided them into two groups based on their performance in BOTMP: a group characterized as clumsy (n = 201) and a group without motor problems (n = 201). They found a 67% agreement between the scores of BOTMP and M-ABC, lower than the percentage reported by Riggen and colleagues (1990) for TOMI. Specifically, the agreement among the three tests in identifying clumsy children resulted in one-third of children identified as clumsy by BOTMP not being recognized by M-ABC, while one-fourth of those identified as not clumsy by BOTMP could be identified as clumsy by M-ABC.

Significant agreement was observed between BOTMP and DCDQ in the percentage of nonclumsy children, but there was less agreement in the number of clumsy children. This is because BOTMP does not include an assessment of movement quality. The norms of a tool may not be valid in a country with a different culture and lifestyle from where the standardization sample was collected. Therefore, many studies have been conducted to verify the appropriate use of M-ABC in countries such as Singapore (Wright et al., 1994), Japan (Miyahara et al., 1998), Hong Kong

(Chow, Henderson, & Barnerr, 2001), and Taiwan (Chow et al., 2006).

Research from these studies concludes that modifications to the norms of the M-ABC battery are necessary due to cultural differences between these countries and the country of standardization, which revealed discrepancies in high scores. Studies on the appropriate application of the M-ABC battery have also been conducted in European countries such as the Netherlands (SmitsEngelsman et al., 1998) and Sweden (Rösblad & Gard, 1998), confirming the validity of the test norms.