Progress in Language Abilities Related to Reading Comprehension – Domain Specific Intervention on Executive Functioning

1. Introduction

The ability to process information from a text is an important educational output of primary education. A reduced ability to work with information contained in a text is usually registered in low-achieving students. One important internal factor influencing reading comprehension concerns the child’s executive functioning (Hannon & Frias, 2012) [2]; (Gaskins, Satlow, & Presley, 2007) [3]; (Samuels, 2013) [4]; (Follmer, 2018) [5]; (Butterfuss & Kendeou, 2018) [6]; (Spencer, & Cutting, 2021) [7]. For example, executive functions play a crucial role in integrating visual and linguistic information, as well as in automatic recall of linguistic skills during reading (Altemeier, Abbott, & Berninger, 2008) [8]; (Jabłoński, 2013) [9]. Moreover, it was suggested that the relationship between executive functioning and reading comprehension may be bi-directional (Follmer, 2018) [5]; (Meixner, Warner, Lensing, Schiefele, & Elsner, 2019) [10], with gains in academic functioning predicting gains in executive functioning, and vice versa (Fuhs et al., 2014) [11]. In line with these findings, the primary objective of this paper is to evaluate the usefulness of a cognitive stimulation program, ExeFun-READ¹. The program is explicitly based on the assumed bi-directional relationship between executive functioning and language abilities in low-performing pupils².

2. The Current Study

2.1. Theoretical framework

2.1.1. Executive functioning and reading comprehension

Executive functions are often defined as a set of interrelated functions that enable purposeful, goal-directed behaviour (Diamond, A., 2013) [12], and are seen as crucial for successful academic achievement (Best et al., 2011) [13]. Generally, it is assumed that there are three lower-order executive functions, working memory, cognitive flexibility, and inhibition, which are the basis for the higher-order, and more complex executive functions such as attentional control, and planning (Diamond, A., 2013) [12].

With regard to reading comprehension, specifically, it was found that both lower and higher-order executive functions have an impact on different reading comprehension processes. Specifically, working memory, a lower-order executive function, was found to influence comprehension and integrating knowledge (Cain, Oakhill, & Bryant, 2004) [14]; (Daneman & Carpenter) [15]; (Hannon & Frias, 2012) [2]; (Just & Carpenter, 1980) [16]. Cognitive flexibility, another lower-order function, was found to play a role for students with reading difficulties (Cartwright, et al., 2017) [17]; (Cartwright, Marshall, Huemer, & Payne, 2019) [18].

In addition, higher order functions, such as attentional control (Samuels, 2013) [4], and planning are crucial in decoding and reading comprehension, and to how readers plan, direct, select, and organize available cognitive structures and processes necessary for comprehension (Gaskins, Satlow, & Presley, 2007) [3]. Moreover, metacognition and self-regulation are important in monitoring and controlling comprehension (Larkin, 2010) [19].

2.1.2. Intervention studies on executive functioning related to reading comprehension

The potential of executive functioning in academic achievement becomes even more salient in the light of recent studies indicating that training can improve executive functioning (Diamond, A. 2012; Cragg & Gilmore, 2014; Diamond & Lee, 2011; Posner, Rothbart, & Tang, 2013; Rueda, Checa, & Combita, 2012) [20,21,22,23,24]. Often, these interventions are grouped as domain-general or domain-specific interventions (Otero, Barker, & Naglieri, 2014) [25]. Several studies indicated that interventions aimed at strengthening children’s executive functioning not only result in better executive functioning, but also transfer to their reading comprehension skills (Cartwright et al., 2020; Jacob & Parkinson, 2015) [26,27]. Similarly, another study (Carretti & Cornoldi, 2014) [28] proved that reading comprehension program for 4^th and 5^th graders focusing specifically on metacognition could foster text comprehension.

In the light of findings that training executive functions can lead to improvements in reading comprehension, the current study will utilize the following procedure. It will employ a pretest-training-posttest design with an experimental condition, an active control and a passive control condition. These will investigate whether the ExeFun-READ intervention had a positive effect on children’s executive functioning and scholastic performance in the reading domain. The first research question considered the effect of the ExeFun-READ program on children’s executive functioning. It was hypothesized that children who received the ExeFun-READ intervention would show more improvement in their executive functions, specifically inhibition, cognitive flexibility, self-regulation, and attentional control, than the children in the active and passive control conditions (Diamond, A., 2012; Cragg & Gilmore, 2014; Diamond & Lee, 2011; Diamond, Barnett, Thomas, & Munro, 2007) [20,21,22,29].

The second research question investigated the effect of the ExeFun-READ program on children’s scholastic performance in the reading domain. In line with previous studies (Cartwright, et al., 2017) [17]; (Carretti & Cornoldi, 2014; Horowitz-Kraus, 2016) [28,30], it was expected that the children who received the ExeFun-READ program would show more improvement in their reading abilities, specified in this study as language abilities related to reading comprehension (semantic, knowledge syntactic knowledge, verbal fluency, verbal analogies) than those in the two control conditions.

The third research question examined the effect of the ExeFun-READ program on the relationship between children’s executive functioning, reading abilities and school performance (specifically Slovak language). Considering that the ExeFun-READ program was assumed to improve children’s executive functions and reading abilities (Horowitz-Kraus, 2016) [30]; (Cragg & Gilmore, 2014; Diamond, A., 2012; Diamond, Barnett, Thomas, 7 Munro, 2007; Diamond & Lee, 2011 [20,21,22,29], it was expected that the relationship between executive functioning and reading abilities, on the one hand, and scholastic performance, on the other hand, would become weaker for children who received the ExeFun-READ program, but not for those in the two control conditions.

3. Materials and Methods

3.1. Participants

The participants were low-performing students from mainstream elementary schools in Slovakia, attending Grade 4. Students were selected if, according to their teachers, they had achieved below average results on Slovak language tests (specifically marks 3 – good, 4 – sufficient, and 5 – insufficient) in the two school years prior to the intervention. Seven elementary schools were involved in the project. 374 pupils took part in the pre-tests. Out of the total number of 374, 151 pupils were selected and involved in the intervention study. A mixture of equalization and random selection was used. The selection criteria were their equal scores in pre-test measures in executive functioning. To measure the level of executive functioning D-KEFS battery was used as further specified in 2.1.3. The Reading test was another tool for the equalization of the research sample (Matějček, 1992) [31]. The test provided information on reading speed, error rate, number and quality of errors, level of comprehension of the read text as well as overall manifestations in reading. Based on the results of measuring the level of executive functioning and reading skills, 151 children were divided into groups of three. In each group there were pupils with approximately the same level of measured indicators. Pupils from the trio were randomly allocated to one of the three conditions, while taking into account their gender, and performance in mathematics and Slovak language. The average age of participants was M = 10.82 years (SD = 0.651).

The division of participants can be found in Table 1.

3.1.2. Design and procedure

The study utilized a pretest-intervention-posttest experimental design with three conditions: experimental group (group 1), active control group (group 2), and passive control group (group 3).

Children’s executive functioning and language abilities related to reading comprehension were tested twice - before the intervention (pretest) and two weeks after the intervention to assess short-term transfer of the experimental effect (posttest). There was a 3.5-month interval between the pretest and posttest. The tests were clinically administered: individually and during school hours and took approximately 90 minutes. The pretest and posttest were administered by ten trained data collectors (school psychologists).

Group 1 was administered the original domain-specific intervention program ExeFun-READ. The intervention consisted of 30 units, and each unit took 45 minutes. The intervention was conducted in the school during school time and was administered twice per week. Trained university students studying for a Master’s degree in teacher training acted as administrators. Group 2 was given 30 extra lessons of stimulation in the language domain and reading comprehension education in addition to the compulsory school curriculum. The teachers, employed at the participating school, worked with a regular Slovak language textbook. Children in group 2 did not receive specific stimulation of executive functioning. Group 3 did not perform any additional tasks.

3.1.3. Materials

Executive functioning (EF). To assess the children’s level of EF, the Delis–Kaplan Executive Function System (Delis, Kaplan, & Kramer, 2001) [32] (D-KEFS) test battery was used. The D-KEFS test battery was adapted for use with the Slovak population. The test’s psychometric characteristics were described by (Ferjenčík, Bobáková, Kovalčíková, Ropovik, & Slavkovská, 2014) [33]. It was found that the internal consistency was below α =0.70³ in the individual indicators of all tests, which from a psychometric point of view tends to be considered as the lower limit of "good" reliability. In this study, the following subtests from the D-KEFS battery were used: (1) D-KEFS Trail Making Test – a test of attention organization and flexibility in five test conditions, capable of abstracting interference factors of visual searching and motor speed; (2) D-KEFS Verbal Fluency, which measures the ability to fluently generate verbal responses to letter prompts, categories and categories switching within 60 seconds; (3) D-KEFS Design Fluency Test, a test of figural fluency (in three control conditions) in the visual domain; (4) D-KEFS Color–Word Interference Test, a version of the Stroop test in four test conditions that measures the ability to inhibit “learnt” behavioural responses.

Language abilities related to reading comprehension. To assess the construct, the following instruments were used: 1. Cognitive Abilities Test (CogAT) (Thorndike & Hagan, 1986) [1] – Verbal battery. The reliability of the Cognitive Abilities Test Verbal battery was found to be good α =0.85 (Thorndike & Hagan, 1997) [34]. Following subtests of CogAT were used: (1) Vocabulary - the subtest reflects the range of vocabulary, especially the passive vocabulary and the flexibility of speech use; (2) Completion of sentences which indicates speech-logical thinking and general knowledge, as to properly manage the task it is necessary to define and understand the meaning of the presented concepts; (3) Classification of terms which evaluates the ability of abstraction, conceptualization as well as verbal fluency. It reflects the scope and accuracy of the logical arrangement of verbal knowledge; (4) Verbal analogies which reflects the level of understanding of the relationships between verbal concepts and categories, logical reasoning in a verbal context and combinational abilities (Thorndike & Hagan, 1997) [34].

ExeFun-READ stimulation program in the linguistic domain. The ExeFun-READ program is domain specific. Reading and linguistic/language material is a curricular area in which cognitive stimulation occurs. The program is designed to address the relations between domain-specific versus thinking-skill-oriented (domain-general) instruction. The relationship is addressed through Target (executive functions, self-regulation, metacognition, thinking skills) and Content (linguistic/language material – Words, Sentence, Paragraphs, Text) stimulation. A child’s active learning is focused on semantic, phonetic, morphological and syntactical comprehension of linguistic material. The program has been structured in such a way to enable the administrator, during an individual interaction with a pupil, to both work with a text selected from the school curriculum and, simultaneously, mediate executive, cognitive, and metacognitive processes in the pupil in order to improve their language abilities, as manifested in comprehension and memorization of the text. In the intervention, while the pupil gathered information, the administrator–student interaction was focused on with the aim of stimulating the student’s attention control and precise visual perception (Partanen, Jansson, Lisspers & Sundin,2015) [35]. The administrators typically asked questions such as: What do you see in front of you? What do we call these items, things? What is this task about? What do you have to do? How are you going to do this task? While solving tasks during the intervention, metacognitive self-regulation was targeted at cognitive planning, strategic thinking, inhibitory control and questions such as the following were asked: What do you need to do to solve the problem? What approach should you use? What is the strategy? What do you have to avoid doing?

The basic criterion for the domain-specific design of intervention units was the multi-level nature of text comprehension. The domain-specific content was divided into modules. It contains 5 different, consecutive modules: Word, Sentence, Paragraph, Text, Self-regulation. Each module consisted of a set of graded tasks. The criteria for the grading and hierarchical organization of the tasks were based on the level of cognitive difficulty (e.g., working memory load, level of abstraction). The modules are further divided into sections (detailed description in Table 2).

As part of the Exefun-READ program one trained administrator worked with two pupils. The program is designed for individual and pair stimulation, which included (1) focusing on the pupil´s ability to generate questions, (2) clarifying and summarizing the information they have read, (3) moving from being passive observers of learning to active teachers, (4) becoming involved in the learning experience as peer tutors. The “think aloud” method was encouraged frequently. In terms of time allocation, the full program covers approximately 60 teaching hours of work with students, at the recommended frequency of 2-3 times a week. It represents material enabling professional tutoring of the pupil throughout the school year. An abbreviated version of the program - the essence of the program was verified in the intervention. This pilot/experimental version consisted of 30 units lasting 45 minutes in a pair stimulation. The full version of the program in the form of methodological material for teachers is described elsewhere (Kovalčíková et al., 2021) [36].

The program was administered by student-teachers who were supervised by the members of the research team. Training of administrators consisted of 30 hours. It was organized in several recursive cycles while reflecting on the intervention issues and unexpected problems that arose during intervention. Members of the research team performed regular supervision of the administration process at schools. Observation of the intervention was the basis of supervision. Three main aspects were considered: (1) the administrator’s activities with the pupil, process of stimulation, (2) the stimulation material and the pupil’s reaction to it, and (3) the course of individual interventions in the current conditions of the school. Field supervision enabled formulating immediate conclusions and recommendations for the further course of the intervention, as well as modifications of stimulation units. The results of the supervision were analyzed at subsequent administrator training sessions.

3. Results

3.1. Effects of training on executive function

Initially, the effects of the intervention on children’s executive functioning were investigated through a repeated measures MANOVA. In this analysis, time (pretest, posttest) was included as a within-subjects factor, and condition (experimental, control 1, control 2) as a between-subjects factor. Dependent variables in this analysis were TMT motor speed time, letter fluency, category fluency, switching fluency, design fluency, and Stroop interference. Results are presented in Table 3. Basic statistics for the different variables are provided in Table 4. The multivariate Time effect was significant (λ = .37, F(6,138) = 38.61, p < .001, η_p² = .63), indicating that, in general, children showed progress on the measurements from pretest to posttest. Additionally, the effect of Time x Condition was significant (λ = .85, F(12,276) = 1.91, p = .033, η_p² = .08), indicating, in combination with examination of the mean scores, that children in the experimental group showed larger progress than children in the two control groups.

Table 4. Basic statistics for scores on all executive function measures at pre- and post-test.

Further inspection of the univariate effects revealed that children show progress from pretest to posttest on all measures. However, a significant group difference was only found in switching fluency (F(2,143) = 6.42, p = .002, η_p² = .08). This is also reflected in the group means, as Group 1 (∆M = 2.05) showed larger progress over time, when compared to Group 2 (∆M =.06) and Group 3 (∆M =.04).

3.2. Effects of training on language abilities related to reading comprehension.

In order to investigate the effects of the intervention on language abilities related to reading comprehension, a second repeated measures MANOVA was conducted. In this analysis, time (pretest, posttest) was included as a within-subjects factor, and condition (experimental, control 1, control 2) as a between-subjects factor. Dependent variables in this analysis were vocabulary, completion of sentences, classification of terms, and verbal analogies. Results are presented in Table 5. Basic statistics for the different variables are provided in Table 4. The multivariate Time effect was significant (λ = .68, F(4,138) = 16.11, p < .001, η_p² = .32), indicating that, in general, children showed progress on the measurements from pretest to posttest. Additionally, the effect of Time x Condition was significant (λ = .81, F(8,276) = 3.94, p < .001, η_p² = .10), indicating, upon examining the mean scores, that children in the experimental group showed larger progress than children in the two control groups.

Further inspection of the univariate effects revealed that children show progress from pretest to posttest on all measures. Additionally, a difference in progress over time between conditions is found for vocabulary (F(2,141) = 11.32, p < .001, η_p² = .14), completion of sentences (F(2,141) = 6.82, p = .001, η_p² = .09), and classification of terms (F(2,141) = 4.64, p = .011, η_p² = .06). The group means t again reflect these differences for vocabulary (Group 1 ∆M = 4.00; Group 2 ∆M = .20; Group 3 ∆M = -.31), completion of sentences (Group 1 ∆M = 3.96; Group 2 ∆M = 1.18; Group 3 ∆M = .91), and classification of terms (Group 1 ∆M = 2.27; Group 2 ∆M = .68; Group 3 ∆M = -.46).

3.3. Effects of training on relationship with Slovak language school results.

Correlations were used to test the relationship between children’s executive functions and their Slovak language school results, and changes in this relationship as a result of training. TMT motor speed, letter fluency, category fluency, switching fluency, and Stroop interference were used as measures of executive functions. Posttest measures for executive functions were split by condition, to test whether different patterns of relationships emerged as a result of training. The results are displayed in Table 6. On pretest, only TMT motor speed (r(148) = -.19, p = .022), switching fluency (r(148) = .28, p = .001), and Stroop interference (r(148) = -.19, p = .021) were significantly related to school results in reading. On posttest, significant correlations were only found for Group 2. Here, school results seemed to relate to TMT motor speed (r(49) = -.39, p = .004), category fluency (r(49) = .33, p = .017), switching fluency (r(49) = .35, p = .013), and design fluency (r(49) = .31, p = .027).

Additionally, correlations were used to test the relationship between children’s language abilities related to reading comprehension and their Slovak language school results, and changes in this relationship as a result of training. Vocabulary, completion of sentences, classification of terms, and verbal analogies were used in the correlation analysis. Again, posttest measures were split by condition, to test whether different patterns of relationships emerged as a result of training. The results are displayed in Table 6. On pretest, strong relationships were found between school results and vocabulary (r(147) = .43, p <.001), completion of sentences (r(147) = .54, p < .001), classification of terms (r(147) = .49, p < .001), and verbal analogies (r(147) = .50, p <.001). These relationships were also found in all three groups at posttest for vocabulary and completion of sentences. However, the relationship between school results and classification of terms was only significant at posttest for Group 3 (r(43) = .47, p = .001). Further, the relationship between school results and verbal analogies was only significant at posttest for Group 1 (r(47) = .30, p = .036) and Group 3 (r(43) = .39, p = .009).

4. Discussion

The ability to process information from a text is an important predictor of school performance at higher levels of schooling (Wharton-McDonald & Erickson (2017) [37]. For children that do not benefit sufficiently from in-class reading instruction, additional individual tutoring of these abilities may be necessary. The current study investigated whether the ExeFun-READ intervention had a positive effect on children’s executive functioning and language abilities related to reading performance. The intervention was based on previous research indicating a possible bi-directional relationship between executive functioning and reading abilities (Follmer, 2018; Meixner, Warner, Lensing, Schiefele, & Elsner, 2019) [5,10] with gains in academic functioning predicting gains in executive functioning, and vice versa (Fuhs et al., 2014) [11].

4.1. Executive Functioning

The results of the current study showed positive effects of the ExeFun-READ intervention on children’s executive functioning. Closer inspection, however, demonstrated that significant group differences were found only in switching fluency. As expected, the experimental group showed larger progress over time, compared to the active control and passive control groups. The intervention employed tasks closely related to switching fluency, such as sorting, grouping and categorizing letters, and switching between categories based on phonetic, morphological, and semantic characteristics of words. For example, the principles of cross-classification of words were used (see Klauer & Phye, 2008) [38]. Practicing these skills appears to have led to increased performance on switching fluency.

However, no significant group differences in improvement were found on the other measures of executive functioning, despite the focus of the ExeFun-READ program on further developing executive functions using linguistic materials. This finding could perhaps be explained by the domain-specific focus of the intervention, which may have led to understimulation of the executive functions not related to switching fluency, or a lack of transfer from practicing executive functions applied to the specific reading tasks to other more general tasks. This finding was not in line with Follmer’s (2018) [5] study, which indicated there may be a bi-directional relationship between executive functioning and reading comprehension. In the current study, the intervention led to improved language abilities related to reading comprehension, but in terms of executive functioning the improvement only extended to switching fluency. An additional explanation can be found in the research results of Dolean, Lervag, Visu-Petra, Melby-Lervag (2021) [39]. They found that only language skills could independently predict development of reading comprehension; executive functions did not have a significant direct effect on the development of reading comprehension in early readers beyond fluent decoding and oral language skills in languages with transparent orthography. These results also suggest that once children learn to decode well, their language skills (and not their executive functions) have a strong effect on the development of reading comprehension.

4.2. Language abilities related to reading comprehension

Regarding the effects of the ExeFun-READ intervention on children’s reading abilities, specified as language abilities related to reading comprehension, significant improvements were found in vocabulary, completion of sentences, and classification of terms in the group of children that received the ExeFun-READ intervention compared to children in the active and passive control groups. The intervention appeared effective in improving children’s abilities in these different domains. In line with the findings of Partanen, Jansson, Lisspers, and Sundin (2015) [35], children’s vocabulary skills may have improved due to the intervention requiring verbalization of the processes involved in solving tasks. In turn, this may have enhanced practicing their active and passive vocabulary to describe objects (e.g., pictures), states (e.g., feelings associated with success while solving tasks), define strategies, and clarify and summarize the information they have read.

Improvements in completion of sentences will likely have resulted from the systematic focus on formulating coherent sentences within the intervention. In the intervention, sentences were viewed as the structural units of paragraphs and texts. In line with prior research on the effectiveness of metacognitive language stimulation programs (Meltzer, 2011; Zanartu, 2015) [40,41], strategies for practicing analyzing texts, searching for keywords, and paraphrasing sentences and paragraphs of the educational text were included in the intervention as they were expected to lead to improved reading comprehension, and may have been central to children’s improvement in sentence completion. These results are in line with research of Zhao, and Guo (2021) [42] that showed the contribution of syntactic knowledge to reading comprehension.

The intervention also led to more improvement in classification of terms. Based on the principles described in the Concept Teaching Model (Hansen & Morgan, 2009) [43], students were supported in making the transition from the concrete meaning of words to abstraction. The intervention included guided practice in categorizing word groups according to selected criteria of superordinate concepts and, vice versa, in concretization (e.g. when decoding the meanings of unknown words from the context).

However, the ExeFun-READ intervention did not lead to improvement in all measured indices of verbal behavior related to reading comprehension. No differences in improvement were found between the conditions on verbal analogies. This might in part be due to the nature of verbal analogies, which in addition to requiring language and reading abilities, are also dependent on inductive reasoning skills. Inductive reasoning or, more specifically, analogical reasoning, is a skill that is closely related to general cognitive abilities, and generally seen as a robust indication of general intelligence. It is closely related to more everyday skills such as problem-solving and transfer and requires abstraction of novel information that is relevant in a specific context and applying it in another (Goswami, 2012; Richland & Simms, 2015) [44,45]. The skills measured here may have been broader than just reading ability and may therefore have gone beyond the scope of the intervention.

4.3. Relations with school performance

In terms of relationships between school results, executive functions, and reading abilities, some unexpected results were found. The intervention was based on research of Follmer (2018) [5]; Meixner, Warner, Lensing, Schiefele, & Elsner (2019) [10] and Fuhs et al. (2014) [11] detailing a close relationship between executive functioning and reading abilities. This assumption was not reflected in the relationships between school results and executive functions, as, except for the moderate relationship between switching fluency and reading, no or weak relationships were found between executive functions and school results in reading. This could either indicate that executive functions are not as closely related to performance in reading as expected, or that the executive functioning measures utilized lacked sensitivity. However, taken together with the other results, this might also indicate the domain-specific nature of executive functions, as described above. Moreover, it was found that relationships between executive functions and school results were no longer found for posttest scores of both the experimental group and the passive control group and seemed to have strengthened for the active control group. A similar effect was seen in previous research with an ExeFun intervention in the mathematics domain, in which the active control group showed a similar pattern of relations with math subskills on the posttest as the general pattern portrayed on the pretest, while both the experimental group and the passive control group showed far weaker relations (Kovalčíková et al., 2021) [36].

4.4. Limitations

In addition to those mentioned above, some additional limitations were found in the current study. Implementation fidelity was not measured or monitored, which may have caused variability in implementations. The intervention was implemented within the school environment. On the one hand, this speaks for strong ecological validity. However, this also led to limited time being devoted to implementation of the intervention, through constraints in the school time and the time teachers could devote to the intervention. As the intervention was implemented after classes, it may not always have been implemented in the ideal circumstances and students and parents may not always have been in the position to give it sufficient priority.

5. Conclusions

The current research found bidirectional relationships between reading ability and switching fluency, similar to those found in previous studies (Follmer, 2018; Meixner, Warner, Lensing, Schiefele, & Elsner, 2019; Fuhs et al., 2014) [5,10,11], but not in other executive functions. Further research should focus on this relationship and the practical value it offers for shaping reading interventions. In terms of effects on language abilities related to reading comprehension, the ExeFun-READ intervention showed promising results: Improvements were found as a result of the intervention in relation to vocabulary, completion of sentences, and classification of terms, which all constitute abilities manifested in reading comprehension. Although more research is necessary, implementation of this intervention may help improve children’s reading abilities and consequently improve their educational opportunities.