Effects of Instructive Feedback on the Acquisition of Multiple Skills: Replication and Extension

1. Introduction

In Applied Behavior Analysis (ABA) to atypical development, behavioral principles and procedures are frequently applied aiming at the teaching of verbal and non-verbal skills, which are commonly impaired in children diagnosed with autism spectrum disorder (ASD). One method to teach them is known as discrete trial teaching (DTT) [1]. DTT is commonly conducted in a structured environment involving one-to-one interactions between interventionist and learner. Skills training concerns up to five steps: 1) provision of a verbal instruction (discriminative stimulus) to the learner; 2) provision of a prompt/cue if needed; 3) response emission by the learner; 4) differential consequence to the learner’s response (either a correction procedure or reinforcer delivery upon an incorrect or correct response, respectively); 5) provision of an interval between teaching trials [2]. Although the effectiveness of DTT in establishing several repertoires in learners with ASD is scientifically proved, strategies to increase its efficiency may be relevant [3].

It has been discussed in the literature that a procedure called instructive feedback (IF) may improve the efficiency of DTT by establishing new repertoire beyond what is directly taught [1,3]. IF involves the definition of a secondary target during the teaching of a primary target. As an example, an interventionist provides differential reinforce-ment when a learner points to the picture of a dog from an array with several pictures and under control of the instruction “touch dog”. In addition to a reinforcer stimulus (e.g., praise), a new information (IF) (e.g., “the dog is an animal”) is delivered in the consequent portion of the contingency. As a result, in addition to the acquisition of what was directly taught (the primary target), the learner may demonstrate the emergence of one or more new skills/secondary targets (e.g., by saying “the dog is an animal” under control of the picture of dog; by pointing to the picture of dog under control of the verbal instruction “show me animal”).

In the literature on IF to ASD, research has demonstrated that teaching with IF may be more efficient than a condition in which this procedure is not used, since learners may acquire more skills (both primary and secondary targets) and in a shorter time [3,4,5,6,7]. Previous investigations have also been conducted with small groups of children with ASD (dyads or triads). In these cases, each learner had opportunities to acquire their own primary and secondary targets and those of their partners through observation, which was verified for some participants [8,9,10]. In addition, since Covid 19, some investigations have also evaluated the effects of ABA-based interventions through telehealth services using internet. Regarding IF, there were cases in which several participating children acquired both their primary and secondary targets [7,10,11]. Another recent research also demonstrated that IF may be important for the acquisition of a second language. Four children with ASD were taught to tact pictures of common items in their primary language. Secondary target consisted of tact responses in their secondary language. Two participants showed generalization across new stimuli exemplars regarding both primary and secondary targets [12].

Some research on IF has also investigated the effects of this procedure on the acquisition of multiple repertoires as secondary targets in children with ASD. In one case [13], simple listener responding maintenance training with IF (e.g., pointing to the picture of the State of Tennessee under control of the instruction “show me Tennessee”. The IF information in the consequent portion of the contingency was “Nashville is the name of the capital of that State”) was defined as primary target and four secondary targets were probed for emergence. The two participants demonstrated acquisition of all, or nearly all, new repertoires consisting of listener responding under control of a characteristic (e.g., selecting the picture of Tennessee under control of the question “what is the capital of Nashville?”); tact of a characteristic (e.g., saying “Nashville” in the presence of the picture of Tennessee and the question “what is the capital of this State?”); intraverbal (e.g., saying “Nashville” under the question “what is the capital of Tennessee?”); reverse intraverbal (e.g., saying “Tennessee” under the question “what State has Nashville as capital?”). The acquired targets were maintained for two weeks.

In a subsequent replication research [14], the teaching of simple tact repertoire with IF (e.g., the learner says “calculator” under control of its corresponding picture and question “what is this?”. The experimenter then says “a calculator has batteries” as IF information) was defined as primary target instead of simple listener responding from the previous study [13], but secondary targets were similar. In addition, unlike the previous investigation, the three participating children with ASD did not know the primary target before training. However, they had already been exposed to teaching repertoires with IF before the research. In the end, the participants obtained all primary and secondary targets, and they showed maintenance over a 2-month period.

In another study [15], the teaching of simple tact responding with IF (e.g., saying “São Paulo” in the presence of a picture portraying the Brazilian State of São Paulo. The IF information provided in the consequent portion of the contingency was “São Paulo is a state in the southeast region”) was also defined as primary target involving four children with ASD as participants. The following four new repertoires were defined as secondary targets: Listener responding according to category (e.g., pointing to the picture of São Paulo under the instruction “show me a state in the southeastern region”); tact according to category (e.g., saying “southeastern region” under control of the picture of São Paulo and question “which region is the state of São Paulo in?”); arbitrary visual pairing (e.g., pairing the figure of southeastern region to the picture of the state of São Paulo); intraverbal according to category (e.g., saying “São Paulo, Rio de Janeiro and Minas Gerais” under the instruction “tell me the names of states in the southeastern region”). As a result, primary targets were acquired by all participants, but IF produced the emergence of secondary targets for only two of them. For one of the children who did not demonstrate emergence through IF, the direct teaching of one of the secondary targets was defined and this resulted in the emergence of the remaining targets. In this study, maintenance was not measured for the two children with whom training with IF produced the emergence of secondary targets.

Overall, the literature showed that IF may have increased the efficiency of teaching repertoires via DTT for learners diagnosed with ASD because some of them acquired primary and secondary targets, in this case, without the need for direct instruction. The current study also aimed to investigate the effects of IF on the establishment of multiple targets, that is, four secondary targets. Previous research, whose training involved an already known repertoire as primary target with IF, may suggest that this facilitated acquisition of secondary targets [13]. In another case, although participants did not know the primary target before training, they already had previous experience with IF procedure, which may also have facilitated the emergence of secondary targets [14]. Conversely, later research, in which participants did not know the primary target before training and had no previous experience with IF, showed that only two out of four learners acquired both primary and secondary targets and maintenance was not assessed. These results warrant further investigation involving new learners unfamiliar with the primary target and the IF procedure. In addition, it is important that maintenance be assessed as well.

Therefore, the current research involved the following guiding questions:

1) Will the training effectively establish the primary target (simple tact) for the children with ASD involved?

2) Will the training of the primary target with IF efficiently produce the emergence of related multiple (four) secondary targets, without need for direct instruction, for the children with ASD unfamiliar with the primary target and the IF procedure?

3) If secondary targets successfully emerge after training the primary target with IF, will all targets maintain two weeks later?

2. Materials and Methods

2.1. Participants

Three white boys between 6 and 8 years old (P1, P2 and P3) from low-income families participated. All children, regardless of the research, received interventions based on Applied Behavior Analysis (ABA) once a week for 1 hour and 30 minutes in rooms of a Brazilian university-based laboratory. They were all enrolled in regular schools, but they did not have any curriculum adaptation program. Before the onset of the current study, the children already show the following skills, which represented an inclusion criterion: tact of objects, pictures and actions regarding an average of 200 stimuli; identification of at least 250 objects and pictures as listeners (selecting them in arrays upon their dictated names); over 70 fill in the blank vocal intraverbals and cases of answering questions of varying complexity; identity and arbitrary matching to sample involving visual stimuli at a generalized level; over 70 listener responses under control of instructions on function, feature and class to which common pictures and objects are related. If a given child did not demonstrate skills at a level similar to or below that described here, he/she would be excluded from the research.

2.2. Setting

Data collection with each child participant occurred in a room from a Brazilian university-laboratory. The room was equipped with a table and three chairs. Throughout the study conditions, a child sat on one chair and an experimenter, who performed procedures and collected data, sat on another chair facing the child. In one part of the sessions, considering the different research conditions, a second observer (research assistant) sat on the third chair to collect data along with the experimenter and subsequently establish a level of agreement between observers.

2.3. Materials

Throughout the research conditions involving baseline and intervention with primary target and probes with four secondary targets, the experimenter used laminated pictures measuring 7x10cm with images representing stimuli that form classes. For P1, P2 and P3, stimuli consisted of pictures portraying Brazilian soccer teams and pictures of Brazilian states to which they belong. More specifically, the pictures used represented the teams known as Figueirense, Avaí and Chapecoense (and related picture of the state of Santa Catarina) and the teams known as Paysandu, Remi and Castanhal (and related picture of the state of Pará). Figure 1 shows all pictures used.

2.4. Dependent Variable

For each participant, one of the dependent variables (DV) corresponded to the number of correct responses representing a primary target, that is, simple tact (e.g., saying “Figueirense” under control of the picture portraying the Figueirense team), across sessions of 12 trials. Other DVs were the number of correct responses representing the following secondary targets across sessions of 12 trials: Listening responding according to category (e.g., pointing to the picture of Figueirense following the instruction “show me a team from the state of Santa Catarina”); tact according to category (e.g., saying “Santa Catarina” under control of the picture of Figueirense and the question “what state is the Figueirense team from?”; intraverbal according to category (e.g., saying “Figueirense, Avaí and Chapecoense” under control of the instruction “name soccer teams from the state of Santa Catarina”); arbitrary visual matching to sample (e.g., pairing the picture of Santa Catarina to the picture of Figueirense team as model).

2.5. Independent Variable

The independent variable (IV) in the current research consisted of teaching the primary target (i.e., simple tact) with the provision of praise and the Instructive Feedback (IF) information in the consequent portion of the contingency across sessions with 12 trials (e.g., saying “Figueirense” under control of the picture of Figueirense team. Thereafter, the experimenter provided praise, showed the picture of Santa Catarina and delivered the IF information “Figueirense is a team from the state of Santa Catarina”). The effects of teaching the primary target with IF information were assessed on the acquisition of both the primary and related secondary targets defined as DVs. Table 1 shows examples of DV and IV for the three participants in the study.

2.6. Experimental Design

In this research, the effects of teaching the primary target with IF information delivered in the consequent portion of the contingency were measured on the acquisition of both the primary (simple tact) and secondary targets (listener responding according to category, tact according to category, arbitrary visual matching-to-sample and intraverbal according to category), which were not directly taught and were probed for emergence.

Experimental control was established via a type of single case research design, that is, concurrent multiple baseline design across participants (as recommended by Cooper et al., 2014). Regarding the primary target, correct responses were measured across sessions related to two conditions per participant, that is, baseline and intervention. Baseline began concurrently for P1, P2 and P3. After low performance was demonstrated with stability, the IV was first applied to P1 whereas baseline continued for the remaining participants. When performance improved for P1, the IV was applied to P2 and baseline continued for P3. Likewise, when performance improved for P2, then IV was applied to P3 until an improvement in performance was also demonstrated. During baseline and intervention conditions with the primary target, intermittent probe sessions were established to measure correct responses on secondary targets to which IF information provided was related. These probes were used to measure the possible emergence of these secondary repertoires after training the primary target with IF.

2.7. Procedures

A) Primary target baseline. Each baseline session involved the presentation of 12 trials to assess simple tact. In each trial, after the experimenter presented a picture and question (e.g., picture of Figueirense team and question “what is this?”), the participating child had up to 5s to emit a response (e.g., saying “Figueirense”). There were no differential consequences programmed for either correct or incorrect responses. A low and stable performance across sessions (three consecutive at least) represented the criterion to terminate baseline condition per participant.

B) Primary target training. As in baseline, sessions consisted of 12 trials. In each trial, the experimenter showed a picture and question, and the participant had up to 5s to respond (e.g., saying “Figueirense” in the presence of the picture of the Figueirense team and question “what is this?”). However, unlike baseline, differential consequences were programmed for correct and incorrect responses. After a correct response/simple tact by the participant, the experimenter delivered verbal praise, the picture of the state (e.g., picture portraying the state of Santa Catarina) of the previous labeled picture and IF information on category (e.g., “Figueirense is a team from the state of Santa Catarina”). In the event of an incorrect response, or failure to respond within the 5s period allowed for the trial, an error correction procedure would be applied. In this case, the experimenter provided the response (e.g., say “Figueirense”) so the participant could repeat it. The criterion to terminate the training condition per participant consisted of at least three consecutive sessions without errors.

C) Primary target maintenance. As in baseline, sessions consisted of 12 trials. In each trial, the experimenter showed a picture and question, and the participant had up to 5s to respond (e.g., saying “Figueirense” in the presence of the picture of the Figueirense team and question “what is this?”). As in baseline, no differential consequences were programmed for correct and incorrect responses. This condition involved three sessions per participant.

D) Secondary target probes. Intermittent probe sessions to assess the emission of secondary targets were intermittently conducted throughout all phases aimed at measuring the primary target/simple tact (i.e., baseline, primary target training with IF, maintenance). Regarding each case of secondary target probe session, each participant had up to 5s to emit a response after an instruction was provided by the experimenter in each of 12 trials. No differential consequences were programmed for correct and incorrect responses. In the case of listener responding according to category, an array of pictures portraying soccer teams and instruction on category were provided, so the participant had the opportunity to point to the corresponding picture from the array (e.g., pointing to the picture of Figueirense after “show me a team from the state of Santa Catarina”).

In the case of tact according to category, a picture and question related to category were provided. Then, the participant had the opportunity to state the category to which the picture belongs (e.g., saying “Santa Catarina” under the control of the picture of Figueirense and question “what state is the Figueirense team from?”). Secondary target of arbitrary visual matching to sample involved the provision of an array of pictures portraying states and another picture representing a soccer team as model. Thereafter, the participant had to point to the picture in the array that corresponded to the model (e.g., pairing the picture of the state of Santa Catarina to the picture of Figueirense team). Finally, the case of intraverbal according to category involved the provision of a verbal instruction in which the experimenter asked the participant to name soccer teams from a given state. The participant, then, had the opportunity to name soccer teams (e.g., saying “Figueirense, Avaí and Chapecoense” under the control of “name soccer teams from the state of Santa Catarina”).

E) Social validity assessment. After data collection with the participating children was finished, their parents answered a Likert scale questionnaire containing the following questions: 1) I believe that the procedures from the study helped my child to communicate more; 2) I consider the skills that my child developed in the research important; 3) I believe that the procedures used in the study may be important to teach new skills that will benefit my child in everyday interactions; 4) I wish that the procedure be used with my child outside the context of research; 5) I recommend the procedure used with my child for new children with atypical development. Each question involved alternatives ranging from “1”, meaning “I completely disagree”, to “5”, meaning “I completely agree”.

2.8. Interobserver Agreement (IOA)

In part of the research, a second observer/research assistant, in addition to the experimenter, also collected data on participants’ performance across conditions involving primary and secondary targets to establish levels of interobserver agreement (IOA). On each trial within sessions, an agreement was verified when both observers recorded the same type of response, whether correct or incorrect. For all participants IOA, regarding the primary target, was determined in approximately 20% of the sessions (four, five and six sessions for P1, P2 and P3, respectively). And, regarding the secondary targets, IOA was determined in 50% of the sessions (four), 55% (five) and 75% (six) for P1, P2 and P3, respectively. For all participants, IOA level corresponded to 100%.

2.9. Ethical Procedures

This study is derived from a project, which was evaluated and approved by the ethics committee in research with humans from Federal University of Maranhão (authorization 6.585.992), Brazil, São Luís-MA. The participants and their parents signed informed consent forms. Identities were kept confidential. Participation in the research could be terminated at any time without any harm.

3. Results

In this research, data for P1, P2 and P3 are presented in the following order: 1) the participants’ performance on the primary target (simple tact repertoire) across baseline, intervention and maintenance sessions; 2) the participants’ performance on the secondary targets (listener responding according to category, tact according to category; arbitrary visual matching-to-sample; intraverbal according to category) across probe sessions before and after the teaching of the primary target. Figure 2 presents data on the number of correct responses regarding the primary target across sessions.

Overall, all participants demonstrated a low and stable performance throughout most of baseline sessions. Although P3 showed few correct responses during the three initial sessions, performance subsequently dropped to zero and remained unchanged and stable as in the cases of P1 and P2. The training condition followed the rationale of the multiple baseline design and was established for the participants in the following order: P1, P2 and P3. As the teaching of simple tact with IF was programmed for each participant, change in performance, compared to baseline, was almost immediate and noticeable for every participant. Although correct responses were established with some variability across sessions, all participants exhibited a stable errorless performance across at least the last three consecutive sessions before training was discontinued. A stable and errorless performance was also demonstrated during maintenance condition two weeks after the end of intervention. Figure 3 shows the participants’ performance in intermittent probes regarding secondary targets before and after primary target training.

During the intermittent probes of secondary targets before training the primary target (corresponding to baseline condition for this repertoire), the three participants, overall, exhibited a low performance across the different targets. P1, across three sessions (from 1 to 3), solely demonstrated on average one and three correct responses regarding listener responding according to category and arbitrary visual matching-to-sample, respectively. Likewise, P2 demonstrated two and four correct responses on average regarding the same secondary targets in four sessions (1, 8, 9 and 10). P3, in turn, only showed one correct response involving listener responding according to category in one out of four sessions (1, 15, 16 and 17). In addition to what has been stated, there were no further correct responses on any secondary targets for all participants. During primary target teaching with IF, intermittent probe sessions were conducted to assess secondary targets as well (four for P1 and P2 and three for P3). For P1 and P2, no errors were committed during the last three probe sessions (11, 12 and 13 for P1 and 20, 21 and 22 for P2). And P3 also did not exhibited errors during his three probe sessions (25, 26 and 27). Finally, two weeks after primary target training was discontinued, a final condition was administered to assess maintenance of the primary target. In addition, probe sessions were also conducted to assess secondary targets. During these sessions, there were also no errors by the participants.

Finally, regarding the social validity questionnaire, the parents/guardians of the children in the study rated each question highly by selecting the option 5 (“I completely agree”). This outcome indicates that the research procedures and findings were acknowledged as important by those responsible for the children with ASD involved in the research.

4. Discussion

Returning to the guiding questions of this research, it may be considered that they were addressed. Training with instructive feedback successfully established the primary target (simple tact) for P1, P2 and P3. Overall, they all have shown a stable performance across many sessions without correct responses in baseline. Training was established for the participants at different times, following the logic of the multiple baseline design. Each participant demonstrated an increasing trend in performance as the independent variable was applied. Data suggests the establishment of experimental control. A maintenance assessment two weeks later showed that the effects of training were long-lasting. In addition, primary target training with IF appears to have influenced the acquisition of four secondary targets that did not undergo direct instruction. All participants showed a low performance on these new targets across three consecutive probe sessions before primary target training with IF. During this training, as the primary target was acquired, additional probe sessions showed significant improvements on all secondary targets across three consecutive sessions. Errorless performance on these targets was also demonstrated two weeks later in a single session. The research procedures and results were recognized as important by the parents/guardians of the children involved. Data from this research appears to be consistent with those of participants in previous studies [13,14,15].

In previous research involving the teaching of a simple repertoire with IF and assessment of emergence of related secondary targets, several children with ASD demonstrated acquisition of all or nearly all targets. However, as was said before, in one case two children were submitted to additional training involving simple listener responding already known by them. This was a way to ensure that the children were exposed to IF information, and this influenced emergence and maintenance of secondary targets [13]. However, it is possible that prior knowledge about the primary target facilitated the subsequent acquisition of secondary targets. In a subsequent replication study, simple tact repertoire was the primary target taught with IF to three children with ASD [14]. Although all secondary targets emerged and were maintained for them, they had previous experience learning skills via IF regardless of the study. So, it is possible that this variable influenced the efficacy and efficiency of the procedure. Finally, another research extended this investigation involving four children with ASD unfamiliar with IF before the onset of data collection. Two children showed emergence of all secondary targets, but maintenance was not assessed [15].

Assuming that the IF procedure may be efficient for the acquisition of multiple secondary targets for some learners without direct teaching, the current study sought to extend the investigation with new children without previous experience with IF. All participants in this study demonstrated acquisition of the primary target through training and all secondary targets by emergence. This provided additional evidence of the effectiveness and efficiency of the procedure with children with ASD unfamiliar with IF. Unlike the previous research, [15], maintenance assessment was conducted and correct performance was demonstrated across all targets two weeks after training of the primary target ended.

In the investigation with the primary target, the stability seen in baseline data with low performance across sessions, and gains during intervention including no errors across at least the last three sessions per participant, contributed to control possible threats to internal validity (e.g., history, maturation, testing effects) [16]. Brazilian soccer teams and states of origin were the stimuli used in the research. Brazil is a country in which soccer is culturally highly valued by the population. Many people are frequently talking about it, and soccer is also a recurring topic on television and social media. In this sense, it may be assumed that history could have influenced the participants’ performance. However, in this study, the selected teams and states are not related to the region in the country the children involved are from. People in this context usually do not show interest in these teams and their states, and do not talk about them. The research team did not specifically disclose specific information about the stimuli used to the children’s caregivers until the end of the study. In addition, data collection took approximately 2 months, which was important to control a maturation effect. The use of multiple baseline design with different baseline durations among the children, and demonstration of change in the primary target solely through the intervention for each child, helped to avoid the threat from testing effects. The establishment of experimental control of training with IF over the acquisition of the primary target also increased its potential influence on the emergence of the four related untaught secondary targets across several probe sessions. Although this process was intermittent, to avoid lengthy testing sessions that could cause fatigue and reduce cooperation, each secondary target was assessed across at least three sessions before and after teaching of the primary target, facilitating analysis of level, trend, and variability of secondary targets. The marked differences in performance on secondary targets between the probing sessions, before and after teaching the primary target, suggest a significant influence of the IF variable on the emergence of secondary targets by all participating children.

Overall, this study sought to follow quality criteria established by the What Works Clearinghouse (WWC) [17] for single case research designs, especially considering the case of multiple baseline across participants. Among the WWC standards, there should be at least three data points per condition/phase with little or no variability. This allows analysis of level, trend and variability through visual inspection of data, across the research conditions. The establishment of the IV should be staggered, that is, carried out at different times for each participant involved. In this sense, the results of the current study, based on teaching the primary target with IF and use of a multiple baseline design, suggest control of potential threats to internal validity such as history, maturation, and testing effects. Still regarding the case of multiple baseline of interest, WWC advises that it should involve a minimum of three participants to allow multiple replications and that, for each participant, DV should change solely when IV is introduced to allow demonstration of experimental control. In addition, IOA should also be established across at least 20% of research sessions to ensure greater reliability of the data. The current study was able to address all criteria pointed out so far. However, limitations must be presented.

Although certain measures are not mandatory for WWC in establishing the methodological quality of a single case study, their importance is recognized. The assessment of generalization, for instance, is recommended for the purpose of external validity. In this study, generalization measures were not obtained. Future investigations should address this issue by assessing primary and secondary targets across new stimuli (e.g., multiple exemplars of pictures), context (e.g., natural environment in addition to a laboratory room) and people (e.g., a new communication partner in addition to the experimenter). This investigation also did not obtain measures of procedural fidelity. Further studies on IF, and multiple secondary targets involving children with ASD with no previous experience with IF, may assess procedural fidelity to ensure that research procedures are implemented appropriately. Furthermore, this measure may increase confidence in the effects of IV on DV.

5. Conclusions

In this research, teaching a primary target (simple tact) with IF successfully established this repertoire for three children with ASD defined as participants. In addition, four related secondary targets (listener responding according to category; tact according to category; arbitrary visual matching-to-sample; intraverbal according to category) seem to have emerged due to exposure to IF information during the teaching of the primary target. The multiple baseline design used, and its results reinforce confidence in the effects of IV on DV. All participating children did not have previous experience with IF before data collection, and the research results provide additional evidence of efficacy and efficiency of IF in establishing primary and secondary targets, as was demonstrated with some children in previous investigation. Furthermore, the results were long-lasting, as maintenance was demonstrated 2 weeks later. The research sought to follow internal validity parameters for procedures established by WWC. Limitations of the investigation referred to generalization and procedural fidelity data which were not obtained and, although not mandatory according to WWC, they may be addressed in future studies.