Pedagogy Focused on Learning: Assessment of Error Processes

1. Introduction

Contemporary adolescence is characterized by complex processes involving intense biological and social changes in a context of profound transformations in the labor market and technologies [1]. In Brazil, secondary education appears disconnected from students' reality and needs [1]. This disconnection generates significant consequences for school dropout rates and learning quality. Approximately 25% of students who enter high school abandon this stage of Basic Education during the first year [1]. The learning level of Brazilian students remains low, as evidenced by the Programme for International Student Assessment (PISA), conducted by the Organization for Economic Co-operation and Development (OECD), which indicates that the majority of 15-year-old Brazilians demonstrate proficiency levels below the minimum expected in Mathematics, Science, and Reading domains, a situation that has remained stable since 2009 [2].

Contemporary education constitutes a social practice focused on the holistic development of students, the exercise of citizenship, and the development of fundamental skills and competencies for the modern world [3,4,5,6]. This objective, however, appears distant from the reality of adolescents in Brazilian high schools. The New High School curriculum, implemented in 2022, seeks to address this challenge through restructured teaching methods, positioning young people at the heart of school life and promoting quality education capable of stimulating deep learning, with emphasis on student protagonism and autonomy [3,7,8].

The difficulty in promoting teaching that substantially improves student learning persists historically, spanning educational public policies of various governments and different ideologies. The effective improvement of learning has been insufficiently addressed in multiple proposals for high school restructuring [3].

The school learning process constitutes a complex phenomenon that mobilizes issues at different educational and social levels and involves various agents [9,10,11]. Tools that directly impact student learning processes can significantly benefit high school restructuring proposals centered on learning [12,13,14].

The Approach-in-Process Test Version 2 presents special relevance in this context. It is a performance-based test that evaluates students' learning approaches when interacting with academic content. It is based on the theory of learning approaches, which identifies two fundamental approaches: deep and superficial [15,16,17,18,19]. The deep approach is characterized by student engagement and the use of strategies that allow meaningful interaction with content, resulting in higher quality learning. The superficial approach manifests through less student engagement and the use of strategies that produce limited interaction with content, resulting in low-quality learning [20,21,22,23,24].

The Approach-in-Process Test Version 2 contributes significantly to the evaluation of learning approaches by differentiating itself from traditional tests in the field, which are based on self-reporting [12,13,14]. This instrument represents an innovation by establishing a measure of approaches based on student performance. Its contribution goes beyond the field of measurement, offering teachers a methodological tool for evaluation and intervention grounded in the learning process of students [25].

The Test allows teachers to evaluate error processes that represent levels of superficial approach in learning school/academic content [12]. The identification of errors can contribute to a pedagogy focused on learning, enabling teachers to understand the specific problems of their students and implement strategic procedural interventions to promote higher quality learning [26,27,28,29]. Educational researchers recognize the importance of errors in teaching and learning processes, considering them a valuable didactic strategy [30,31,32,33,34]. Piaget highlights the fundamental role of errors in adaptation, evolution of thought, and subject autonomy [35]. Shulman argues that teachers need to understand not only what students learn correctly but also the types of errors they make in their activities [36]. Research conducted by Metcalfe in the USA demonstrated that errors made by students can promote new learning [37]. The identification and analysis of errors, without punitive intent, allows teachers to intervene directly in the learning process, facilitating the acquisition of new knowledge with higher quality.

This study aims to categorize the errors produced by students in their responses to the Approach-in-Process Test Version 2 on the content "Adolescence as a social construction" in the Life Project discipline of the New Brazilian High School curriculum. These categories represent distinct forms of superficial approach and allow teachers to build a pedagogy focused on learning.

2. Materials and Methods

2.1. Participants

The study included students formally enrolled in secondary education at a state public school, located in a socioeconomically disadvantaged urban area in inland Minas Gerais, Brazil. The institution demonstrates very low performance on the ENEM (National High School Exam), with an overall average of 474.71 [38], excluding the essay section. Teachers report that students show little engagement in studies, low commitment in classes, indiscipline, and learning difficulties.

Students attended morning classes. The sample consisted of three 1st-year classes (N = 49) and one 2nd-year class (N = 22), totaling 71 students. Of these, 40 were female (56.3%) and 31 were male (43.6%). The average age was 15 years (SD = 0.88 years), ranging from 14 to 18 years.

2.2. Instruments

Approach-in-Process Test Version 2

The test is a performance-based tool that evaluates students' learning approaches while engaging with academic content [12]. Developed in 2022 by Cristiano Mauro Assis Gomes and Marina Nogueira dos Santos Rodrigues [12], the test assesses learning approaches via six behaviors that represent levels of deep approach [12,13,14,39].

The test consists of six questions, each evaluating a distinct deep approach behavior:

Question 1: "Describe in your own words, and with as much detail as possible, a concept about the subject taught";

Question 2: "Elaborate a concrete example that shows your understanding of a concept";

Question 3: "Create a scheme in which you clearly show how the fundamental concepts of the taught content relate to each other";

Question 4: "Seek more information and deepen your knowledge about the subject, whether by searching the internet, reading books, watching documentaries, etc";

Question 5: "Identify possible misunderstandings about the taught content";

Question 6: "Create a challenging exercise that stimulates reflection on a learned concept".

The questions follow a standardized structure, exemplified in Figure 1 by question 2. Each question begins with a sentence that specifies the content being evaluated, containing a space to be filled in by the teacher. Four items follow: the first asks the student to indicate whether they can perform the described behavior; the second, open-ended in nature, evaluates the deep approach based on the student's performance; the third evaluates how much the behavior was encouraged by the classes; and the fourth evaluates the frequency of the behavior in the student's daily school life [12,39].

Question 4 differs structurally: item 1 does not assess the student’s ability to perform the behavior, but directly asks whether the student conducted in-depth study on the evaluated content during class sessions. In other aspects, it remains identical to the other questions.

For correction of item 2, the Correction Guide developed by Cristiano Mauro Assis Gomes [39] is used. This should be completed by the teacher before applying the test, preferably under the supervision of a specialist. The Guide is structured in five sections: definition of the evaluated content, selection and naming of central concepts, contextualization, fundamental properties of the concepts, and reference answers. Detailed information about each section can be found in [39]. In the present study, the Guide was completed for the content "Adolescence as a social construction" by the first author under the supervision of a test specialist [40]. This content is part of the Life Project discipline of the New Brazilian High School curriculum.

The Test allows the evaluation of superficial approach levels through qualitative analysis of errors made in item 2. These errors can be categorized, enabling the identification of elements in the learning process that require pedagogical intervention [29,30,37,41].

The Test and the Correction Guide offer diverse pedagogical applications: reflection on teaching practices, reformulation of pedagogical activities, promotion of classroom instruction centered on the learning process, and encouragement of student self-assessment [12,13,39,40,42,43].

2.3. Data Collection Procedure

The study was conducted after obtaining approval from the school, subject teacher, students, and guardians, following ethical research principles and ensuring anonymity, addressing risks, and clarifying potential benefits. Only underage students who agreed to the Assent Form and whose guardians agreed to the Informed Consent Form participated, as well as adult students who agreed to the Informed Consent Form.

The Approach-in-Process Test Version 2 was group-administered in the first semester of 2023 by the class teacher and the first author of the study in 4 classes at the school, lasting 50 minutes. The teacher did not incorporate the behaviors evaluated by the test into his pedagogical practice, thus not exerting pedagogical influence on the teaching and learning of the content. The test functioned exclusively as a diagnostic instrument to evaluate the level of learning approach employed by students in acquiring the content "Adolescence as a Social Construction" in the Life Project discipline. Students received course credit for participation and could choose to respond partially or completely to the test.

2.4. Data Analysis

The data analysis developed in four stages:

a) Preliminary analysis: Four one-hour online meetings were held between the first author and the subject teacher. Student responses were evaluated jointly, analyzing both conceptual accuracy and alignment with the target behaviors evaluated by the test.

b) Correction review: Two two-hour online meetings were conducted between the first author and the test specialist to review the previous correction. Stages a and b identified and consolidated the errors made by students.

c) Error categorization: Four three-hour online meetings were held between the first author and the test specialist. Errors were analyzed and compared, creating categories that most accurately represented the error types. The categorization process developed recursively throughout the meetings.

d) Reliability assessment: The first author requested a judge with solid understanding of the content and categories to identify possible errors in 17 responses from 15 students and classify them according to the established categories. For reliability validation, a minimum criterion of 70% agreement between the judge’s and the first author’s classifications was established.

3. Results and Discussion

Students could submit the test to the administrator even if they answered only one item per question. Of the total, 46 students submitted the test with at least one question incompletely filled out, while 25 students submitted the test with complete filling (Table 1). Complete filling of a question is considered: (1) when the student selects the "yes" option in item 1, indicating ability to execute the deep approach behavior in the taught content, and answers items 2, 3, and 4; or (2) when selecting the "no" option in item 1, indicating inability to execute the behavior, and answers item 4. The questions with the highest frequency of incomplete filling were those involving content deepening behavior (question 4) and identification of possible misunderstandings (question 5). Question 1, which involves the behavior of writing a content concept in one's own words, presented the lowest frequency of incomplete filling (Table 1).

Students’ written responses to item 2 are crucial, as they allow the evaluation of student performance in learning approach behaviors and the categorization of errors. Fifty (50) students produced at least one written response, totaling 168 responses. Twenty-one (21) students did not produce any written response throughout the test (Table 2).

Written production was low, considering that the test has six questions and each question allows for a written response (item 2). With 71 participating students, the possible number of written responses would be 426. The 168 written responses produced (Table 2) represent 39.44% of the total potential. Performance was extremely low, with only 10 correct responses out of the 168 produced (Table 2). This result is significant considering that the evaluated content is limited and relatively accessible, indicating that students presented an extremely low level of deep approach to the content "Adolescence as a Social Construction." These results align with the school's very low performance on the ENEM and teachers' perception of low engagement, commitment, and learning difficulties among students. The test performance demonstrates congruence with the learning pattern recognized by the institution's educators.

Student responses in item 4 complement the analysis of their responses in item 2. Table 2 shows 281 responses to item 4 of the test. Of this total, 64 (22.78%) correspond to option 1 (never or rarely executing the deep approach behavior in habitual academic life), 184 (65.48%) to option 2 (depends on the occasion), and 33 (11.74%) to option 3 (frequently or always). The majority of students reported performing deep approach behaviors depending on the occasion. Relating the percentage of written production in item 2 and the responses in item 4, a certain congruence is observed. The production of 40% of possible written responses approximates the predominant self-assessment of executing behaviors depending on the occasion. The very low performance suggests, however, that even when occasionally executing the behaviors, students possibly do so inadequately, resulting in low-quality learning.

Question 1 presented the highest number of written responses, followed by questions 2, 3, and 6. Questions 4 and 5 recorded a significantly lower number of written responses. Students showed greater willingness to articulate content concepts in their own words (Question 1), but were less inclined to elaborate concrete examples (Question 2), form schemes (Question 3), and create challenging exercises (Question 6), and considerably less willingness to write responses that required content deepening (Question 4) and identification of possible errors (Question 5). The behaviors that generated less engagement were precisely those requiring deepening of content and of metacognition.

In the context of very low performance and intense superficial approach, the identification, analysis, and understanding of errors acquires fundamental importance. These errors, when well understood, can constitute pedagogical opportunities for student development and the strategic elaboration of teaching activities and dynamics. As Piaget [35] argues, errors, when well addressed, can contribute significantly to the development of thought, adaptation, and student autonomy.

3.1. Brief Presentation of the Content "Adolescence as a Social Construction"

For understanding the error categories elaborated in this article, the content "Adolescence as a Social Construction" is briefly presented.

The socio-historical approach constitutes the perspective that defines the content. It possesses a central concept, divided into two premises: a) Adolescence is a social construction and not a natural development process; b) Every social construction is determined by the socio-historical context. From these premises, we may concluded that: Adolescence is shaped exclusively by socio-historical and cultural factors.

Adolescence results from a social latency initiated in the industrial revolution, when young people were removed from the adult world and work [6,44]. Although this social latency constitutes adolescence, the socio-historical and cultural context of each region produces differentiated manifestations. Young people residing in large urban centers, for example, experience adolescence differently from those living in small towns.

3.2. Error Categories

The categorization showed excellent reliability, with 95.65% agreement between the researcher and the judge. Of the 17 responses from 15 students, 22 errors were identified and categorized in the same manner by both. The judge identified one additional error, representing the only disagreement.

Table 3 shows the categories, brief description, and frequency of errors in descending order. Despite classification into 11 categories, in terms of frequency, they can be allocated into 3 groups. The first group, formed by the first two categories in Table 3, represents 74.87% of the errors. The second group, composed of the third, fourth, and fifth categories, represents 19.23% of the errors. The third group, formed by the five remaining categories, represents 5.89% of the errors.

3.2.1. Group 1

Group 1 represents three-quarters of the errors committed by students, possessing central importance for pedagogical purposes. In planning a pedagogy focused on the student learning process, these errors would be prioritized. Activities and assessments could be redesigned focusing on these error processes.

The Judgment of Incapacity category in group 1 represents more than half of the errors committed (Table 3). This error consists of the student's self-assessment that they are not capable of performing the deep approach behavior in the taught content, selecting the "no" option of item 1 in a test question. It has significant pedagogical importance as it indicates that the student does not feel competent to risk a minimally adequate response.

The judgment of incapacity is less severe when it occurs in more complex deep approach behaviors, such as that required in question 6 (elaborate a challenging exercise). This behavior is relatively sophisticated, requiring adequate understanding of the content and the ability to formulate challenges creatively. However, the judgment of incapacity acquires greater pedagogical severity when it occurs in simpler behaviors, such as elaborating a concrete example (question 2) or writing a content concept in one's own words (question 1).

The frequency of judgment of incapacity proved similar regardless of the deep approach behavior demanded (Table 3). This uniformity suggests that students may have interacted poorly with the content, negatively impacting learning. This error indicates that these students probably operated through a strong superficial approach, signaling to the teacher the need to intervene first in fostering less superficial approaches that mobilize simpler behaviors, before addressing more sophisticated behaviors.

The Common Sense category represents almost a quarter of the errors committed (Table 3). It occurs in case of the student, when presenting or explaining a concept, not using properties of the taught concept, basing themselves only on personal conceptions, beliefs, or common sense elements related to the theme. The following example illustrates this type of error, in response to item 2 of question 1: "Adolescence being the main phase of development of a person where an adolescent develops their social circle and doing all a networking with other people." [Original in Portuguese: "A adolescência sendo a principal fase de desenvolvimento de uma pessoa onde um adolescente desenvolve seu ciclo social e fazendo todo um networkim com outras pessoas."]

The student does not present properties of the concept of adolescence as per the taught curriculum. In the socio-historical perspective addressed, adolescence is understood as a social construction, not as a natural development process, defined as a product of social latency arising in the industrial revolution [6,44], when young people were removed from the adult world and work, experienced differently according to sociocultural contexts. The response ignores these properties, defining adolescence as: (1) the main phase of development; (2) a moment of social circle development; (3) a moment to promote social work relations. The understanding is based exclusively on prior knowledge, without connection to the taught concept, constituting a pedagogically significant error.

This error occurs with greater frequency in questions 1, 2, 3, and 6, especially in question 1 (Table 3), which specifically demands the presentation of a concept in one's own words.

3.2.2. Group 2

Group 2 represents approximately one-sixth of the errors committed, a significantly lower frequency than that of group 1, without diminishing their pedagogical importance.

The category Concept by Topics without Explanation appeared only in the first three questions of the test, mainly in question 3. It is characterized by the presentation of the concept by topics or themes in a fragmented manner and without argumentation. An example of this category, in response to question 1: "Industrial revolution - social, historical factors influence social construction." [Original in Portuguese: "Revolução industrial - fatores sociais, históricos influenciam na construção social."] The student presents the concept of adolescence in topics, using only some elements without explaining them. This error possibly relates to lack of habit or insufficient development of the ability to present an argued response, compromising conceptual understanding.

The Lack of Understanding on How to Execute the Behavior category occurs when the student produces a response indicating absence of comprehension about executing the deep approach behavior. This error manifested predominantly in more complex behaviors, evaluated in questions 4, 5, and 6 of the test (Table 3). An example of this category, in response to item 2 of question 5 (identify possible misunderstandings): "The adolescent at some point in their life always makes mistakes." [Original in Portuguese: "O adolescente alguma hora em sua vida sempre erra."] The student did not understand that they should present a misunderstanding committed or possible in relation to the content.

The hypothesis that this error would result only from misunderstanding the instructions is not supported, as in this case the frequency would be similar across different questions, which does not occur (Table 3). The incorrect execution of the behavior probably results from its unfamiliarity or absence in the student's repertoire.

The Confuses Concrete Example with Abstract Example category is linked to question 2 of the test (Table 3). It occurs when the student presents a conceptual example instead of a concrete example. This error indicates difficulty in differentiating concrete from theoretical examples, compromising the transposition from theory to practice. An example of this category: "The concrete example for adolescence as a social construction is that it originated from the industrial revolution." [Original in Portuguese: "O exemplo concreto pela adolescência como construção social é que se originou da revolução industrial."] The student considers the historical origin of adolescence as a concrete situation, when in fact it constitutes an essential and abstract property of the concept.

This error may relate to lack of familiarity with the behavior of elaborating concrete examples about abstractions or to superficial understanding of concepts. Pedagogical interventions with diversified activities can help students develop the ability to distinguish abstract and concrete situations, highlighting their characteristics in different contexts.

3.2.3. Group 3

Group 3 consists of the five last error categories (Table 3), with significantly lower occurrence (7.69% of the total) than the previous groups.

The category Confuses Deepening with Mere Search for New Information is linked to question 4 of the test. In this error, the student considers that deepening knowledge is equivalent to searching for new information. Although the search for new information can contribute to deepening, there is no direct relationship between the two. An example of this category: "I am learning this in my course and have already done some research on the topic of adolescent as a social construction." [Original in Portuguese: "Eu estou aprendendo isso em meu curso e já fiz algumas pesquisas sobre o tema do adolescente como construção social."] This response does not demonstrate deepening, as the student mentions research conducted without presenting their previous conceptual understanding and the resulting deepening. This error suggests the need for pedagogical intervention to clarify the concept of content deepening and its implementation.

The Wrong Understanding of the Concept category showed occurrences only in questions 1, 2, and 6, with greater concentration in the last one, which demands a more complex deep approach behavior: elaboration of a challenging exercise with an answer. The error is characterized by misunderstanding properties of the concept. An example of this category, in response to question 6: "What factors differentiate adolescents today from adolescents prior to the Industrial Revolution? Answer: As we know, social, cultural, and historical factors strongly influence. But the development of technology and the new resources that young people have today differentiate them from the skills of the last century." [Original in Portuguese: "Quais fatores diferenciam os adolescentes com os dias de hoje dos adolescentes antecedentes a Revolução Industrial? Resposta: Como já sabemos, os fatores sociais, culturais e históricos influenciam fortemente. Mas o desenvolvimento da tecnologia e os novos recursos que os jovens possuem nos dias de hoje diferencia os das habilidades do século passado."]

The student does not understand that Industrial Revolution constitutes a temporal landmark that defines the beginning of adolescence. In the socio-historical perspective, before the Revolution there was no adolescence and, consequently, no adolescents. The error consists in assuming the existence of adolescents before and after the Industrial Revolution. This type of error indicates relatively better understanding of the content compared to errors like Common Sense, suggesting that error categories can indicate different levels of understanding and learning.

In the Reproduction of Information category, the student cannot explain the concept in their own words, reproducing teaching material or teacher presentations. This error is especially evident when the student is asked to express concepts in their own words. An example of this category, in response to question 1: "It is qualified as a social concept that generates repercussions on subjectivity and current development, with problems and conflicts." [Original in Portuguese: "É qualificado como um conceito social que gera repercussão na subjetividade e no desenvolvimento atual, com problemas e conflitos."] The student presents fragments from the textbook in a fragmented, decontextualized, and disarticulated manner. This response has significant pedagogical relevance, as beyond reproduction, it does so incorrectly, suggesting incomprehension of the meaning of the copied excerpts.

The category Confuses Difficulty with Misunderstanding is linked to question 5 of the test. The student reports difficulty with a concept or part of the content without identifying possible errors resulting from this difficulty. An example of this category: "I had difficulty understanding how the adolescent enters into the question of society, I only understood the situations of their lives." [Original in Portuguese: "Tive dificuldade para compreender como o adolescente entra na questão da sociedade, só compreendi a situações de suas vidas."] The student only mentions difficulty with part of the content without addressing errors. This response suggests possible lack of familiarity with the behavior of monitoring one's own errors or potential errors related to the content.

In the Incomplete Conceptual Definition category, the error occurs when the student correctly presents only some properties of the concept, omitting fundamental properties for its complete definition. This error may result from partial or fragmented understanding of the concept. Among the presented categories, it possibly indicates the best level of learning of superficial approaches. An example of this category: "Adolescence as a social construction - I learned that in adolescence each person (adolescent) is shaped according to their region, state, country, era. Each has their way of being, of playing, those from the capital are raised in one way and those born in the interior in another way." [Original in Portuguese: "A adolescência como construção social - Aprendi que na adolescência cada pessoa (adolescente) é moldada de acordo com sua região, estado, país, época. Cada um possui seu jeito de ser, de brincar, quem é da capital é criado de uma forma e quem é nascido no interior de outro jeito."]

4. Conclusions

The results of this study showed that secondary education students from a socioeconomically disadvantaged public school presented a very low deep approach to the content "Adolescence as a social construction" with only 2.35% of responses meeting the criteria for correctness. The categorization of errors into 10 types, particularly Judgment of Incapacity and Common Sense, reveals patterns of superficial interaction with the content. These results corroborate teachers' perception of disengagement and learning difficulties.

The Approach-in-Process Test Version 2 proved to be a valuable diagnostic tool by allowing systematic identification of cognitive and metacognitive gaps. This instrument provides support for targeted pedagogical interventions, such as activities for elaborating concrete examples or conceptual schemes, which can mitigate superficial approaches.

The limited sample size and single-discipline focus limit generalizations of the results. Despite this limitation, the findings contribute to the debate on high school reform, highlighting the need for strategies that integrate procedural assessment and adaptive teaching practices.

Future studies could explore the application of the Approach-in-Process Test Version 2 in different contexts, disciplines, and educational levels. Additionally, it will be relevant to investigate the effectiveness of interventions based on error analysis for the transition from superficial to deep approaches. Such advances are essential to aligning educational policies with the demands of a transforming society and the labor market, thus reducing learning disparities and dropout rates.