Practical Work in Natural Sciences Education: Development and Validation of a Qualitative Data Collection Instrument

1. Introduction

Practical work (PW) represents a central pillar in natural sciences education, playing a key role in promoting scientific literacy and in developing students’ cognitive and procedural skills (Abrahams & Reiss, 2012). This pedagogical approach, often associated with hands-on and minds-on methodologies, fosters active and meaningful learning experiences, enabling the integration of theory and practice and supporting the construction of contextualised scientific knowledge (Harrison, 2016; Wei et al., 2019).

However, despite the consensus on the pedagogical value of PW, its effective implementation in schools faces significant constraints related to limited resources, time, teacher training, and curricular alignment (Akuma & Callaghan, 2019; Ramnarain & de Beer, 2013). As noted by Ferreira and Morais (2014), the conceptual and operational complexity of PW requires clear articulation among its conceptual, procedural, and evaluative dimensions; otherwise, it risks being reduced to a merely technical activity, lacking scientific and pedagogical intentionality.

In this context, the need for valid and reliable instruments to collect data on teachers’ perceptions and practices regarding PW becomes evident. Research in the didactics of science has emphasised that the development and validation of such instruments allow for a deeper understanding of how teachers interpret, implement, and reflect on PW within their professional practice (Abrahams et al., 2014; Hartman & Squires, 2024).

This study addresses that gap by developing and validating a comprehensive interview guide designed to collect qualitative data on PW in natural sciences education, specifically within the third cycle of Portuguese basic education. Grounded in a systematic literature review (SLR) (Oliveira & Bonito, 2023) and an analysis of the national curricular guidelines (DGE, 2018a, 2018b, 2018c; Martins et al, 2017), the instrument aims to explore teachers’ perceptions regarding the conceptual, evaluative, operational, and curricular dimensions of PW, promoting a holistic understanding of scientific practice in educational contexts.

The focus of this study is the characterisation of practical work in natural sciences Education in the 3rd cycle of basic education within the Portuguese education system. In addition, the study aims to contribute to this characterisation through the development of the qualitative data collection instrument, illustrated and characterised in this manuscript. The overarching research question that guided all procedures in this investigation was the following: How do natural sciences teachers in the 3rd cycle of basic education perceive and implement practical work in their teaching practice?

In the reference list of the main body of this scientific article, we include the references of the authors cited in the text, as well as those underpinning the domains, subdomains, objectives, questions, criteria, and indicators of the interview protocol, whose full version can be consulted in Supplementary File S1 – Qualitative Data Collection Instrument. These authors correspond to the corpus of 53 manuscripts selected through the previously conducted SLR. Additionally, legal regulations from the legislation of the Portuguese Republic, as well as the national curriculum guidelines for the subject of natural sciences in the 3rd cycle of basic education, were also consulted and incorporated into the reference list.

2. Methods

2.1. Systematic Literature Review on Practical Work

The first step in this process was conducting an SLR on the state of the art of practical work in science education. This review was carried out in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement guidelines (Page et al., 2021) and conducted across five distinct databases: B-on, ERIC, Google Scholar, Scopus, and Web of Science. Table 1 outlines the objective of the SLR, its research question, the keywords used in the search equations, and the inclusion/exclusion criteria applied to the selected databases.

Table 2 identifies the databases, query options, query criteria, and document count that resulted in the initial set of 163 potentially relevant manuscripts, which were subsequently subjected to a screening process.

In the initial phase of the SLR, 163 potentially relevant studies were identified. Following a screening process - which included the application of inclusion and exclusion criteria, removal of duplicates, and elimination of out-of-scope studies - a final corpus of 53 relevant studies was established for document analysis. Figure 1 presents the flow diagram illustrating the processes of identification, screening, and inclusion that led to the selection of these 53 manuscripts for documentary analysis.

Through the graphical illustration in Figure 2, a simplified overview of the process followed throughout the various stages of the SLR on practical work in science education can be established.

This SLR highlighted four major dimensions of the dynamics of practical work in science education: 1) the conceptual dimension; 2) the advantages dimension; 3) the evaluation dimension; and 4) the disadvantages dimension. Figure 3 illustrates the categories of practical work dimensions under analysis.

2.2. Design of the Interview Guide

By triangulating the data obtained from the SLR with the curricular guidelines for natural sciences in the third cycle of basic education and relevant legal frameworks, it was possible to structure the first version of the interview guide (including its objectives, questions, criteria, and indicators). In addition to the four dimensions considered in the SLR, the analysis of legal regulations and curriculum guidelines enabled the inclusion of three further relevant dimensions in the first version of the interview guide: the operationalisation dimension, the textbook dimension, and the curricular dimension.

More specifically, the intention to conduct an interview-based survey with teachers of the natural sciences subject in the third cycle of basic education led to the development of a comprehensive, semi-structured interview guide, grounded in the Kaufmann (2004) approach, which places particular emphasis on the context in which actions occur, and the meanings constructed within it. With the aim of analysing Biology and Geology teachers’ perceptions regarding the PW carried out in the natural sciences subject at this educational level within the Portuguese education system, this instrument was developed and validated to enable an interpretative phenomenological analysis (IPA) (Hartman & Squires (2024). This approach is intended to make it possible to interpret detailed, first-person accounts of the PW promoted in the natural sciences curriculum, thereby unveiling its underlying meanings. Reflecting on the value of IPA, we would acknowledge that its layered analytical structure—encompassing the exploration of content, contextual dimensions, and linguistic features such as metaphor—would offer a vivid and meaningful portrayal of participants’ lived experiences. By comparing these elements within and across cases, the approach would allow for a deep and intimate understanding of the phenomena under study. Our engagement with IPA would reveal the immersive nature of the methodology, providing a comprehensive framework that could assist researchers in grasping the complexities embedded in participants’ accounts. Moreover, IPA would prompt researchers to question and bracket their own taken-for-granted assumptions, enabling participants’ underlying experiences to surface more clearly. This process would not only enhance the analytical depth but also contribute to a more nuanced and authentic representation of participants’ perspectives.

The interview protocol developed herein was designed with the intention of conducting interviews with 23 natural sciences teachers within the Portuguese education system. For the study in which this qualitative data collection instrument will be applied, one natural sciences teacher from the 3rd cycle of basic education will be selected from a school or school cluster in each district of mainland Portugal, totalling 18 teachers. Regarding the archipelago of the Autonomous Region of the Azores, one teacher will be selected from each island group (Eastern, Central, and Western), based on convenience sampling, resulting in a total of three teachers. For the archipelago of the Autonomous Region of Madeira, two teachers will be selected, also through convenience sampling, currently teaching on Porto Santo Island and Madeira Island, respectively.

As previously demonstrated, the construction of the instrument was based on an SLR, concerning PW in science education at the pre-university level (UNESCO, 2012), which enabled the identification of the state of the art regarding the international adoption of this methodology. In a second phase, an analysis was conducted of the Portuguese curricular guidelines for the teaching of natural sciences in the third cycle of basic education, with the aim of gathering elements that would facilitate an adaptation as closely aligned as possible with the scope of this educational level. To this end, the Student Profile at the End of Compulsory Schooling framework (Martins et al., 2017) and the Essential Learnings defined for the natural sciences subject in the third cycle were examined (DGE, 2018a, 2018b, 2018c). Additionally, legal frameworks within Portuguese legislation were analysed for their potential impact on the dynamics of PW in science education (Portugal, 2018a, 2018b, 2018c), with a view to triangulating this information with the previously identified documents.

Based on the triangulation of information from the SLR, the curricular guidelines, and the legal frameworks, the questions that comprise the interview guide were defined. These questions are intrinsically linked to a set of criteria and indicators designed to serve as a guide for the interviewer, helping to steer the interview toward relevant topics and to more effectively capture the meanings attributed by natural sciences teachers to the phenomena under investigation. These sets of criteria and indicators will also serve, subsequently, as facilitators for the content analysis of the participants’ discourse, particularly within the topics embedded in each dimension and subdimension of the semi-structured interview. Furthermore, to support the processes previously described, a set of clearly defined objectives was established for each of the outlined dimensions and subdimensions, clarifying the scope and purpose of the questions addressed in each area.

Subsequently, the first version of the instrument underwent a review process supported by five experts from four Portuguese public universities. Once validated, the guide was tested in a pilot interview with a natural sciences teacher from the Lisbon district, leading to its final version. A synthesis of this entire process is illustrated in Figure 3, which depicts the key stages and the sequence of steps followed.

3. Results

The SLR made it possible to identify four major dimensions associated with the dynamics of practical work in science education: 1) the conceptual dimension; 2) the advantages dimension; 3) the evaluative dimension; and 4) the disadvantages dimension. Figure 4 provides a synthesis of these results.

These four dimensions identified through the SLR were subsequently incorporated into the structure of the qualitative data collection instrument, to which the operationalisation dimension, the textbook dimension, and the curricular dimension were added. These additional dimensions emerged from the triangulation of data with relevant legal frameworks in Portuguese legislation and with the natural sciences curriculum guidelines for the 3rd cycle of basic education, as described in the previous section.

The overall structure of the first version of the interview guide is presented in Table 3, illustrating the distribution of the defined questions (items) across the respective dimensions and subdimensions under study.

An analysis of the overall structure of the first version of the interview guide reveals that it constituted an instrument comprising seven dimensions, which were further subdivided into nineteen specific subdimensions, encompassing a total of thirty-eight items.

Following the development of this initial version of the interview script, contact was established with five experts from four different Portuguese public universities (University of Aveiro, University of Lisbon, University of Minho, and University of Porto), with the aim of obtaining their contributions towards the validation of this qualitative data collection instrument. Accordingly, and in order of response to the invitation, the five experts are coded as presented in Table 4, which also includes the number of optimisation suggestions proposed by each expert.

The experts’ recommendations primarily revolved around the following aspects: (a) Paying attention to the potentially ambitious length of the interview script and the effects of fatigue during extended interviews; (b) Simplifying the concepts of (multi)interdisciplinarity and transdisciplinarity, instead asking whether the teacher “usually conducts project work with colleagues from other disciplines, and in what ways?”; (c) Balancing the number of items/questions addressing both the advantages and disadvantages of project work; (d) Avoiding overly goal-oriented or directive questions, thereby allowing space for the interviewees’ reasoning and discourse; (e) Requesting concrete examples or descriptions of everyday teaching practices.

After incorporating the experts’ suggestions, the interview script was then validated and adopted its final structure, after a pilot interview, being structured as illustrated in Table 5.

The final version of the interview script adopted a more simplified structure, due to a reduction in the number of subdimensions. Nevertheless, six additional items were included overall. These details can be more thoroughly examined in the comparative table between the initial and final versions of the interview script (Table 6).

An analysis of the previous table reveals that both versions contain an identical number of dimensions. However, the initial version is structured into 19 subdimensions, whereas the final version comprises 14 subdimensions. Regarding the number of items, the initial version includes 38 questions, while the final version comprises 44 questions.

To provide a more comprehensive and detailed overview of the structure of this qualitative data collection instrument, Table 7 presents a fragment of the specific structure of its final version, focusing on the first dimension under analysis (conceptual dimension). Including the full qualitative data collection instrument in the main manuscript would exceed the recommended length for this article. To provide a concise yet informative overview of its structure, only an illustrative excerpt is presented. The complete instrument is available in Supplementary File S1 – Qualitative Data Collection Instrument.

In general, the specific structure of the qualitative data collection instrument comprises seven distinct columns. From left to right, these include the identification of the dimension under analysis, followed by its corresponding subdimension. Next is the column outlining the objectives, followed by the column listing the items/questions. The criteria column is also included, which is particularly useful during the interview process, as well as the indicators column, which plays a key role during the content analysis phase—both of which are highly valuable in these two critical stages of the interview-based inquiry. Finally, the last column presents the authors and/or legal frameworks that underpin the preceding content.

Following the validation process by the panel of reviewers, a pilot interview was conducted on May 26, 2023, with a natural sciences teacher from the 3rd cycle of basic education, who has been teaching at the same school in the Lisbon district for 27 years. During the semi-structured interview, which lasted 85 minutes, the script proved to be a robust instrument, enabling an intuitive and comprehensive collection of qualitative data. Additionally, the instrument also demonstrated its effectiveness in facilitating the content analysis process of the collected data, owing to the clear definition of its objectives, criteria, indicators, authors, and legal frameworks - elements supported by an SLR on the state of the art of PW in science education. For the reasons outlined above, this interview script is considered suitable for holistically analysing natural sciences teachers’ perceptions of PW in their daily teaching practice, whether individually or in an integrated manner.

Finally, and as previously mentioned, the complete and final version of this qualitative data collection instrument is included as a supplementary file to the main body of this manuscript (see Supplementary File S1 – Qualitative Data Collection Instrument).

4. Discussion and Conclusions

The development of this qualitative data collection instrument aims to contribute to the characterisation of the dynamics of practical work across seven distinct dimensions, all of which are incorporated into the interview guide, whose construction and validation process is presented in this manuscript. Designed to interview natural sciences teachers in the 3rd cycle of basic education, the instrument is intended to support the in-depth collection of teachers’ accounts, thereby contributing to the grounding of policies, curriculum guidelines, the optimisation of school administration, and the promotion of collaborative practices that enhance the effectiveness and sustainability of practical work in natural sciences education with young students.

This study does not seek to generalise data concerning the acceptability and consistency of the interviews. Instead, it focuses on describing the design and validation process of the qualitative data collection instrument, carried out through expert review and pilot testing. The validation process involved assessment by five experts from four Portuguese public universities, which ensures the survey’s credibility and scientific rigour. None of the experts involved in reviewing this qualitative instrument is affiliated with the authors’ university or engaged in joint projects, thereby eliminating any potential conflict of interest. At this stage, the data are used to confirm the instrument’s clarity, reliability, and internal consistency, rather than to draw conclusions that can be extrapolated to broader populations. The methodological emphasis lies in validating and consolidating the tool, not in generalisation. Future research may include broader and more quantitative validation of this instrument.

For a holistic discussion of the results, it is also highly relevant to establish a general characterisation of the qualitative data collection instrument, with particular emphasis on its potential. Accordingly, the following considerations can be made: a) It is a robust instrument that enables the collection of data across seven dimensions of practical work dynamics in the natural sciences subject of Lower Secondary Education. b) It subsequently facilitates the process of content analysis, supported by its clearly defined criteria and indicators. c) It contributes to the analysis of teachers’ perceptions regarding their individual pedagogical practice and may be particularly useful in processes of self-assessment and peer evaluation. d) It also supports the analysis of teachers’ perceptions of their pedagogical practice in interaction with colleagues and the broader educational community, offering valuable insights into the dynamics of practical work in natural sciences education.