A Multimodal Neurophysiological Approach to Evaluate Educational Contents in terms of Cognitive Processes and Engagement

1. Introduction

The quest to understand and optimize the learning process is a cornerstone of educational research. Unraveling the cognitive mechanisms underlying knowledge acquisition not only enhances theoretical understanding but also paves the way for more effective, personalized and inclusive teaching methodologies. By integrating insights from neuroscience, psychology, and educational technology, researchers can develop personalized learning strategies that cater to diverse learner needs, ultimately improving educational outcomes. While traditional assessment methods like tests and questionnaires provide valuable information, they offer a limited window into the dynamic, internal cognitive and affective states that shape learning. To gain a deeper understanding, researchers are increasingly turning to neurophysiological measures, capturing the subtle physiological changes that accompany relevant cognitive processes of the learner such as mental effort, attention, engagement, and distraction. This approach, often termed educational neuroscience, promises to provide objective, real-time insights into the learner's experience, moving beyond subjective self-reports and behavioral observations.

The application of neurophysiological signals-based approaches, such as the ones based on electroencephalography (EEG), electrodermal activity (EDA), and photoplethysmography (PPG), in real-world educational settings is gaining a constant interest among the scientific community [1,2]. Unlike fMRI or other neuroimaging techniques that require constrained laboratory environments, EEG, EDA, and PPG are relatively portable and less sensitive to movement artifacts, making them suitable for studying learning in more naturalistic contexts. This is crucial for capturing the complexities of real-world learning experiences, which often involve dynamic interactions and a variety of stimuli. Researchers have begun to use these tools to investigate learning in classrooms, online environments, and even during outdoor activities [3,4,5,6]. The existing literature demonstrates the potential of these neurophysiological signals to reveal subtle differences in cognitive and affective states during various learning tasks. Studies have shown that EEG can differentiate between levels of mental workload during problem-solving [7,8,9], identify periods of inattention during lectures [10,11,12], and even predict learning outcomes based on neural activity patterns [13,14,15]. EDA and PPG have been used to assess stress levels during exams, monitor engagement during interactive simulations [13,16], and detect emotional responses to different types of learning materials [17,18,19,20].

However, a comprehensive investigation of how different types of learning materials, commonly used in real-world educational settings, impact cognitive responses from a neurophysiological point of view, remain an area to be explored [21,22,23,24]. Specifically, the comparative effects of visually rich educational videos, interactive training videos designed to promote skill acquisition, and traditional text-based learning materials on students' cognitive and affective states need to be systematically examined using a unified neurophysiological framework. Many existing studies focus on a single modality (e.g., only EEG) or compare very dissimilar learning activities, making it difficult to draw direct comparisons. This study aims to address these gaps by investigating the neurophysiological impact of three common learning contents: an educational learning video with advanced graphic solutions and a simpler communicative language style, a more traditional academic content based on a PowerPoint presentation with a voice-over, and classical text reading. The overarching objective is to objectively characterize the distinct neurophysiological reactions associated with each learning content, using a multimodal neurophysiological approach and, more importantly, to provide objective insights about which one could be the most engaging and efficient learning environment. By simultaneously recording brain activity, electrodermal and cardiovascular responses, it was aimed to capture the interplay between cognitive workload, attentional focus, and emotional engagement elicited by each type of material.

2. Materials and Methods

2.1. Participants

Ten volunteers from Sapienza University, including Master thesis students, Ph.D. candidates, and staff members, ranging in age from 24 to 37 years (M = 28.6, SD = 4.56), were involved in the presented study. They participated in the study without any reward. The experimental task consisted in using the three selected educational contents, related to the same topic, maintaining focus and striving to absorb as much information as possible, as they were informed of a subsequent questionnaire. Before starting, each participant received a clear explanation of the study's procedures and provided written informed consent. While the overall procedure was explained, the specific focus on comparing cognitive engagement across different material types was not explicitly highlighted to avoid influencing their natural viewing behavior. Following the observation period, a comprehensive debriefing session clarified the study's full objectives. Permission to use any visual recordings from the session was also secured. This research was conducted in accordance with the Declaration of Helsinki (1975, revised in 2008) and received ethical approval from the Sapienza University of Rome ethics committee (protocol code 2024/03-002 and 21/03/2024).

2.2. Experimental Protocol

This study presented each participant with three distinct educational contents concerning a single topic: Bluetooth wireless technology. This topic was chosen for its broad relevance in order to avoid any bias due to individual interests. Limiting the study to a single topic was deemed appropriate for this initial investigation. The three educational contents were:

• Task “Educational video”: A popular science video (6 minutes 49 seconds) incorporating short video clips, infographics, and practical examples (Educational video link).
• Task “Academic video”: A traditional lecture-style video (7 minutes 17 seconds) featuring PowerPoint slides and a voice-over narration (Academic video link).
• Task “Text reading”: An encyclopedic text excerpt, designed to be read within approximately 7 minutes (Text reading link).

Participants performed each task individually, seated comfortably in front of a computer monitor with audio delivered via external speakers (Figure 1). The order of presentation of the three contents was randomized across participants to mitigate potential order effects, such as habituation or expectation. Neurophysiological data were recorded continuously from each participant during each task. Prior to the start of each experimental session, a 60-second resting-state baseline was recorded for each participant at their workstation.

2.3. Neurophysiological Data Collection and Processing

2.3.1. Electroencephalography (EEG)

During the experimental tasks, participants' cerebral activity (EEG) was captured using the Mindtooth Touch EEG wearable system (Brain Products GmbH, Gilching, Germany & BrainSigns srl, Rome, Italy, [25]). The device embeds eight recording EEG channels, placed over the prefrontal and parietal regions, specifically at the AFz, AF3, AF4, AF7, AF8, Pz, P3, and P4 locations, in accordance with the 10-10 International System, reference and ground specifically on the right and left mastoids, and sample rate of 125Hz.

Initially, a pre-processing phase was carried out to identify and correct both physiological and non-physiological artifacts unrelated to the cerebral activity of interest, such as ocular, muscular, and movement-induced signals. In this regard, the EEG signal was band-pass filtered with a 5th-order Butterworth filter in the interval 2–30 (Hz). The eye blink artefacts were detected and corrected by employing the o-CLEAN method [26]. For further sources of artefacts, such as the ones deriving by muscular activity and movements, ad-hoc algorithms based on the EEGLAB toolbox [27] were applied. More specifically, one statistical criterion was applied to the 1-second-long pre-processed EEG signal. Firstly, EEG epochs with the signal amplitude exceeding ±80 μV were marked as “artefacts’’. Secondly, such EEG epochs marked as “artefacts” were removed from the EEG dataset in order to proceed for the EEG processing by considering exclusively the cleaned channels.

Once the EEG pre-processing steps were completed, the Global Field Power (GFP) was calculated for the EEG frequency band of interest for computing the mental states on which the present study focused. Therefore, the EEG GFP features were computed within the Theta, Alpha, and Beta frequency bands. It has to be underlined that the GFP was chosen as the parameter of interest describing brain EEG activity since it has the advantage of representing, within the time domain, the degree of synchronization on a specific cortical region of interest in a specific frequency band [28], [29]. In terms of technical implementation, the GFP was mathematically computed according to the approach described by Vecchiato and colleagues [30]. Concerning the EEG GFP features computation, the frequency bands were defined according to the Individual Alpha Frequency (IAF) value [31] computed for each participant. In order to compute the IAF, a 60-seconds-long experimental condition was collected while the participants kept their eyes closed, since the Alpha peak is consistently prominent in such a condition. Subsequently, the EEG GFP was computed over all the EEG channels for each 1-second-long epoch through a Hanning window of the same length (1 s, which means 1 Hz of frequency resolution according to the time resolution required from the presented approach) as follows:

$${GFP}_{band,region}={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{x}_{i,band}^2\left(t\right),$$

where N is the number of the considered EEG channels and $x_{i,band}^2$ is the i-th EEG channel filtered within the selected EEG frequency band. After the EEG data preprocessing, the EEG GFP-derived features were computed according to the research objectives. More specifically, the mental workload, attention, and cognitive engagement were computed according to the following:

$$Mentalworkload\mathrm{}={\displaystyle \frac{FrontalThet{a}_{GFP}}{ParietalAlph{a}_{GFP}}}={\displaystyle \frac{\frac{1}{5}\sum _{i=1}^5{x}_{i,theta}^2\left(t\right)}{\frac{1}{3}\sum _{i=1}^3{x}_{i,alpha}^2\left(t\right)}},$$

$$\begin{gathered} Attention\mathrm{}=\mathrm{}{\displaystyle \frac{FrontalBet{a}_{GFP}}{FrontalThet{a}_{GFP}}}={\displaystyle \frac{\frac{1}{5}\sum _{i=1}^5{x}_{i,beta}^2\left(t\right)}{\frac{1}{5}\sum _{i=1}^5{x}_{i,theta}^2\left(t\right)}}, \\ Engagement\mathrm{}=\mathrm{}{\displaystyle \frac{ParietalBet{a}_{GFP}}{Parietal\mathrm{}Thet{a}_{GFP}+ParietalAlph{a}_{GFP}}}={\displaystyle \frac{\frac{1}{3}\sum _{i=1}^3{x}_{i,beta}^2\left(t\right)}{\frac{1}{3}\sum _{i=1}^3{x}_{i,theta}^2\left(t\right)+\frac{1}{3}\sum _{i=1}^3{x}_{i,alpha}^2\left(t\right)}}, \end{gathered}$$

where $FrontalThet{a}_{GFP}$ and $FrontalBet{a}_{GFP}$ were computed by considering the AFz, AF3, AF4, AF7, and AF8 EEG channels, while the $ParietalThet{a}_{GFP}$, $ParietalAlph{a}_{GFP},ParietalBet{a}_{GFP}$ were computed by considering the Pz, P3, and P4 EEG channels. In this context, the mental workload index calculation was based on prior research using EEG to assess mental workload [5,32,33,34]. The Attention index was defined as the inverse of the Theta-Beta Ratio, an established EEG marker of attentional deficits in Attention Deficit Hyperactivity Disorder (ADHD) [35,36] and a validated measure of distributed attention during concurrent task performance [37,38]. The Engagement index was defined according to prior scientific works which validated the cognitive aspect of engagement in similar learning context [39]. Subsequently, the above-described neurophysiological metrics were z-score normalized according to the eyes open-condition, considered as baseline. Then, the mental workload and attention metrics were furtherly combined to compute the Human Distraction index (HDI), based on the assumption that a high workload does not necessarily imply task engagement, since mental resources can also be devoted to activities not related to the task being performed (i.e. mind wandering [40,41,42]. Such an index was already successfully proposed by Ronca and colleagues [43] in a highly realistic simulated driving context. Such an index was defined according to the following:

$$HDI=\mathrm{M}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{k}\mathrm{l}\mathrm{o}\mathrm{a}\mathrm{d}-\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}=\mathrm{}{\displaystyle \frac{FrontalThet{a}_{GFP}}{ParietalAlph{a}_{GFP}}}-\mathrm{}{\displaystyle \frac{FrontalBet{a}_{GFP}}{FrontalThet{a}_{GFP}}}$$

2.3.2. Photoplethysmography (PPG)

Photoplethysmography (PPG) signals, crucial for evaluating volumetric variations in blood content within biological tissues, were collected using the advanced Shimmer3 GSR+ apparatus (Shimmer Sensing, Dublin, Ireland). This instrument was securely positioned on the wrist of the participant's non-dominant hand, ensuring stable and precise signal capture at a sampling rate of 64 Hz. The untarnished PPG datasets underwent a digital filtration process using a 5th-order Butterworth band-pass filter, specifically spanning a frequency range of 1–5 Hz. This filtering protocol was carefully designed to effectively exclude the underlying continuous components and mitigate the influence of any gradual signal drift, whilst accentuating the characteristic pulsatile oscillations inherent in the PPG signal, which are indicative of cardiac activity. To further delve into the detailed cardiac rhythms, the distinguished Pan–Tompkins’s algorithm [44] QRS Detection was deployed. This algorithm is adept at discerning pulse-associated peaks, which subsequently facilitated the computation of the Inter-Beat Intervals (IBI signal). The resultant IBI datasets, however, occasionally contained aberrations or artefacts. To address this, the comprehensive HRVAS Matlab suite [45] was harnessed to refine and enhance the data quality. Subsequently, these purified IBI signals were algorithmically processed to deduce the Heart Rate (HR), typically represented in the metric "Beats per minute". To ensure individual specificity and normalization, the derived HR metrics for each participant were systematically adjusted by deducting their personalized baseline HR mean and then normalizing the outcome using their unique HR standard deviation.

2.3.3. Electrodermal Activity (EDA)

Electrodermal activity (EDA) signals, indicative of fluctuations in the skin's electrical conductance as a consequence of modulations in sweat gland activity, were acquired employing the advanced Shimmer3 GSR+ unit (Shimmer Sensing, Dublin, Ireland), the same cutting-edge apparatus mentioned previously. This device was positioned on the wrist of the participant's non-dominant hand to guarantee optimal data fidelity, with EDA signals being recorded at a detailed sampling rate of 64 Hz. The raw EDA signals underwent a series of computational refinements. Initially, the signals were passed through a low-pass filter with a discriminative cut-off frequency set at 1 Hz, serving to emphasize the desired frequency components and attenuate unwanted high-frequency noise. Post this filtration step, an advanced artefact correction tool within the Matlab framework was wisely employed to repair and eliminate any evident signal anomalies, such as sudden discontinuities and unwarranted spikes. To further refine and process these signals, the Ledalab suite [R] was deployed, a specialized open-source software toolbox seamlessly integrated within the Matlab (MathWorks Inc., Natick, MA, USA) ecosystem, tailored specifically for advanced EDA signal analysis. Employing the principles of continuous decomposition analysis [46], the suite enabled the separation of the EDA signal into its constituent tonic (SCL) and phasic (SCR) components. The SCL, or Skin Conductance Level, represents the gradual and sustained modulations in the EDA signal and predominantly encapsulates the overarching arousal state of the participant. Conversely, the SCR, or Skin Conductance Response, mirrors the transient and rapid fluctuations in the EDA and is conventionally attributed to discrete stimulus-driven physiological reactions. However, given the inherent limitations stemming from the involved devices' relatively low sampling frequency, this study had to prudently prioritize and focus on the more steady-state SCL component for its analytical rigor. As a final step in the data normalization process, the extracted SCL values for each individual were systematically adjusted to a standard scale. This was achieved by deducting the participant-specific baseline SCL, followed by normalization using the unique SCL standard deviation pertinent to each participant.

2.4. Subjective and Behavioral Data Collection

Following each experimental condition, participants completed a five-question quiz designed to assess comprehension and retention of the material presented. Crucially, the quiz questions were specific to the content of each individual modality, preventing potential learning transfer effects across conditions. Practically speaking, even if all the three contents were related to the same topic, the quiz was designed in order to be related to information contained only in that specific content. In addition to the comprehension quiz, participants answered five questions evaluating their subjective learning experience, adapted from previously validated questionnaires [47,48]. More specifically, these inquiries solicited participants to assign a rating, on a numerical continuum from 1 to 10, reflecting their subjective experiences concerning:

• The simplicity with which they could comprehend the disseminated information.
• The facility with which they could internalize the content.
• The capacity to sustain attention during the entirety of the task.
• The degree of interest elicited by the employed narrative modality.
• The extent of engagement is provoked by the narrative approach.

In the evaluation process, careful attention was taken to ensure procedural consistency; every participant was presented with an identical set of questions, which bolstered the uniformity and comparability of the assessment metrics.

3. Results

This section was divided into different subparagraphs in order to organize all the results on the basis of the related data source, i.e., questionnaires (subjective results), learning performance (behavioral results), and neurophysiological ones.

Among all the comparisons, only the analysis providing statistically significant results are here reported.

3.1. Neurophysiological Results

The statistical analysis performed on the autonomic-derived parameters, i.e., SCL and SCR computed from the EDA and the PPG-derived features, did not reveal any statistical effect between the tested experimental conditions.

Concerning the EEG-based parameters, ANOVA showed a statistically significant main effect of the experimental conditions in terms of HDI and Engagement index (HDI: F = 6.560, p = 0.007; ω² = 0.204; Engagement index: F = 19.641, p < 0.001; ω² = 0.357) (Figure 2 and Figure 3). More specifically, the post-hoc analysis showed that during the Educational video condition, participants exhibited the lowest HDI (p < 0.01), and the highest Engagement index (p < 0.006). Any statistically significant differences were observed between the Academic video and Text reading conditions in terms of HDI and Engagement index (all p > 0.10).

3.2. Correlation Analysis

Finally, a conclusive statistical analysis was performed to assess the correlation between the neurophysiological measurements related to the impact of the different learning material and their respective subjective scores provided by the participants along the experimental conditions. More specifically, the repeated measure correlation analysis [49] was performed between the EEG-based Engagement index and its respective subjective score (i.e., EP). As shown in Figure 4, the analysis revealed a strong and significant correlation between these two measurements (R = 0.621; p = 0.002), indicating that the measured cognitive impact of the different learning materials in terms of engagement was temporally coherent with the subjective perceptions of the participants.

3.3. Subjective Reuslts

The subjective data collected along the experimental protocol were combined in order to obtain two subjective scores. The first one is associated with the required cognitive resources perception (CRP), computed by averaging the subjective scores related to the first three dimensions of the above-described questionnaire (i.e., the higher the better, since high scores were associated with easy task perceptions). The second one is associated with the subjective engagement perception (EP) (i.e., the higher the better, since high scores were associated with engaging task perceptions), obtained by averaging the latter two dimensions of the above-described questionnaire.

The statistical analysis revealed a significant main effect of the experimental condition associated with the subjective CRP and EP scores (CRP: Friedman chi-squared = 10.474, p = 0.001, η² = 0.335; EP: Friedman chi-squared = 15.436, p < 0.001, η² = 0.459). More specifically, the post-hoc analysis showed that the Text reading condition was perceived as more requiring in terms of cognitive resources and less engaging (all p < 0.01). Additionally, the post-hoc analysis showed that the Educational video condition resulted statistically more engaging (p < 0.001) compared to the Text reading one (Figure 5).

3.4. Behavioral Results

The non-parametric repeated measure analysis performed on the behavioral data, i.e., the correct answers recorded for each experimental condition, revealed that participants committed the highest error rate during the Text reading condition (Friedman chi-squared = 6.178, p = 0.032, η² = 0.116; post-hoc: p = 0.02) (Figure 6).

4. Discussion

This study aimed to neurophysiologically characterize the impact of different learning materials by employing a multimodal approach that integrated EEG, EDA, and PPG recordings, in parallel with subjective measurements. The results demonstrated that different educational contents elicit distinct neurophysiological responses, highlighting how cognitive load, engagement, and attentional mechanisms vary depending on the format of the learning content.

The findings indicate that video-based learning materials, particularly Educational video enriched with visual elements and structured narration, were associated with higher engagement levels, as evidenced by EEG-derived metrics. This is consistent with previous literature suggesting that dynamic and multimodal learning experiences enhance cognitive processing and memory retention. Conversely, the text-based learning condition (i.e., Text reading) elicited higher cognitive distraction and a lower engagement index, as reflected in EEG-derived parameters. This aligns with prior research showing that passive reading demands sustained attention and greater cognitive effort, potentially leading to increased mental fatigue. The strong correlation between subjective engagement scores and neurophysiological engagement indices further validates the robustness of the proposed multimodal framework. This consistency between self-reported experiences and objective measures highlights the potential of neurophysiological monitoring as a reliable tool for assessing learning effectiveness in real-world educational settings.

However, some limitations must be acknowledged. The study involved a relatively small sample size, limiting the generalizability of the results. Additionally, the experimental design focused on a single topic, which may not fully capture the variability of different subject matters. Future research should expand the dataset, include a broader range of learning materials, and explore individual differences in cognitive responses.

Despite these limitations, this study represents a step forward in understanding how learning materials shape cognitive and emotional engagement, paving the way for more personalized and effective digital learning experiences.

5. Conclusions

This study investigated the impact of different learning materials on specific cognitive processes by employing a multimodal approach that integrated EEG, EDA, and PPG recordings with subjective measurements. The findings demonstrated that different educational modalities elicit distinct neurophysiological responses, providing objective insights into how cognitive load, attentional mechanisms, and engagement vary depending on the format of the learning content.

The results underscore the potential of neurophysiological measures to optimize learning experiences by providing real-time, objective assessments of cognitive processes specifically related to learning. More importantly, the proposed multimodal approach relies on wearable equipment for collecting neurophysiological signals, coupled with signal processing methods fully compatible with real-world applications [4,50,51,52,53].

5.1. Future trends

The effectiveness of the proposed neurophysiological framework in distinguishing between different learning conditions highlights its potential applicability in adaptive learning environments. By integrating real-time neurophysiological feedback, future e-learning systems could dynamically adjust content delivery to optimize learning efficiency and reduce cognitive overload.

Future e-learning platforms could leverage this approach to dynamically adjust instructional materials based on individual cognitive responses, enhancing learning efficiency and reducing distractive behaviors [54], [55,56,57]. In conclusion, this research represents a preliminary, but significant step forward in understanding how different learning materials impact cognitive and engagement, paving the way for more adaptive and personalized educational experiences.