Submitted:
23 December 2025
Posted:
25 December 2025
You are already at the latest version
Abstract
This paper presents a repeated cross-sectional longitudinal (trend) analysis of students’ self-perceived sustainability competence development across three waves surrounding participation in the Art Nouveau Path, a heritage-based mobile augmented reality game designed to foster sustainability competences, located in Aveiro, Portugal. In total, 1,094 questionnaires were collected using a GreenComp-grounded instrument adapted from the GreenComp-based Questionnaire (GCQuest) to this context (25 items; 6-point Likert). Data were gathered at three stages: baseline (S1-PRE; N = 221), immediately post-intervention (S2-POST; N = 439; n = 438 retained for scale scoring after applying a predefined completeness criterion), and follow-up (S3-FU; N = 434). Because responses were anonymous, waves were treated as independent samples rather than within-student trajectories. The Embodying Sustainability Values domain score and item-level response distributions were compared across waves using ordinal-appropriate non-parametric group comparisons, effect-size estimation, and descriptive threshold indicators. Results indicate an improvement from baseline to post-intervention, followed by partial attenuation at follow-up while remaining above baseline. Mean scores increased from 3.70 (S1-PRE) to 4.64 (S2-POST) and then stabilized at 4.13 (S3-FU). These findings, while exploratory, suggest that this heritage-based augmented reality game may have enhanced perceived sustainability competences. A structured program of follow-up activities is proposed to help sustain gains.
Keywords:
1. Introduction
2. Theoretical Framework
2.1. Cultural Heritage Education and Digital Mediation
2.2. AR and Mobile Game-Based Learning
2.3. Educational Data Mining and Learning Analytics as an Evaluation Lens
2.4. Measuring Sustainability Competences Through Perceived Competence
2.5. Synthesis
3. Methods and Materials
3.1. Research Design and Study Procedures
3.2. Context and Intervention Setting
3.3. Participants
3.4. Data Entry, Questionnaire Waves, and Analytical Samples
3.5. Instruments and Measures
3.5.1. GreenComp-Based Perceived Competence Questionnaire (S1-PRE, S2-POST, S3-FU)
3.5.2. The ESV Score as Measures Used in This Study
3.5.3. Derived Indicators for Threshold-Based Analyses
3.5.4. The GCQuest Validation Context
3.6. Data Processing and Scoring
3.7. Statistical Analysis
3.8. Cross-software Verification
3.9. Ethical Considerations and Data Access
4. Results
4.1. Data Completeness and Internal Consistency
4.2. Domain-Level Evolution of ESV
4.3. Proportions of Students Reaching Higher Competence Bands
4.4. Item-Level Patterns in ESV
4.5. Triangulation Between Item Discourse Features and S1-S2-S3 Trajectories
5. Discussion
5.1. Summary of the Main Findings and Linkage to the RQ
5.2. Summary of the Main Findings and Linkage to the RQ
5.3. Interpreting Competence Bands: What Shifts in High Endorsement Do and Do Not Imply?
5.4. Item-Level Insights and Implications for Game and Tasks Design
6. Conclusions
6.1. Main Conclusions
- (1)
- The post-intervention wave (S2-POST) presents a marked uplift in perceived sustainability values relative to baseline (S1-PRE), accompanied by a clear upward distributional shift and a higher prevalence of students in higher endorsement bands, with moderate-to-high endorsement showing clearer maintenance at follow-up (S3-FU) than very high endorsement. In substantive terms, a short, carefully designed, place-based mobile AR experience may make sustainability values salient and strengthen students’ value-oriented self-appraisals linked to care, responsibility, and stewardship in relation to built heritage and sustainability concepts;
- (2)
- The trajectory indicates partial attenuation over time rather than a stable plateau. At follow-up (S3-FU), ESV scores decrease relative to the immediate post-intervention measurement (S2-POST) but remain clearly above baseline (s1-PRE) at the domain level. This pattern is consistent with a residual positive trace of the experience while suggesting that the highest levels of endorsement are difficult to maintain without reinforcement beyond the gameplay session;
- (3)
- item-level trajectories indicate heterogeneous sensitivity within ESV. Items closely aligned with concrete, place-centered forms of care and responsibility show more robust retention, whereas more abstract or conceptually dense formulations show weaker long-term differentiation. Methodologically, this reinforces the value of reporting domain-level indicators alongside item-level patterns when evaluating competence-oriented ESD interventions in authentic, technology-mediated contexts;
6.2. Limitations
6.3. Future Paths
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
| ESD | Education for Sustainable Development |
| AR | Augmented Reality |
| MARG | Mobile Augmented Reality Game |
| DTLE | Digital Teaching and Learning Ecosystem |
| POI | Point of Interest |
| GCQuest | GreenComp-based Questionnaire |
| RQ | Research Question |
| EfS | Education for Sustainability |
| GBL | Game-Based Learning |
| EDM | Educational Data Mining |
| LA | Learning Analytics |
| ESV | Embodying Sustainability Values |
| SEM | Structural Equation Modeling |
| KSA | Knowledge, Skills, and Attitudes |
| CFA | Confirmatory Factor Analysis |
| DWLS | Diagonally Weighted Least Squares |
| CFI | Comparative Fit Index |
| TLI | Tucker–Lewis Index |
| SRMR | Standardized Root Mean Square Residual |
| RMSEA | Root Mean Square Error of Approximation |
| IQR | Interquartile Range |
| GDPR | General Data Protection Regulation |
| SD | Standard Deviation |
| CI | Confidence Interval |
| df | Degrees of Freedom |
References
- UNESCO, Education for Sustainable Development: A Roadmap. UNESCO, 2020.
- UN, “Transforming our world: the 2030 Agenda for Sustainable Development (A/RES/70/1),” UN General Assembly, 2015. [CrossRef]
- Council of Europe, “Council of Europe Framework Convention on the Value of Cultural Heritage for Society,” 2005. [CrossRef]
- UNESCO. Convention Concerning the Protection of the World Cultural and Natural Heritage. 1972. Available online: https://whc.unesco.org/archive/convention-en.pdf (accessed on 23 December 2025).
- A. Lerario, “The Role of Built Heritage for Sustainable Development Goals : From Statement to Action,” Heritage, vol. 5, pp. 2444–2463, 2022. [CrossRef]
- M. Akçayır and G. Akçayır, “Advantages and challenges associated with augmented reality for education: A systematic review of the literature,” Educ. Res. Rev., vol. 20, no. February, pp. 1–11, 2017. [CrossRef]
- I. Radu, “Augmented reality in education: a meta-review and cross-media analysis,” Pers Ubiquit Comput, vol. 18, pp. 1533–1543, 2014. [CrossRef]
- G. Bianchi, U. Pisiotis, M. Cabrera, Y. Punie, and M. Bacigalupo, The European sustainability competence framework. 2022.
- P. Martín-Ramos et al., “Operationalizing the European sustainability competence framework: Development and validation of learning outcomes for GreenComp,” Open Res. Eur., vol. 5, p. 203, Oct. 2025. [CrossRef]
- J. Ferreira-Santos and L. Pombo, “The Art Nouveau Path: Promoting Sustainability Competences Through a Mobile Augmented Reality Game,” Multimodal Technol. Interact., vol. 9, no. 8, p. 77, Jul. 2025. [CrossRef]
- J. Ferreira-Santos and L. Pombo, “The Art Nouveau Path: Integrating Cultural Heritage into a Mobile Augmented Reality Game to Promote Sustainability Competences Within a Digital Learning Ecosystem,” Sustainability, vol. 17, no. 18, p. 8150, Sep. 2025. [CrossRef]
- J. Ferreira-Santos and L. Pombo, “The Art Nouveau Path: Trajectory Analysis and Spatial Storytelling Through a Location-Based Augmented Reality Game in Urban Heritage,” ISPRS Int. J. Geo-Information, vol. 14, no. 12, p. 33, 2025. [CrossRef]
- A. Chatsiopoulou and P. D. Michailidis, “Augmented Reality in Cultural Heritage : A Narrative Review of Design , Development and Evaluation Approaches,” Heritage, vol. 8, no. 421, p. 33, 2025.
- H. E. Brady and R. Johnston, “Repeated Cross-Sections in Survey Data,” in Emerging Trends in the Social and Behavioral Sciences, Wiley, 2015, pp. 1–18.
- M. J. Lebo and C. Weber, “An Effective Approach to the Repeated Cross-Sectional Design,” Am. J. Pol. Sci., vol. 59, no. 1, pp. 242–258, Jan. 2015. [CrossRef]
- K. L. Mansfield, S. Puntis, E. Soneson, A. Cipriani, G. Geulayov, and M. Fazel, “Study protocol: the OxWell school survey investigating social, emotional and behavioural factors associated with mental health and well-being,” BMJ Open, vol. 11, no. 12, p. e052717, Nov. 2021. [CrossRef]
- K. Winter et al., “Concept and methodology of the Health Behaviour in School-aged Children (HBSC) study - Insights into the current 2022 survey and trends in Germany.,” J. Heal. Monit., vol. 9, no. 1, pp. 99–117, Mar. 2024. [CrossRef]
- J. Ferreira-Santos, L. Pombo, and M. M. Marques, “GreenComp-based Questionnaire (GCQuest) - ING,” 2024. [CrossRef]
- B. Pelzer, R. Eisinga, and P. H. Franses, “‘Panelizing’ Repeated Cross Sections,” Qual. Quant., vol. 39, no. 2, pp. 155–174, Apr. 2005. [CrossRef]
- C. Achille and F. Fiorillo, “Teaching and Learning of Cultural Heritage: Engaging Education, Professional Training, and Experimental Activities,” Heritage, vol. 5, no. 3, pp. 2565–2593, 2022. [CrossRef]
- L. Smith, Uses of Heritage. Routledge, 2006.
- F. Choay, As questões do Património. Edições 70, 2021.
- F. Bandarin and R. van Oers, The Historic Urban Landscape. Wiley, 2012.
- ICOMOS, “Charter for the Conservation of Historic Towns and Urban Areas,” 1987. [CrossRef]
- E. Avrami, S. Macdonald, R. Mason, and D. Myers, Values in Heritage Management, vol. 1, no. 1. Los Angeles: The Getty Conservation Institute, 2019.
- E. Kamjou and M. Scott, “The heritage-climate change nexus: towards a values-based adaptive planning response for cultural landscapes,” J. Environ. Plan. Manag., pp. 1–20, 2025. [CrossRef]
- T. Ard, M. S. Bienkowski, S.-L. Liew, F. Sepehrband, L. Yan, and A. W. Toga, “Integrating Data Directly into Publications with Augmented Reality and Web-Based Technologies – Schol-AR,” Sci. Data, vol. 9, no. 1, p. 298, Jun. 2022. [CrossRef]
- J. Xu and Y. Pan, “The Future Museum: Integrating Augmented Reality (AR) and Virtual-text with AI-enhanced Information Systems,” J. Wirel. Mob. Networks, Ubiquitous Comput. Dependable Appl., vol. 15, no. 3, pp. 373–394, 2024. [CrossRef]
- Ferreira-Santos, J.; Pombo, L. The Art Nouveau Path: Valuing Urban Heritage Through Mobile Augmented Reality and Sustainability Education. Heritage 2026, 9, 4. [CrossRef]
- C. Avila-Garzon, J. Bacca-Acosta, Kinshuk, J. Duarte, and J. Betancourt, “Augmented Reality in Education: An Overview of Twenty-Five Years of Research,” Contemp. Educ. Technol., vol. 13, no. 3, p. ep302, Apr. 2021. [CrossRef]
- M. Dunleavy and C. Dede, “Augmented Reality Teaching and Learning,” in Handbook of Research on Educational Communications and Technology, 4th ed., J. M. Spector, M. D. Merrill, J. Elen, and B. M. J., Eds. New York: Springer, 2014, pp. 735–745.
- N. Pellas, P. Fotaris, I. Kazanidis, and D. Wells, “Augmenting the learning experience in primary and secondary school education: a systematic review of recent trends in augmented reality game-based learning,” Virtual Real., vol. 23, no. 4, pp. 329–346, Dec. 2019. [CrossRef]
- W.-T. Wang, Y.-L. Lin, and H.-E. Lu, “Exploring the effect of improved learning performance: A mobile augmented reality learning system,” Educ. Inf. Technol., vol. 28, no. 6, pp. 7509–7541, Jun. 2023. [CrossRef]
- L. T. De Paolis, C. Gatto, L. Corchia, and V. De Luca, “Usability, user experience and mental workload in a mobile Augmented Reality application for digital storytelling in cultural heritage,” Virtual Real., vol. 27, no. 2, pp. 1117–1143, Jun. 2023. [CrossRef]
- C. Alonso-Fernández, A. Calvo-Morata, M. Freire, I. Martínez-Ortiz, and B. Fernández-Manjón, “Game Learning Analytics: Blending Visual and Data Mining Techniques to Improve Serious Games and to Better Understand Player Learning,” J. Learn. Anal., vol. 9, no. 3, pp. 32–49, 2022. [CrossRef]
- Y. J. Kim, J. A. R. Valiente, D. Ifenthaler, E. Harpstead, and E. Rowe, “Analytics for Game-Based Learning,” J. Learn. Anal., vol. 9, no. 3, pp. 8–10, Dec. 2022. [CrossRef]
- D. Gašević, S. Dawson, and G. Siemens, “Let’s not forget: Learning analytics are about learning,” TechTrends, vol. 59, no. 1, pp. 64–71, Jan. 2015. [CrossRef]
- S. K. Banihashem, H. Dehghanzadeh, D. Clark, O. Noroozi, and H. J. A. Biemans, “Learning analytics for online game-Based learning: a systematic literature review,” Behav. Inf. Technol., vol. 43, no. 12, pp. 2689–2716, Sep. 2024. [CrossRef]
- I. Daoudi, “Learning analytics for enhancing the usability of serious games in formal education: A systematic literature review and research agenda,” Educ. Inf. Technol., vol. 27, no. 8, pp. 11237–11266, 2022. [CrossRef]
- P. Butterworth, N. Watson, and M. Wooden, “Trends in the Prevalence of Psychological Distress Over Time: Comparing Results From Longitudinal and Repeated Cross-Sectional Surveys,” Front. Psychiatry, vol. 11, Nov. 2020. [CrossRef]
- K. Ye, A. Bilinski, and Y. Lee, “Difference-in-differences analysis with repeated cross-sectional survey data,” Heal. Serv. Outcomes Res. Methodol., Oct. 2025. [CrossRef]
- G. Cebrián, M. Junyent, and I. Mulà, “Current practices and future pathways towards competencies in education for sustainable development,” Sustain., vol. 13, no. 16, p. 8733, 2021. [CrossRef]
- B. Lira et al., “Large studies reveal how reference bias limits policy applications of self-report measures,” Sci. Rep., vol. 12, no. 1, p. 19189, Nov. 2022. [CrossRef]
- R. K. Yin, Case Study Research Design and Methods, 5th ed. Thousand Oaks, CA: SAGE Publications, Inc., 2015.
- M. Barth and I. Thomas, “Synthesising case-study research - ready for the next step?,” Environ. Educ. Res., vol. 18, no. 6, pp. 751–764, 2012. [CrossRef]
- S. Mckenney and T. Reeves, “Education Design Research,” in Handbook of Research on Educational Communications and Technology: Fourth Edition, 2014, p. 29.
- T. Anderson and J. Shattuck, “Design-Based Research,” Educ. Res., vol. 41, no. 1, pp. 16–25, Jan. 2012. [CrossRef]
- Municipal Educational Action Program of Aveiro 2024–2025 (PAEMA). Available online: https://tinyurl.com/PAEMAveiro (accessed on 12 November 2025).
- R. Garland, “The Mid-Point on a Rating Scale: Is it Desirable?,” Mark. Bull., vol. 2, pp. 66–70, 1991, [Online]. Available: https://rangevoting.org/MB_V2_N3_Garland.pdf. (accessed on 21 November 2025).
- D. Beglar and T. Nemoto, “Developing Likert-scale questionnaires,” in JALT2013 Conference Proceedings, 2014, pp. 1–8.
- L. South, D. Saffo, O. Vitek, C. Dunne, and M. A. Borkin, “Effective Use of Likert Scales in Visualization Evaluations: A Systematic Review,” Comput. Graph. Forum, vol. 41, no. 3, pp. 43–55, Jun. 2022. [CrossRef]
- M. M. Marques, J. Ferreira-Santos, R. Rodrigues, and L. Pombo, “Mobile Augmented Reality Games Towards Smart Learning City Environments: Learning About Sustainability,” Computers, vol. 14, no. 7, p. 267, Jul. 2025. [CrossRef]
- J. Ferreira-Santos, M. M. Marques, and L. Pombo, “GreenComp-Based Questionnaire (GCQuest): Questionnaire Development and Validation,” [unpublished work]. Dec. 30, 2024. [CrossRef]
- J. Ferreira-Santos, L. Pombo, and M. M. Marques, “GreenComp-based Questionnaire (GCQuest) - PT,” 2024. [CrossRef]
- JASP Team, ‘JASP.’ JASP, Amsterdam, The Netherlands, 2025, [Online]. Available: https://jasp-stats.org/. (accessed on 12 November 2025).
- Ferreira-Santos, J. (2025). GCQUEST SEM EduCITY analysis in JASP 0.19.3. Zenodo. [CrossRef]
- Art Nouveau Path MARG’s Zenodo Community. Available online: https://zenodo.org/communities/artnouveaupath/records/ (accessed on 23 December 2025).








| Wave | Raw Data (N) |
Analytic Data (N) |
Missing cells (Raw Data) | Out-of-range | Response range | Cronbach’s alpha | McDonald’s omega |
|---|---|---|---|---|---|---|---|
| S1-PRE | 221 | 221 | 0 | 0 | 1 to 6 | 0.72 | 0.72 |
| S2-POST | 439 | 438 | 7 | 0 | 1 to 6 | 0.88 | 0.88 |
| S3-FU | 434 | 434 | 0 | 0 | 1 to 6 | 0.75 | 0.76 |
| Wave | N | Mean | Standard Deviation (SD) | 95% Confidence Interval (CI) |
|---|---|---|---|---|
| S1-PRE | 221 | 3.70 | 0.54 | [3.63, 3.77] |
| S2-POST | 438 | 4.64 | 0.50 | [4.59, 4.68] |
| S3-FU | 434 | 4.13 | 0.36 | [4.09, 4.16] |
| Contrast | Delta (Mean B - Mean A) | 95% CI (Delta) | Welch t | Degrees of Freedom (df) | p (Holm) | Hedges g |
|---|---|---|---|---|---|---|
|
S1-PRE vs S2-POST |
+0.93 | [0.85, 1.02] | -21.57 | 413.12 | p < .001 | +1.82 |
|
S2-POST vs S3-FU |
-0.51 | [-0.57, -0.45] | +17.26 | 798.23 | p < .001 | -1.17 |
|
S1-PRE vs S3-FU |
+0.43 | [0.35, 0.50] | -10.59 | 324.71 | p < .001 | +0.99 |
| Threshold | S1-PRE (n/N) | S2-POST (n/N) | S3-FU (n/N) |
|---|---|---|---|
| ≥ 4.0 | 64/221 (28.96%) | 388/438 (88.58%) | 326/434 (75.12%) |
| ≥ 4.5 | 20/221 (9.05%) | 310/438 (70.78%) | 43/434 (9.91%) |
| ≥ 5.0 | 0/221 (0.00%) | 72/438 (16.44%) | 11/434 (2.53%) |
| Pattern | Items | Summary (Delta) |
|---|---|---|
|
Largest increases S1-PRE to S2-POST |
Q7, Q17, Q6, Q15, Q5 | +1.25, +1.24, +1.22, +1.19, +1.19 |
|
Largest decreases S2-POST to S3-FU |
Q23, Q3, Q17, Q25, Q5 | -0.77, -0.70, -0.68, -0.67, -0.64 |
|
Items not significant S1-PRE vs S3-FU (Holm) |
Q2, Q3, Q9, Q10, Q13, Q23, Q24, Q25 |
Long-term differences not robust after multiplicity control |
| Pattern | Items | Summary (Delta S2–S3 = S3 minus S2) |
|---|---|---|
|
Largest decreases S2-POST to S3-FU |
Q23, Q3, Q17, Q25, Q5 | -0.77, -0.70, -0.68, -0.67, -0.64 |
| Smallest decreases S2-POST to S3-FU (best retention) | Q12, Q21, Q10, Q1, Q9 | -0.25, -0.29, -0.32, -0.35, -0.41 |
| Item Category (per KSA) |
Items (Q) | n items | Delta S1–S2 | Delta S2–S3 | Delta S1–S3 |
|---|---|---|---|---|---|
| Knowledge (K) | Q4, Q10, Q16, Q21, Q24 | 5 | 0.85 | -0.45 | 0.40 |
| Skills (S) |
Q2, Q3, Q6, Q7, Q8, Q12, Q14, Q15, Q18, Q19, Q20, Q22, Q23 |
13 | 0.98 | -0.51 | 0.47 |
| Attitudes (A) |
Q1, Q5, Q9, Q11, Q13, Q17, Q25 |
7 | 0.91 | -0.55 | 0.36 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).