A Measurement Framework for Serious Games Addressing Food Loss and Waste (SG-FLW)

1. Introduction

Serious games and gamified digital interventions are increasingly proposed as tools to reduce household food waste [1,2]. However, the field suffers from a fundamental measurement paradox: interventions that claim to change behavior rarely measure behavior, and those that measure behavior rarely quantify environmental impact. Most studies evaluate only knowledge gains or attitudinal shifts, leaving actual effectiveness as an open question [3,4]. Prior reviews have catalogued existing interventions and their design characteristics [5,6], but no standardized tool exist to systematically assess the quality of evidence these interventions produce.

Food loss and food waste (FLW), though often conflated, refer to different stages of the supply chain: food loss occurs during production, post-harvest handling, and processing, while food waste arises at the retail and consumer levels [7,8]. The European Union (EU) Waste Framework Directive mandates that Member States develop prevention programs with comparable monitoring, and the European Commission’s Joint Research Centre has published evaluation frameworks for assessing the performance of food waste prevention actions [9,10]. Yet most digital interventions have not been aligned with these reporting standards, and food waste is measured in inconsistent units (kilograms, packages, or self-reported estimates) when it is measured at all [1,11]. SG-FLW focuses on the consumer-facing stage where serious games and gamification operate, but its measurement logic applies across the supply chain.

This measurement gap extends beyond food waste. Hognon et al. [12] showed that Climate Fresk evaluations are predominantly qualitative, rely on small non-randomized samples, and include no long-term follow-up; they argue that evaluation must also consider cognitive and attitudinal processes, not only behavioral outcomes. The same pattern holds for food waste: studies evaluated without behavior-change frameworks risk overestimating impact by focusing on immediate or self-reported outcomes [12], and gamification-driven engagement may not sustain change once novelty effects subside [13].

Existing frameworks each address part of this problem but none covers the full measurement chain from baseline to persistence. GRADE [14] assesses evidence certainty but is designed for clinical trials, not behavioral interventions. RE-AIM [15] includes maintenance but does not grade baseline or environmental conversion. Kirkpatrick [16] sequences learning before behavior but provide no waste-specific measurement criteria. Typical serious game evaluation rubrics assess engagement and learning outcomes without connecting them to environmental impact. Appraisal checklists (TIDieR, Cochrane Risk of Bias, MMAT) address reporting completeness, internal validity, or mixed-methods quality, but none grades domain-specific measurement rigor across the full chain. SG-FLW (Serious Games for Food Loss & Waste) differs by integrating these concerns into a single five-dimensional measurement chain, synthesizing principles from eight sources (CONSORT/GRADE, Behavior Change Technique Taxonomy v1 (BCTTv1) [17], Kirkpatrick, Theory of Planned Behavior (TPB) [18], Cohen’s d, ISO 14040–14044, RE-AIM, CONSORT-SPI [19]). It evaluates the quality of evidence that an intervention reduces food waste, not whether the game is “good” or “engaging.”

The framework is designed for researchers, reviewers, and practitioners working with digital interventions targeting food waste, including serious games, gamified mobile applications, interactive installations, and dashboard-based campaigns. SG-FLW supports two complementary modes of use: retrospectively, to evaluate the evidence quality of published studies (as demonstrated in this paper); and prospectively, to guide the design of new interventions by setting target measurement levels before development begins. A research team designing a serious game can use the dimensional profiles to plan their evaluation strategy, for example committing to B2/K3/D2/E1/L2 as a target profile, thereby embedding rigorous measurement into the study design rather than treating it as an afterthought. Note that SG-FLW evaluates measurement quality, not intervention quality: a high profile means strong evidence exists, while a low profile indicates evidence is missing, not that the intervention failed. The contributions of this paper are: (i) defining the five SG-FLW dimensions and their 0–3 ordinal scales; (ii) specifying four derived indices for quantitative comparison; and (iii) applying the framework to thirty-five interventions, revealing a systematic complementary measurement gap (no study achieves both K≥2 and D≥2).

This paper is organized in 7 sections. Section 2 presents the SG-FLW framework architecture and its five dimensions. Section 3 defines the four derived calculable indices. Section 4 describes the coding protocol. Section 5 presents the evidence certainty classification. Section 6 applies the framework to thirty-five interventions and reports reliability and robustness analyses. Finally, Section 7 discusses implications and concludes.

2. Framework Architecture

SG-FLW evaluates five dimensions, each rated on a 0–3 ordinal scale following a causal logic chain: an intervention must first establish a Baseline (what was waste before?), then assess Knowledge & attitudinal outcomes (did people learn?), demonstrate Direct Behavioral change (did people act differently?), ideally convert that change into Environmental impact (what does it mean for the planet?), and finally assess whether the change Persists over time (abbreviated L for Longitudinal, the measurement method). Higher scores indicate greater measurement rigor and lower susceptibility to bias. A 0–3 ordinal scale was chosen to balance coder reliability with field-wide applicability, matching the practical evidence tiers observed across the heterogeneous literature; finer granularity (e.g., splitting B3 into calibrated vs. uncalibrated) reduced agreement in pilot coding without adding discriminative value. This causal chain also serves as a design guide: when used prospectively, each dimension prompts researchers to decide before development what level of measurement rigor they will implement at each stage.

2.1. B — Baseline Measurement Quality

Measures the rigor of pre-intervention food waste quantification, grounded in CONSORT and GRADE. Without a baseline, waste reduction cannot be calculated; self-reported baselines introduce recall and social desirability bias. For indices that combine baseline and post measures (e.g., Waste Reduction Rate (WRR)), baseline and outcome must be commensurate in unit and aggregation level.

2.2. K — Knowledge & Attitudinal Assessment

Captures whether and how the intervention assessed cognitive, attitudinal, or awareness outcomes, the intermediate processes that precede behavioral change. Grounded in Kirkpatrick’s training evaluation hierarchy [16], where Levels 1–2 (reaction and learning) must precede Level 3 (behavior), and in the Theory of Planned Behavior [18], which posits that attitudes and perceived behavioral control shape intentions before action. Many serious games primarily target these precursors; K captures the quality of that assessment independently of whether behavior was measured. K and D are operationally distinct: K evaluates assessments of what participants know, believe, or intend (e.g., a food storage quiz, an attitude scale), while D evaluates measurements of what participants do (e.g., weighed plate waste, observed sorting behavior). A study may score K3/D0 or K0/D3; neither dimension implies the other. Because K grades measurement rigor rather than construct type, it groups factual knowledge, attitudes, and self-efficacy into a single scale; this trades construct-level precision for cross-study comparability, a limitation most relevant at K2–K3 where different TPB determinants carry different predictive weight for behavior.

2.3. D — Direct Behavior Measurement

Captures how behavioral outcomes are recorded, grounded in BCTTv1 and Cohen’s d. Knowledge is not behavior: quiz scores and self-reported intentions correlate weakly with actual food waste behavior [18].

2.4. E — Environmental Conversion

Assesses whether behavioral change is translated into environmental impact metrics, grounded in LCA methodology (ISO 14040–14044). Reducing 1 kg of meat waste has a fundamentally different impact than reducing 1 kg of bread waste; without conversion, behavioral data cannot inform sustainability policy.

2.5. L — Persistence (Longitudinal Measurement)

Evaluates whether behavioral effects are assessed over time after the intervention ends, drawn from RE-AIM’s Maintenance dimension: “long-term effects ≥6 months after the most recent intervention contact” [15].

3. Calculable Indices

When minimum dimensional requirements are met, SG-FLW enables four derived indices computed from the study data.

WRR — Waste Reduction Rate (requires B≥2, D≥3):

$$\mathrm{WRR}={\displaystyle \frac{W_{\mathrm{pre}}-W_{\mathrm{post}}}{W_{\mathrm{pre}}}}$$

(1)

Proportion of waste eliminated (0–1.0). Pre and post measurements must be commensurate in unit and aggregation level.

BII — Behavioral Impact Index (individual-level; requires B≥3, D≥3):

$$\mathrm{BII}=d={\displaystyle \frac{M_{\mathrm{pre}}-M_{\mathrm{post}}}{S{D}_{\mathrm{pooled}}}}$$

(2)

Standardized effect size (Cohen [20]; 0.2/0.5/0.8 = small/medium/large). Positive values indicate waste reduction; apply Hedges’ correction when $N<20$.

EII — Environmental Impact Index (requires E≥1, D≥3):

$$\mathrm{EII}={\displaystyle \sum _{c=1}^k}\Delta W_c\times E{F}_c$$

(3)

where $\Delta W_c$ represents the waste reduction (kg) for food category c and $E{F}_c$ is its emission factor (kg CO₂eq/kg). EII requires D3; behavioral proxies (D2) are insufficient unless converted to mass units in-study.

PI — Persistence Index (requires L≥3, D≥2):

$$\mathrm{PI}\left(t\right)={\displaystyle \frac{\Delta W\left(t\right)}{\Delta W\left(t_0\right)}}$$

(4)

Ratio of the behavioral effect at time t to the initial post-intervention effect. With ≥3 time points, the decay constant $\lambda$ can be estimated and the behavioral half-life $T_{MM1/2}=ln2/\lambda$ computed.1

4. Coding Protocol

SG-FLW coding follows three principles adapted from CONSORT-SPI and BCTTv1: (1) Code what is reported, not what is implied: if a study does not explicitly report weighing of food waste, do not infer it; (2) Independent dimensions: each dimension is coded independently, so B0/K2/D3/E0/L0 is a valid profile; (3) Conservative coding: when evidence is ambiguous, code the lower level.

For each dimension, coders follow a decision path: first determine whether the relevant data exist (score 0 if absent), then classify its quality level (1–3) based on the criteria in Table 1, Table 2, Table 3, Table 4 and Table 5. Following BCTTv1 methodology [17], two coders independently rated each intervention; inter-rater reliability is reported in Section 6.3. In multi-arm trials where arms differ in measurement level, the profile reflects the arm with the strongest measurement chain.

5. Evidence Certainty Classification

Inspired by but distinct from the GRADE approach [14], SG-FLW profiles map to a measurement-chain certainty level (Table 6). Certainty here refers to certainty about waste reduction: substantiating a reduction claim requires both a commensurate baseline (B) and post-intervention behavioral measurement (D). SG-FLW classifies measurement-chain adequacy, not causal identification. SG-FLW therefore produces two orthogonal outputs: waste-reduction certainty (determined by B, D, E, L) and mechanism evidence (captured by K). Knowledge gains do not constitute evidence of waste reduction; K is reported separately because it captures the cognitive precursors that the attitude–behavior gap [18] shows are necessary but insufficient. For Moderate, D2 must capture behaviors linked to waste outcomes (not merely engagement metrics). A commensurability rule applies: baseline and outcome must match in unit and aggregation level (e.g., group-level baseline requires group-level outcome); violation downgrades the study one level (e.g., B3 in kg/household/week paired with D3 in g/person/day; no study in this corpus required downgrade). Moderate accepts behavioral proxies (D2), but waste-mass indices (WRR, EII) require D3. High requires E≥1 and L≥2 because policy-relevant waste reduction claims need environmental conversion (reducing 1 kg of meat waste differs from 1 kg of bread waste) and evidence of sustained change; in practice, E0 dominates because most studies lack food-category composition data needed for conversion. A study can reach High for behavioral certainty (B3/D3/L2) without E, but not for sustainability certainty, which demands environmental conversion; High is intentionally rare, representing policy-grade evidence sufficient for SDG 12.3 reporting. This is a GRADE-inspired measurement-chain classification, not a full GRADE assessment of bias and precision.

6. Framework Application

We applied SG-FLW to thirty-five interventions drawn from three sources: (1) nine interventions from our qualitative review [3]; (2) twenty-nine interventions from our systematic review [1]; and (3) two behavioral studies using cloud-based automatic weighing. Duplicates were identified by matching intervention name, authors, and year across sources; five serious games appeared in both reviews and were counted once. The merged corpus comprises games ($n=7$), serious games ($n=10$), and gamification ($n=18$) per the taxonomy in [1]. This category split is descriptive; SG-FLW is category-agnostic. Two coders independently rated each intervention (Section 4); reliability (${\kappa }_w=0.96$) and borderline decisions are in Section 6.3. The corpus spans prototype-stage studies scoring all zeros to quasi-experimental trials at Moderate certainty, showing the framework differentiates across evidence levels.

6.1. Worked Examples

To illustrate the coding process, we present two contrasting cases.

Worst case — Verloop [21]: This serious game for food waste data collection was evaluated using qualitative methods only. B: No pre-intervention waste data was collected (B0). K: Qualitative observations were gathered informally (K1). D: No behavioral outcomes, not even self-reported, were measured (D0). E: No environmental conversion (E0). L: No follow-up (L0). Profile: B0/K1/D0/E0/L0, Very Low certainty. The study provides no evidence of behavioral impact, although K1 acknowledges that qualitative data was collected.

Best case — Seta et al. [22]: This gamified self-monitoring app was tested with 126 households using a cloud-based automatic weighing system (SmartMat). B: Individual household food waste was weighed during a two-week baseline period (B3). K: No formal knowledge or attitudinal assessment was conducted; the study focused exclusively on behavioral waste measurement (K0). D: Post-intervention waste was directly measured in grams per household per week, yielding a 45% reduction as reported by the authors (D3). E: No environmental conversion was reported (E0). L: A follow-up approximately three months post-intervention was conducted, below the six-month RE-AIM threshold (L1). Profile: B3/K0/D3/E0/L1, Moderate certainty. This is the highest-scoring intervention on the behavioral evidence chain and the only one with any post-intervention follow-up, yet it illustrates a complementary gap: strong behavioral evidence without cognitive assessment.

6.2. Full Results

Table 7 presents the complete SG-FLW profiles for all thirty-five interventions.

SG-FLW provides a common vocabulary for comparing what was measured across a heterogeneous field. A Very Low profile means the study’s measurement focus lay elsewhere, not that the intervention failed. We define a complementary measurement gap as the pattern where studies scoring K≥2 score D=0, studies scoring D≥2 score K=0, and zero studies achieve both K≥2 and D≥2 simultaneously.

Of the thirty-five interventions, twenty-five score K>0 (K1: $n=11$; K2: $n=11$; K3: $n=3$), but twenty of those score D0. Yu et al. [52] is the nearest bridge, reaching D2 through app-logged actions but with only K1 and no baseline (B0/K1/D2). In Kirkpatrick’s terms [16], most studies remain at Levels 1–2 without progressing to Level 3 (behavior), and the documented attitude–behavior gap in food waste [4,11] confirms the need for objective data.

Conversely, five studies achieve Moderate certainty (B≥2, D≥2), yet all score K0. Dolnicar et al. [41] weighed plate waste (34% reduction); Joyner et al. [24] measured cafeteria vegetable consumption as a waste-reduction proxy (99.9% increase in servings taken); Soma et al. [44] conducted curbside audits across 164 households ($p=0.07$); and Seta et al. [22,55] deployed cloud-based automatic weighing across 126 and 119 households respectively (45% reduction, $d=0.341$). These studies reach Kirkpatrick Level 3 but bypass Level 2, leaving cognitive mechanisms unexplored. Where reported data permits, the indices from Section 3 can be computed: Dolnicar yields WRR=0.34, Seta (a) yields WRR=0.45 and BII$=d=0.341$; the remaining Moderate studies lack the dispersion statistics that BII requires.

No intervention achieves E≥1, only one [22] includes follow-up (L1), and none reaches High certainty. The highest combined K+D is K2/D1 (Löchtefeld [36]; Pajpach et al. [51]; Santa Cruz et al. [54]). In summary: 20/35 score K>0 with D=0; 5/35 score D≥2 with K=0; 0/35 achieve both K≥2 and D≥2. In cross-tabulation: of 14 studies with K≥2, none achieves D≥2; all 6 with D≥2 score K≤1. These patterns hold across all three categories and match the evaluation deficits Hognon et al. [12] identified in climate education, suggesting a field-wide gap rather than a sampling artifact.

6.3. Reliability and Robustness

To assess coding reliability, two coders independently rated all five dimensions for a purposive subsample of twelve interventions selected to span the full evidence range and include all identified borderline cases: Altarriba, Verloop, Seiler, Löchtefeld (Very Low); Pajpach, Yu, Santa Cruz (Low); Joyner, Dolnicar, Soma, Seta (a), Seta (b) (Moderate). Both coders were trained on the codebook (Table 1, Table 2, Table 3, Table 4 and Table 5) using three pilot studies before independent coding. Table 8 reports per-dimension results. ${\kappa }_w$ was computed for B, K, and D; E and L had no variance in the subsample (all E0; all L0 except Seta (a) L1, agreed by both coders), so $\kappa$ is undefined for those dimensions. The high agreement reflects the design of the scale: B and D levels are anchored in observable reporting artifacts (weighing, audits, app logs) rather than subjective judgments, which constrains coder disagreement. The two disagreements were each one level apart: Seiler K3 vs. K2 (TAM as validated instrument vs. structured scale) and Yu D2 vs. D1 (app-logged actions vs. self-initiated records). After discussion, both were resolved to the higher category (K3, D2) because the studies met the formal criteria in Table 2 and Table 3; Seiler explicitly hypothesizes acceptance as a determinant of adoption behavior and uses a validated instrument (TAM) with A/B design, satisfying the K3 mediator criterion; Yu’s 800+ app-logged actions are server-recorded, not self-reported. E and L reliability cannot be assessed in this corpus due to insufficient variance; future application to studies with E≥1 or L≥2 should include these dimensions in reliability testing. $\kappa$ values may also be lower in corpora with greater E/L variance, where finer distinctions between adjacent levels become necessary.

Table 9 documents six borderline scoring decisions where adjacent levels were considered. These cases operationalize the boundaries of each scale and illustrate decision rules.

We tested four threshold perturbations: (1) L2 threshold lowered from 6 to 3 months: Seta (a) L1→L2, but E0 blocks High; (2) consumption proxies excluded from D2: Joyner drops to Low (5→4 Moderate), gap unchanged; (3) TAM/SUS always K2 unless tied to a behavioral mediator: Seiler K3→K2, no D change, gap persists; (4) Moderate requires D≥3: Joyner removed (5→4), gap persists (still 0/35 with K≥2 and D≥2). Across the corpus, K and D are negatively correlated (Spearman $\rho =-0.35$, $p=.04$; computed on raw ordinal scores from Table 7, $n=35$, asymptotic p-value with ties; the correlation is primarily descriptive given ordinal data), suggesting that the complementary gap is a systematic pattern in this corpus rather than an artifact of threshold placement.

7. Discussion and Conclusion

As Hognon et al. [12] argue, evaluation should not be reduced to behavioral outcomes alone: the cognitive processes that serious games activate are themselves dimensions of impact, and K makes this visible. Yet the Theory of Planned Behavior [18] and Kirkpatrick’s hierarchy [16] both treat knowledge as necessary but insufficient for behavioral change. The complementary gap (Section 6) confirms this: without the measurement of actual behavioral outcomes, the field cannot establish whether cognitive gains translate into waste reduction [4]. This gap partly reflects disciplinary silos: HCI and education researchers prioritize learning outcomes (K), while public health and environmental science groups prioritize objective outcome measurement (D) without embedding games. Evidence from the corpus supports this reading: all five Moderate studies (D≥2) are gamification interventions, while all three K3 studies are games or serious games. Early-stage studies scoring K2 or K3 with D0 are not failures; they represent the first half of the measurement chain, and SG-FLW’s prospective mode can help them plan the second.

Inconsistent measurement compounds this problem [4,6]. SG-FLW’s profiles complement both systemic perspectives such as Rossitto et al.’s Digital Environmental Stewardship [56] and game-focused evaluation frameworks (engagement, usability, flow); it assesses only the evidence chain for waste reduction, not whether the game is well-designed. Used prospectively (Section 1), target profiles such as B2/K3/D2/E1/L2 make both K and D visible from the outset, avoiding the pattern where cognitive and behavioral measurement are treated as mutually exclusive.

The framework has limitations, SG-FLW evaluates measurement quality, not intervention quality; a low profile indicates missing evidence, not that the intervention failed. Its ordinal scale does not capture all study design nuances (K, for instance, groups distinct TPB constructs into a single rigor scale), and borderline cases are expected (Table 9); dual coding mitigates but does not eliminate subjectivity. E and L are empirically unanchored in this corpus (all E0; only one L>0), so E1–E3 distinctions remain untested until studies begin to report environmental conversion. Our corpus is drawn from two prior reviews, and the framework was iteratively refined on these same studies; independent application to a different corpus might yield different gap patterns. To mitigate overfitting, each scale level is anchored to observable reporting artifacts (weighing, validated instruments, app logs) rather than corpus-specific features. Reliability (${\kappa }_w=0.96$; Section 6.3) is based on a single coder pair coding 12 of 35 studies; resource constraints prevented full dual-coding, but the purposive subsample spans the full evidence range and includes all borderline cases. SG-FLW does not capture intervention dosage or implementation fidelity, nor participatory outcomes from co-design processes. In continuous-measurement interventions where the instrument is also the game mechanic (e.g., SmartMat), B and D may be conflated through reactivity; this is a known issue in behavioral measurement generally, not specific to SG-FLW. Because the causal chain is domain-agnostic, SG-FLW could serve energy, water, or carbon domains; Hognon et al. [12] identified identical evaluation deficits in climate education.

Future studies should pair cognitive assessments (K) with objective behavioral measures (D≥2) and baselines (B≥2) to test the K→D pathway predicted by Ajzen [18], aligning with the EU’s call for standardized monitoring [9,10]. A minimal pathway to B2/K2/D2/E1/L1 requires: aggregate waste weighing before the intervention (B2), a structured pre/post questionnaire (K2), app-logged or photo-documented waste actions (D2), one published emission factor applied to the waste stream (E1), and a three-month follow-up (L1). None of these steps demands specialized equipment; upgrading to the full target B2/K3/D2/E1/L2 requires only substituting validated instruments for K and extending follow-up to six months. Environmental conversion (E≥1) should become routine for SDG 12.3 progress, and longitudinal follow-up (≥6 months) is needed to separate novelty from sustained change [12]. Reporting participant demographics [57,58] would enable moderator analyses across SG-FLW profiles.