Multi Criteria Decision Analysis Framework for Evaluating Tools Supporting Renewable Energy Communities

1. Introduction

The transition to renewable energy is a central priority of modern energy policy, driven by climate goals, energy security concerns, and declining technology costs. Renewable energy sources (RES) are now recognized as essential for building sustainable and resilient energy systems. At the European Union (EU) level, directives such as the Renewable Energy Directive (EU) 2018/2001 [1] and its amendment Directive 2023/2413 [2], together with the Internal Electricity Market Directive (EU) 2019/944 [3], set binding targets for renewable deployment and promote citizen participation in community-based initiatives.

Within this framework, Renewable Energy Communities (RECs) and Citizen Energy Communities (CECs) have become key instruments for local empowerment, active consumer involvement, and the integration of distributed renewables. Their main purpose is to generate environmental, economic, and social benefits rather than financial profit, ensuring inclusiveness, cooperation with system operators, and fair access to resources. Accelerating REC development is not only a policy goal but also a socio-technical necessity for achieving the EU’s 2040 climate neutrality targets, as decentralized systems enhance resilience and citizen engagement [4].

With the rise of decentralized energy production and ECs, demand for planning and management tools has also increased. They have become essential for efficient, optimized, and well-controlled implementation of energy communities (ECs) within energy systems, providing technological support for system optimization, energy modeling, and data management to ensure effective operation and to help achieve social and economic benefits for the communities. [5]. Various comparisons and evaluations of energy modelling tools (EMTs) their use cases can be found in the recent academic literature. Reviews are usually focused on software tools for urban energy system modelling [6], urban building energy monitoring [7], and on hybrid energy system designs [8].

For instance, the authors of [9] provide a valuable initial overview of digital tools supporting RECs, their analysis is primarily descriptive and limited to qualitative classification across three REC implementation phases: design, creation, and operation. The evaluation does not apply a formalized scoring system, weighting mechanism, or structured decision-support framework, which restricts reproducibility and comparability across tools. Additionally, the scope of assessment omits key functional areas that are increasingly critical for REC implementation, such as EV integration, demand response aggregation, user-centric design features, and support for socio-legal frameworks. Notably, no attention is given to usability aspects, participatory tools, or how tools address localized regulatory or governance contexts—all of which are essential for real-world community adoption and scalability. As such, while the study offers a broad typological landscape of available tools, it leaves unaddressed the question of which ones perform best under varying user priorities or functional requirements.

The authors of [10] proposed a qualitative evaluation approach for energy system modelling frameworks, examining properties like transparency (open-source), collaborative potential, and structural flexibility. Their approach is tailored to the structural and philosophical characteristics of modelling tools—primarily at the developer and framework level. It lacks consideration of emerging themes crucial to REC tools—such as social participation, regulatory adaptation layers, and open interface usability.

A systematic review in [11] offer a timely and structured review of twelve EMTs relevant to RECs, their focus lies predominantly on the representation of data inputs, simulation features, and technical outputs across a general REC planning workflow. Although the study identifies which tools support techno-economic and, to a lesser extent, spatial and environmental modelling, it does not apply a formal scoring system or weighting mechanism to evaluate tool performance systematically. Their assessment is qualitative in nature and does not distinguish the relative importance of each criterion. In addition, the review highlights—but does not deeply explore—the lack of social participation tools, EV integration, or user-oriented outputs such as dashboards and governance support. Moreover, the authors do not benchmark tools using real-world operational criteria such as interoperability, usability, or readiness for community deployment, leaving a gap in actionable decision support for creators of RECs.

The paper [12] emphasizes that most academic modelling of EC focuses primarily on technical and economic outputs, often neglecting dimensions such as co-creation, educational support, and citizen empowerment. This mirrors gaps identified in digital tool assessments, where features enabling collaborative design, shared decision-making, and local capacity-building are rarely incorporated. The authors of [13], through qualitative interviews, further illustrate the evolving role of digital mediation within RECs. Their study finds that while tools increasingly support technical functions like monitoring and demand response optimization, stakeholders express a clear demand for more participatory and relational features. These include capabilities for peer knowledge exchange, shared project initiation, and coordination with external institutional actors—functionalities largely absent in current tool offerings.

The article [14] offers a technically detailed review of open datasets and tools for local ECs, serving as a valuable technical catalogue. However, it lacks a systematic evaluation of platform usability and practical applicability from a stakeholder perspective. Key dimensions such as user participation, regulatory adaptability, and educational support are overlooked, and no structured comparison or decision-support framework is provided to guide tool selection based on multidimensional performance.

Across the literature, a range of methodological approaches have been used to evaluate and compare EMT. A common approach is feature-based benchmarking: many reviews compile a checklist of functionalities and assess which tools have which features. For example, [9] mapped each tool against services needed in each project phase (design, creation, operation), essentially creating a feature matrix to identify strengths and voids. Similarly, [11] compared tools by the types of inputs they require and outputs they produce (technical, financial, environmental, spatial), using a structured comparison framework. These studies rely on document analysis and tool documentation to score capabilities, often presented in comparative tables. Another technique employed is multi-criteria decision analysis (MCDA) or scoring frameworks. Some reviews explicitly rank or score tools against multiple criteria. For instance, broader energy system model evaluations have introduced qualitative scoring on dimensions like transparency, complexity handling, and collaboration support. According to [15], stakeholder integration in energy system modelling remains limited, as most frameworks still prioritize technical and economic performance over participatory and social dimensions. Their systematic review, based on a SWOT analysis of more than eighty studies, highlights that current modelling tools rarely include mechanisms for co-design, user feedback, or stakeholder interaction throughout the modelling process. The authors argue that this technocentric orientation reduces the applicability of such tools for community-level energy planning and decision-making. They identify opportunities for improvement through the adoption of hybrid approaches—combining MCDA, participatory modelling, and digital interfaces—to better capture local priorities, social acceptance, and behavioural factors. These findings reinforce the relevance of incorporating social and participatory criteria into the KPI framework used to evaluate REC tools.

Some researchers have taken a case study simulation approach to tool evaluation, which are applied to a common scenario to compare results and user experience. For example, [16] conducted a practical comparison of an optimization model vs. a simulation model for the same municipal energy system, highlighting differences in outcomes and modelling effort. Though not focused on REC tools per se, this demonstrates the value of side-by-side case studies to reveal how tool choice can influence planning recommendations.

Another important methodological dimension concerns the integration of stakeholder perspectives and usability evaluations into tool assessment. Although relatively uncommon, some studies have included surveys or interviews to capture user feedback regarding the functionality and applicability of EMTs. For instance, the Finnish study on digital mediation within REC collected practitioner insights on missing or underdeveloped platform features, highlighting the need for participatory design and relational functionalities [13]. Moreover, the authors of the paper [15] have called for the inclusion of user-centred evaluation criteria—such as interface usability, documentation quality, and learning curve—within tool benchmarking frameworks. Nevertheless, quantitative usability metrics such as time required to configure simulations, frequency of operational errors or user satisfaction ratings, remain largely absent from the comparative literature. This methodological gap indicates that most reviews still assess tools based on documented functionalities rather than empirical user testing, limiting the understanding of real-world performance and accessibility.

Despite the proliferation of digital tools designed to support RECs, existing evaluations remain fragmented and limited in scope. Prior studies have primarily focused on technical modeling capabilities or high-level descriptions of tool functionalities, often omitting critical dimensions such as environmental performance, social engagement, and regulatory adaptability. Key reviews in the field have revealed several recurring gaps: the limited integration of electric mobility and multi-vector energy systems, insufficient support for participatory design and citizen interaction and an overarching emphasis on feasibility indicators at the expense of broader sustainability and usability considerations. Moreover, while some frameworks qualitatively assess platform features, they often lack reproducible scoring methodologies, structured prioritization of evaluation criteria, and actionable insights for community stakeholders.

To address these gaps, this study introduces a structured and reproducible evaluation framework that can rank REC tools according to multiple stakeholder-valued criteria. The framework combines key performance indicators (KPIs) with MCDA. The framework spans six functional criteria—technical, operational, economic, environmental, social, and quality and adoption—disaggregated into 23 measurable sub-criteria. The framework applies a stakeholder weighted aggregation using the weighted sum model (WSM) [17] to capture the priorities of non-expert community members. Each criterion is scored on a normalized scale, and aggregated scores are calculated using both expert-informed and data-driven weighting methods. This dual-layered approach enables sensitivity analysis to reflect varying stakeholder priorities and enhances transparency in tool comparison.

The main contributions of this paper are summarized as follows:

1. a comprehensive KPI–MCDA framework for evaluating REC software across technical, economic, environmental, social, usability, and governance criteria is presented;

2. incorporation of multiple weighting methods (stakeholder-derived vs. data-driven) to enhance transparency and replicability is used;

3. demonstration of the framework on representative tools to highlight underexplored features such as EV integration, participatory co-design modules, usability, transparency metrics, and legal adaptability is offered.

By structuring evaluation in this way, our approach moves beyond prior studies to provide actionable, multidimensional performance scores that reflect the diverse needs of REC practitioners and community stakeholders.

The remainder of the paper is structured as follows: Section 2 presents the methodology, including tool-selection criteria, the KPI calculation and the MCDA process; Section 3 reports the evaluation results across a sample of tools; the discussion and conclusions are summarised in the last two sections.

2. Methodology

Most tools were sourced from peer-reviewed articles, reviews, and EU project deliverables. To capture practice-oriented tools under-represented in journals, the authors complemented the literature scan by systematically querying official project and vendor websites, documentation portals, user manuals, open repositories, demo or sandbox instances, and publicly available installers using controlled keywords (e.g., “REC tool/platform,” “energy sharing software,” “community microgrid,” “EV smart charging for RECs,” “REC toolkit”).

Building on this corpus, the authors conduct a structured desk-based [18,19] evaluation of digital tools that support the design, creation, or operation of RECs. The objective is to compare alternatives on a common, multi-criteria basis that reflects the needs of a non-expert audience (for inst. consumers, or/and community initiators), while remaining transparent and reproducible.

The methodology has two pillars:

• A KPI framework structured across six dimensions
• A scoring rubric with normalized 0/0.5/1 criteria and stakeholder-weighted aggregation via (MCDA), using a WSM [20,21,22] as the primary ranking method.

MCDA methodologies have been widely applied across areas including renewable energy planning, transportation, sustainability assessment, quality management, and supply chain optimization. Their adaptability allows researchers to customize criteria and weighting systems to adapt the methodology to specific context of research [23]. MCDA provides a structured and systematic framework for managerial decision-making by explicitly incorporating multiple criteria or objectives into the evaluation and ranking of alternative options. MCDA formalizes the decision-making process, thereby enhancing both the quality of decisions and the quality of decision-making practices, by explicitly documenting the applied criteria, their relative weights, and the rationale behind scoring [24].

The paper reviews digital tools, software tools, and toolkits that support the assessment, planning, setup, or operation of RECs, including both European and global cases.

Study selection was guided by two eligibility criteria designed to ensure that each entry could be evaluated transparently and on equal methodological footing. First, a platform or tool had to be documented through an official, publicly accessible source—such as a project website, product page, user or administrator manual, or open repository—detailing functionalities relevant to REC. Second, the platform had to be accessible for independent inspection, either via a public demo or sandbox, a time-limited trial, an openly available installer or codebase, or sufficiently detailed technical documentation to enable functional verification without vendor mediation. Solutions were excluded when they failed to satisfy either of these two conditions.

The KPI assessment methodology includes five consecutive steps for evaluation of each tool (Figure 1).

First, the KPI definition step identifies the main performance domains, that describe the tool’s functional and qualitative characteristics. Second, the ranking of criteria specifies the most relevant sub-criteria within each domain and organizes them according to their importance. Third, the evaluation of each sub-criterion within every KPI dimension is performed, reflecting the tool’s actual features, functionality, and performance. Fourth step consists of the weight coefficient calculation that determines how significant each criterion is compared to the others, applying expert judgment and objective data-driven methods. Finally, during the WSM implementation, all weighted scores are combined into a single overall index, which allows comparing the tools and identifying the most effective one.

2.1. KPI Definition

Based on the analysis of existing studies on KPI frameworks [25,26,27,28,29] for energy systems and community-oriented, six main KPI dimensions were defined to evaluate digital tools for REC: technical modelling, operational, economic, environmental, social, and quality and adoption (Figure 2).

2.2. KPI Sub-Criteria

The second step focuses on identifying and structuring detailed criteria within each KPI dimension.

2.2.1. Technical Modelling

In the technical modelling (TECH) KPI dimension, the authors identified and evaluated a set of sub-criteria that capture the platform’s ability to represent the technical complexity of RECs and their multi-energy interactions. These sub-criteria address the modelling depth, analytical scope, and realism of technical functionalities.

The criterion “Energy vectors” (TECH_vec) [30] assesses the extent to which a platform models multiple energy carriers and their couplings within a community context. Multi-vector capability is essential for representing sector coupling (e.g., electricity-to-heat via heat pumps, combined heat and power, thermal storage, and electric-vehicle smart charging), thereby enabling integrated techno-economic assessment rather than electricity-only appraisals. A platform scores higher when it can model several carriers and their interactions, rather than electricity alone.

The criterion “Optimisation” (TECH_opt) evaluates whether the platform goes beyond basic simulation to offer design or operational optimization, and whether the optimization objective space is single- or multi-dimensional.

“Simulation capability” (TECH_sim) criterion evaluates the temporal fidelity and breadth of the simulation engine used for techno-economic assessment and operational studies. Higher capability entails time-series simulation over full annual cycles, sub-hourly granularity where needed (e.g., EV/BESS control), and consistent mass-/energy-balance handling across coupled vectors [31].

“Forecasting” (TECH_forec) criterion assesses whether the platform provides endogenous forecasting of key time series relevant to REC planning and operation (e.g., load/consumption, renewable generation).

The “LV/MV grid constraints or losses” (TECH_grid) criterion addresses the representation of distribution-network feasibility—voltage bounds, thermal limits, losses, reverse power flows, and curtailment—at low and medium voltage levels. For RECs, distribution constraints often determine admissible asset sizing and operational envelopes [32].

“Spatial/GIS capabilities” (TECH_spat) criterion evaluates geospatial awareness and place-based modelling, including building-level siting, roof orientation and shading, community perimeter rules (e.g., same-substation constraints), and proximity to thermal networks or other infrastructures. Spatialized modelling is a prerequisite for actionable planning and compliance with jurisdiction-specific REC boundaries.

2.2.2. Operations and Control

Operations and control (OPER) dimension assess whether a platform is ready for day-to-day operation of an EC: running diverse assets, ingesting telemetry, producing actionable analytics and reports, settling energy sharing, exposing reliable interfaces, and coordinating flexibility

The criterion “Asset classes” (OPER_ascl) valuates the scope and diversity of assets that the platform can natively model, monitor, or control within an energy community framework. It reflects both the breadth (variety of asset types) and depth (level of technical detail and control granularity) of device integration [33].

“Analytics and reporting” criterion (OPER_analyt) [34] evaluates the platform’s data analytics, performance assessment, and reporting capabilities, which are essential for operational optimization and strategic decision support. It covers the transformation of raw telemetry data into actionable insights, including statistical analysis, KPI tracking, anomaly detection, and trend forecasting. Advanced tools integrate predictive or prescriptive analytics, provide automated performance summaries, and enable custom report generation for various stakeholders (operators, policymakers, or community members).

“Demand response and flexibility aggregation” (OPER_flex) [35] assesses the platform’s ability to aggregate and activate flexibility from distributed energy resources (DERs) and controllable loads—including batteries, heat pumps, and smart appliances. It focuses on how effectively the system can enrol flexible assets, predict available flexibility, nominate resources for activation, and execute automated control strategies in response to internal signals or market events.

“EV management” sub-criterion (OPER_EV) evaluates the platform’s native capabilities for EV charging management, including smart-charging policies, schedule configuration. Emphasis is placed on the coordination of charging with price/tariff signals, renewable generation forecasts, and distribution-network constraints, as required for cost-effective and grid-compliant operation in energy communities.

2.2.3. Economic

Economic KPI dimension (ECON) evaluates a platform’s capability to support techno-economic assessment and market realism [36].

“Financial indicators” (ECON_fin) assesses whether the platform implements standard project-finance indicators [37] for community-scale assets and portfolios, enabling rigorous techno-economic appraisal and comparability across scenarios.

The sub-criterion “Tariff/market models” [12] (ECON_tar) evaluates how realistically the platform represents end-user tariffs and market price signals—from flat/static rates to time-of-use, dynamic wholesale/retail, or real-time pricing—since tariff fidelity materially affects REC economics and operational strategies.

“Sensitivity analysis” (ECON_sens) tests robustness of techno-economic results to variation in key drivers (prices, load/generation, discount rate, etc), via batch “what-if” runs or formal uncertainty modules.

“Benefit-sharing calculators” [38] (ECON_shar) whether the platform provides transparent and configurable mechanisms for distributing collective benefits and costs within the REC. It focuses on how energy, financial savings, and operational costs are allocated among members based on predefined or dynamic sharing rules.

2.2.4. Environmental

Environmental (ENVIR) [28] KPI dimension evaluates the platform’s capability to quantify, monitor, and optimize the environmental performance of RECs. Its purpose is twofold: first, to assess what environmental impacts are measured, focusing on robust and transparent carbon accounting; and second, to examine how these metrics influence decision-making through the integration of environmental objectives or constraints into system design and operational optimization.

“Carbon accounting” (ENVIR_carb) sub-criterion verifies whether the tool quantifies greenhouse-gas impacts of REC designs/operations with sufficient temporal and geographic resolution.

Environmental objective support (ENVIR_obj) Assesses whether environmental performance is treated as a first-class decision driver. Mature tools embed emissions (or emission intensity) as a design/operation objective alongside cost and reliability, or as binding constraints. This enables transparent trade-off analysis and policy-aligned planning.

2.2.5. Social

This KPI dimension evaluates how effectively the platform supports transparent participation, communication, and decision-making within the REC [39]. It focuses on whether members have clear visibility of data, results, and impacts, as well as access to interactive tools or dashboards that promote understanding and engagement. The dimension also examines the presence of mechanisms for capturing member preferences, feedback, and co-design inputs, ensuring that community decisions reflect user priorities rather than purely technical optimization. Additionally, it considers whether the platform provides built-in guidance, tutorials, or simplified interfaces that help users participate meaningfully without requiring advanced technical skills.

The sub-criterion “Member portals and transparency dashboards” (SOC_trans) addresses user-facing transparency. It evaluates how clearly the platform communicates energy, financial, and environmental data to community members through dashboards, portals, and reports. This criterion focuses on member-level visibility and trust, ensuring that users can understand and verify community performance and decision-making outcomes.

The “Co-design features” (SOC_des) sub-criterion assesses the platform’s capability to actively involve community members in planning, decision-making, and operational processes through participatory design functionalities.

“Education” (SOC_educ) sub-criterion evaluates the platform’s ability to educate, guide, and support users—both community members and administrators—through a combination of integrated assistance tools and external documentation. It measures how effectively the platform lowers technical barriers and ensures that users can understand, operate, and expand the system confidently.

2.2.6. Quality Indicators

Quality indicators (QUAL) KPI dimension evaluates the overall maturity, robustness, and user-readiness of the platform supporting RECs. It focuses on how well the tool performs in practical implementation, ensuring reliability, usability, and long-term sustainability [40]. The dimension consists of six sub-criteria described below.

Usability (QUAL_us) sub-criterion assesses how intuitive and user-friendly the platform interface is for different user groups (e.g., administrators, members, operators). Tools that offer multilingual interfaces, contextual help, and user-tailored dashboards demonstrate enhanced usability.

The sub-criterion “Reliability and performance” (QUAL_perf) evaluates the platform’s technical robustness, responsiveness, and stability under different operational conditions. It includes aspects such as system uptime, error handling, data integrity, and computational efficiency. Tools that maintain consistent performance during peak data loads or simulation runs, and that provide redundancy and backup mechanisms, score higher.

The “Openness” (QUAL_open) refers to the technical and architectural openness of the platform. It evaluates whether the system provides open-source access, transparent algorithms, well-documented application programming interfaces, and compliance with interoperability standards. The focus is on developer- and integrator-level transparency—ensuring reproducibility, interoperability, and long-term vendor independence.

“Scalability” (QUAL_scab) assesses the platform’s capacity to scale in terms of data volume, number of users, and integration scope. It examines whether the system can handle larger communities, new asset types, or expanded geographic coverage without performance degradation.

2.3. KPI Calculation and Tool Evaluation

In this study, each sub-criterion is scored on a three-level scale 0/0.5/1, where 0 denotes no support (or no verifiable evidence), 0.5 denotes partial support, and 1 denotes full support with documented/demonstrated evidence.

The dimension-average KPI score for tool i is computed as the arithmetic mean over its sub-criteria [25]:

$${\overline{KPI}}_{i,\mathrm{d}}={\displaystyle \frac{1}{n_d}}·\sum _{j=1}^{n_d}{x}_{ij}^{\left(d\right)}$$

(1)

where d is the dimension label (d ∈ {TECH, OPER, ECON, ENVIR, SOC, QUAL}); n_d is the total amount of sub-criteria of dimension d; j is the index of the sub-criterion within dimension d (j ∈ {1, …, n_d}); i is the index of the tool (i ∈ {1, …, I}); $x_{ij}^{\left(d\right)}$ is sub-criterion score for dimension d of tool i on sub-criterion j, ({0;0.5;1}). Descriptions of the scoring levels for every sub-criterion in each dimension are provided in Appendix 1.

To compute the overall performance score Si for the i-th evaluated platform, the WSM [41,42], is applied. This model aggregates the normalized performance scores from each of the six defined KPI dimensions, according to their assigned weight coefficients w_b:

$$S_i=\sum _{d\in \{TECH,OPER,ECON,ENVIR,SOC,QUAL\}}{w}_d·{\overline{KPI}}_{i,d}$$

(2)

where $w_b$is the weight of dimension d, and $\sum w_b=1$.

2.4. Weight Coefficient Determination

In this review the authors have employed four well-established approaches to determine the weighting coefficients [43] w_d for the six offered KPI dimensions: rank order centroid (ROC) [44,45] analytic hierarchy process (AHP, pairwise comparisons) [46,47,48], entropy weight method (EWM) [49] and the criteria importance through intercriteria correlation (CRITIC) [50]. These methods were chosen for their reproducibility, transparency, and firm grounding in MCDA literature [51]. ROC and AHP approaches use the authors expert judgment in a structured manner, while the EWM and CRITIC derive weights objectively from the variability and correlation of the data. All four methods produce a normalized weight vector (summing to 1), which is then used to weight each dimension’s contribution in the overall performance evaluation by WSM model. The comparative analysis of different weight sets is provided in Section 3.

2.4.1. ROC

In this case, the ROC method serves as a transparent, non-compensatory weighting approach suitable in the absence of stakeholder-derived numerical weights. It relies on the assumption that decision-makers (here, the authors) can establish a rank order of KPI dimension importance (from 1 to 6), even if they cannot specify precise magnitudes of difference between them.

To determine the final ranking, the authors agreed to use the average rank method, which is widely applied in multi-criteria analysis. The formula for calculating the average rank for each criterion is [52]:

$${\overline{R}}_c={\displaystyle \frac{\sum _{a=1}^{Auth}{R}_{ca}}{Auth}}$$

(3)

where ${\bar{R}}_c$ is the average rank of criterion $c$; $R_{ca}$ is the rank assigned to criterion $c$ by author $a$; Auth is the number of authors.

For C criteria ranked from 1 (most important) to C (least important), the ROC weight for criterion c of dimension d is [44]:

$$w_c^{\left(d\right)}={\displaystyle \frac{1}{C}}·\sum _{r=c}^C{\displaystyle \frac{1}{r}}$$

(4)

where C is the total number of criteria (ranges); r is the index of rank positions starting from c to C (r ∈ {1, …, C}).

2.4.2. AHP

In this case, AHP was applied as a structured expert judgment method in the absence of stakeholder surveys. The research team constructed a 6x6 reciprocal pairwise comparison matrix A, covering the six KPI dimensions, using a simplified Saaty scale [53]. Each matrix element $a_{dr}$ reflects the perceived importance of dimension d relative to dimension r.

$$A=\left[\begin{array}{cccccc}1& a_{dr}& a_{dr}& a_{dr}& a_{dr}& a_{dr}\\ {\displaystyle \frac{1}{a_{dr}}}& 1& a_{dr}& a_{dr}& a_{dr}& a_{dr}\\ {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& 1& a_{dr}& a_{dr}& a_{dr}\\ {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& 1& a_{dr}& a_{dr}\\ {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& 1& a_{dr}\\ {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& {\displaystyle \frac{1}{a_{dr}}}& 1\end{array}\right]$$

(5)

where A is the pairwise comparison matrix of the dimensions, where $a_{dr}$>0, $a_{dr}$=0, $a_{dr}$=1/ $a_{dr}$.

The weight coefficient by AHP method is calculated as follows [46]:

$$w_d={\displaystyle \frac{(\prod _{r=1}^{n_d}{a}_{dr}{)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$n_d$}\right.}}{\sum _{r=1}^{n_d}(\prod _{r=1}^{n_b}{a}_{kr}{)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$n_d$}\right.}}},$$

(6)

where a_dc indicates how much more important dimension d is compared to dimension r; $(\prod _{r=1}^{n_d}{a}_{dr}{)}^{{\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$n_d$}\right.}}$ is geometric mean of row d.

2.4.3. EWM

The fundamental idea of EWM [54] is that a dimension provides more decision-making value if its values are more dispersed (less uncertain or more informative). Conversely, dimensions with uniformly distributed scores are considered less informative and are therefore assigned lower weights. Because it uses only data (no expert judgements), EWM was selected to cross-check subjective schemes (ROC and AHP) in hybrid weighting.

The normalized weight for each KPI dimension by EWM is evaluated as follows [55]:

$$w_b={\displaystyle \frac{d_d}{\sum _{k=1}^{n_d}{d}_d}}$$

(7)

where k is an auxiliary index, k ∈ {1, …, n_b}; $d_d$ is the divergence of each dimension b and computes as:

$$d_d=1-E_d$$

(8)

where E_d is the entropy of dimension d. Dimensions with higher divergence are considered more informative.

The entropy is defined as follows:

$$E_d=-{\displaystyle \frac{1}{\ln I}}\sum _{i=1}^I{p}_{id}·\ln p_{id}$$

(9)

where $p_{id}$ represents the normalized proportion of tool i performance on KPI dimension d, relative to all other tools for that dimension.

Thу (9) expression is bounded in [0,1], where E_d = 1 implies a uniform distribution (low discriminating power), but E_d = 0 implies high contrast (high information content).

The normalized proportion of tool i performance on KPI dimension b is calculated as follows:

$$p_{id}={\displaystyle \frac{{\overline{KPI}}_{id}}{\sum _{i=1}^I{\overline{KPI}}_{id}+\epsilon }}$$

(10)

where ε is a small positive constant (in this case, ε=10-12) to prevent division by zero. This normalization ensures $\sum _i^I{p}_{id}=1$, mimicking a probability distribution over tools for each dimension.

Entropy captures the contrast intensity or information richness of each criterion. The greater the dispersion of data, the higher the criterion’s ability to distinguish alternatives, and thus the higher its assigned weight. As a result, EWM tends to assign the highest weights to criteria that best differentiate the performance of alternatives—those that contain the most “information” about the decision space.

2.4.4. CRITIC

The CRITIC method [50] is another objective approach that assigns weights by considering both the contrast intensity (standard deviation) and conflict (correlation) among criteria. It emphasizes criteria that vary strongly and are less correlated with others [56].

The weight for each KPI dimension by CRITIC is evaluated as follows:

$$w_d={\displaystyle \frac{C_d}{\sum _{k=1}^{n_d}{C}_k}}$$

(11)

$C_d$ is the information content (contrast intensity) of the dimension d; $C_k$ is the information content of the dimension k.

The ${С}_d$ parameter is calculated as afollows:

$${\mathrm{С}}_d={\sigma }_d·\sum _k^{n_d}(1-p_{dk})$$

(12)

where ${\sigma }_d$ is the standard deviation of dimension b; ρ_dk is the Pearson correlation [57] between dimensions d and k. The correlation analysis between dimensions is calculated by vector ${KPI}_{id}$.

The ${\sigma }_d$ is evaluated by Equation (13):

$${\sigma }_d=\sqrt{{\displaystyle \frac{1}{I-1}}·\sum _{i=1}^I{({\overline{KPI}}_{id}-{\mu }_d)}^2}$$

(13)

where

$${\mu }_d={\displaystyle \frac{1}{I}}·\sum _{i=1}^I{\overline{KPI}}_{id}$$

(14)

The Pearson correlation between dimensions is estimated as:

$$p_{dk}={\displaystyle \frac{\sum _{i=1}^I({\overline{KPI}}_{id}-{\mu }_d)·({\overline{KPI}}_{ik}-{\mu }_k)}{\sqrt{{\left(\sum _{i=1}^I({\overline{KPI}}_{id}-{\mu }_d)\right)}^2}·\sqrt{{\left(\sum _{i=1}^I({\overline{KPI}}_{ik}-{\mu }_k)\right)}^2}}}$$

(15)

3. Results

3.1. Overview of the Selected EMTs

The objective of this section is to present and interpret the comparative results of the evaluated software tools across the six defined KPI dimensions. Each tool was assessed based on its performance across 23 sub-criteria spanning technical, operational, economic, environmental, social, and quality-related dimensions. Scoring followed the normalized 0/0.5/1 scale described earlier, enabling consistent comparison and aggregation across functionally diverse tools.

Based on the defined selection criteria, Table 1 provides an overview of the selected software tools, including their primary goals, applicable spatial scale, and scope. The tools’ focus areas range from microgrid dispatch optimization to municipal-scale REC planning, reflecting the diversity of functionalities and use cases relevant to community energy systems.

The tools span a broad spatial spectrum—from individual households and microgrids (e.g., MiGRIDS Lite, OpenEnergyMonitor) to municipal and district-level communities (e.g., LocalRES, eNeuron), and even national or cross-border implementations (e.g., Powerledger, Energy Web, REScoopVPP). Several tools, such as OpenEMS or Hive Power FLEXO, explicitly support scalability from small communities to industrial or grid-level integration. Tools such as eNeuron, LocalRES, and Cleanwatts emphasize multi-vector optimisation (electricity, heat, mobility), while others like Powerledger, and Energy Web focus on market mechanisms, trading, and blockchain infrastructure. Tools like GreenPocket and REScoop Energy Community Platform emphasize community engagement, transparency, and governance, while Rectool and PROCSIM focus on planning and dataset-based modelling.

3.2. Results by Methodological Step

This section presents the outcomes of the KPI-based evaluation following the sequence of methodological steps defined in the assessment framework. Each step reflects a specific stage of the analysis—from KPI definition and ranking of sub-criteria scoring, KPI dimension’s weighting, to final evaluation of each tool.

3.3. KPI Results

According to Equation (1), the dimension-average KPI score for each tool is computed. Results of the scoring levels for every sub-criterion in each dimension are provided in Appendix 2. The results of KPI scores for each considered tool is presented in Figure 3.

Figure 3 shows that KPI values differ substantially across the tools. Some tools clearly outperform others in specific categories, while a few demonstrate a more balanced profile. For example, solutions such as BECoop, MiGRIDS Lite, LocalRES and PROCSIM appear to specialize in narrow functional areas, achieving high scores only in selected KPI criteria and low scores in others. This specialization suggests that these tools are tailored for specific use cases rather than offering comprehensive functionality across all dimensions.

Tools such as eNeuron, CityxChange, ClearWatts, Hive Power FLEXO and Powerledger exhibit consistently high scores in almost every criterion. This means that they offer well-balanced functionality, combining robust engineering capabilities with sustainability and user adoption features, making them well-rounded solutions.

By contrast, other tools show more uneven profiles. For example, OpenEMS, REScoop VPP and Energy Web have noticeably lower scores across multiple criteria, but high results in certain categories.

Overall, the results confirm that top-performing tools do comprehensive values across all dimensions, making them universal and strong candidates for integrated energy community solutions.

3.4. Ranking of KPI Dimension

In this paper, all authors independently proposed their rankings for six evaluation criteria that are essential for decision-making in the studied context. Each author assigned ranks from 1 (most important) to 6 (least important). The purpose of this stage was to consolidate these individual rankings into a unified order of importance, which would later serve as the basis for calculating weight coefficients for each criterion. According to the methodology described in Subsection 2.1, to determine the final ranking, the authors agreed to use the average rank method, which is widely applied in multi-criteria analysis. The collected data and results of consolidation method is summarized in the following Table 2.

By applying a transparent mathematical approach, the team ensured that the final ranking reflects the collective expertise of all contributors.

3.5. Results of Weight Coefficients

The final the results of all weight calculations, based on the Section 2, are presented in Table 3. A constructed a 6×6 reciprocal comparison matrix by AHP method is given in Appendix 3.

The sum of the weight coefficients assigned to all KPI dimensions equals 1.0 for each of the four methods, confirming that the normalization was performed correctly, and that each method maintains a valid weight distribution across all evaluated criteria.

Table 3 demonstrates that the ROC weights align with the established priorities. TECH and OPER dimensions have the highest weights—0.4083 and 0.2417 respectively, confirming their critical role in the decision-making framework. Mid-ranked dimensions such as QUAL (0.1583) and ECON (0.1028) add moderate value, while SOC (0.0611) and ENVIR (0.0278) contribute less. This spread shows a clear focus on technical and operational factors, while environmental and social considerations, though included, have a less pronounced effect on the total performance score.

The AHP weights closely follow the consolidated ranking, with TECH (0.3735) and OPER (0.2545) emerging as the most influential dimensions. QUAL (0.1620) and ECON (0.1021) occupy intermediate positions, while SOC (0.0650) and ENVIR (0.0430) remain the least significant. This distribution confirms the strong emphasis on technical and operational dimensions, while still incorporating qualitative and sustainability considerations into the decision-making framework.

EWM produced a distribution that differs significantly from that of expert-based methods. ENVIR (0.3386) and ECON (0.2369) emerged as the most influential criteria due to their high variability, while TECH (0.1328) and OPER (0.1262) received moderate weights. SOC (0.1040) and QUAL (0.0614) contributed the least. This outcome highlights the value of incorporating data-driven weighting to complement subjective assessments and strengthen the overall evaluation’s robustness.

The last method, CRITIC produced a criterion with high variability and low correlation. ENVIR (0.2046) and SOC (0.1811) emerged as the most influential, reflecting their distinctive informational contribution. OPER (0.1643) and QUAL (0.1591) received moderate weights, while ECON (0.1536) and TECH (0.1373) were the least significant. This outcome demonstrates how CRITIC complements other methods by highlighting criteria that reduce redundancy and enhance the robustness of the overall evaluation.

ROC and AHP methods pointed to TECH and OPER as the top priorities. That pretty much matches what the authors already thought was important. EWM and CRITIC methods work a bit differently since they base their weights on the data itself and are data-driven techniques. These methods identified the ENVIR and ECON domains as the most significant, emphasizing the role of environmental and economic variability in the dataset. This really puts the spotlight on how much environmental and economic data can shift the results.

The differences between the weighting methods reveal that ROC and AHP reflect broader strategic or expert-defined priorities, while EWM and CRITIC uncover insights rooted in the intrinsic patterns and variability of the data.

3.6. Results of Final Score

Figure 4 represents the overall performance scores of the tools under four different weighting methods. Each bar represents the KPI score sum for a tool, calculated using the WSM and normalized weight sets (see Equation 2). The figure provides a visual comparison of how the choice of weighting method influences the overall performance assessment.

Tools such as eNeuron, CityxChange, Cleanwatts, Hive Power FLEXO and Powerledger consistently achieve the highest scores using all four weighting schemes. This indicates that these tools deliver balanced performance across technical, operational, economic, and sustainability criteria, regardless of the weight approach.

While the overall ranking of leading tools remains stable, differences appear in mid- and low-performing tools. For example, BECoop, MiGRIDS Lite, REScoopVPP, Compile Toolbox and Rectool Simulator show noticeable variation between ROC and AHP methods and data-driven methods (EWM, CRITIC). This suggests that expert judgment emphasizes technical and operational aspects, whereas objective methods highlight variability in environmental and economic indicators. ROC and AHP consistently assign greater weight to technical and operational KPIs, resulting in higher scores for tools with strong engineering and control capabilities.

Under EWM and CRITIC methods, tools with strong environmental and economic performance—such as (+)CityxChange, Cleanwatts, Powerledger, Energy Web and LocalRES—gain relative advantage compared to expert-based weighting. This reflects the higher variability and lower correlation of these criteria in the dataset, which prioritize objective methods.

4. Discussions

This paper presents a structured KPI–MCDA evaluation of digital tools supporting RECs, aiming to close existing gaps in platform assessment methodologies. Previous literature has primarily focused on technical and economic functionalities, often omitting critical aspects such as user engagement, regulatory adaptability, EV implementation or platform usability. The results confirm that while several tools perform well across multiple functional dimensions, many others show strong performance in only a narrow set of criteria, which limits their broader applicability in real-world community contexts.

During the analysis, several methodological and practical considerations emerged that shaped the final approach and should be considered when interpreting the results. Firstly, the selection of the pairwise comparison scale for the AHP component required careful deliberation. The authors evaluated two options: the discrete odd-number scale commonly used in AHP (e.g., 1, 3, 5) and broader ranges such as 1–5 or 1–9, as proposed in classical AHP literature. While a 1–9 scale offers granularity, empirical observations indicated that the perceived differences in criterion importance across tools were moderate rather than extreme. Therefore, to avoid overstating the influence of marginal differences, the authors opted for a limited 1–5 scale. This compromise balances methodological rigor with real-world interpretability and reflects the underlying distribution of expert judgment. Another practical issue concerned the uniqueness of pairwise ratings. After internal deliberation, the authors concluded that permitting repeated scores across criteria was more realistic. In real-world settings, multiple criteria may be viewed as equally important, and forcing strict ordinal rankings could misrepresent true expert preferences. Allowing repeated values helped retain the semantic integrity of expert assessments and avoided introducing artificial precision into the weighting model. Recognizing the inherent subjectivity of expert-derived weights, the authors further triangulated results using two objective, data-driven methods: EWM and CRITIC. These methods rely solely on the observed variability and correlation within the data and offer complementary perspectives.

The authors intend to explore this balance further in future work, focusing on how different combinations of platform functionalities—such as real-time flexibility control, co-design interfaces, and environmental optimization—can be integrated into scalable, user-friendly digital infrastructures. Particular attention will be paid to the role of interoperability standards, open-source architectures, and modular design in enhancing platform adaptability across diverse regulatory contexts. Additionally, future studies will investigate the incorporation of user feedback and empirical testing to refine the scoring framework and better align platform evaluation with the lived experiences of REC stakeholders.

5. Conclusions

The results of this study provide valuable insights that extend beyond the numerical rankings of the evaluated platforms. The consistent top performance of eNeuron, (+)CityxChange, Cleanwatts, Hive Power FLEXO, and Powerledger across all weighting schemes reflects not a methodological coincidence, but the robustness and comprehensiveness of their design. These platforms simultaneously address technical, operational, economic, environmental, social, and quality domains, demonstrating that balanced multi-domain integration, rather than specialization in one area, determines overall excellence in the multi-criteria evaluation context. Meanwhile, the variability observed among mid-ranked platforms such as BECoop, MiGRIDS Lite, REScoopVPP, Rectool, and PROCSIM under different weighting scenarios reveals that the relevance and performance of each tool are highly dependent on stakeholder priorities. This indicates that platform suitability is contextual and should align with whether a project is primarily technology-driven or community-oriented.

From a practical perspective, the findings suggest that tool selection should be guided by project-specific objectives rather than overall ranking. For ECs that prioritise operational control, flexibility, and integration of EV or demand response, tools with strong operational capabilities are more appropriate. Conversely, initiatives focusing on early-stage design and planning would benefit from tools that demonstrate superior capability in feasibility and spatial analysis. The six KPI dimensions proposed in this study can serve as a comprehensive framework for structuring technical specifications, procurement processes, and evaluation protocols, helping to prevent the omission of critical functionalities such as network constraint modelling, energy-sharing mechanisms, or multi-objective optimisation. In addition, the analysis underscores the importance of modularity and interoperability: while universal tools can serve as a system’s core, the integration of specialised solutions through open interfaces and scalable architectures can improve overall system performance and adaptability.

At the policy level, the results highlight the need to embed environmental and economic sharing dimensions in national and regional regulatory frameworks. Since the rankings are most sensitive to these domains under data-driven weighting schemes, policymakers should consider incorporating requirements for carbon accounting, transparent benefit allocation, and lifecycle performance tracking into public funding programmes and pilot project evaluations. Establishing a baseline set of functional requirements for REC digital tools—covering grid constraint modelling, flexibility management, multi-objective optimisation, and user transparency—would reduce fragmentation among projects and ensure interoperability across different governance levels. Furthermore, the use of publicly available KPI reporting and disclosure of weighting schemes would enhance comparability and accountability across municipal and community-led initiatives.