Model for Innovation Project Selection Supported by Multi-Criteria Methods Considering Sustainability Parameters

1. Introduction

Innovation projects are often developed with the aim of meeting customer needs, achieving strategic goals, and ensuring the sustainability of organizational success [1,2]. These projects constitute a fundamental basis for business competitiveness. However, organizations face budgetary constraints while, at the same time, there is an abundance of candidate projects [3,4,5]. In this context, a significant share of organizational performance is directly associated with the proper selection and execution of innovation projects [1,2,6,7].

A crucial proportion of a company’s success is due to the selection and pursuit of the right innovation projects [1,2,6,7]. The portfolio selection problem was first introduced by [8], and this notion has since been addressed by several researchers. It involves the simultaneous comparison of a set of projects in order to achieve an optimal ranking [9,10,11]. In this process, it is important to adopt consistent criteria and align them with the organization’s business strategies [12,13]. Evaluation based on a uniform set of criteria eliminates unfair competition among projects, which may arise when projects are compared using different reasoning for each evaluation [14].

With regard to sustainability, its relevance in the development of innovation projects is evident. Through innovation, many companies are able to incorporate practices directed toward sustainable development. Moreover, organizational results should not be restricted solely to profit generation but must also encompass social and environmental concerns, thereby fulfilling the balance proposed by the triple bottom line: people, planet, and prosperity [15,16]. Thus, integrating sustainability aspects into the project selection process has become an increasing requirement, aligned with stakeholder expectations.

In practical terms, the project selection process should consider consistent criteria that encompass risk, sustainability, organizational, and strategic factors, among others. However, the literature still reveals a gap regarding comprehensive frameworks for innovation project portfolio selection that incorporate sustainable parameters and identify specific criteria for sustainable innovations [4,5,11,17,18,19,20,21,22].

There are discussions pointing out that measuring potential innovation outcomes considering sustainability aspects is a challenging task, as it requires complex analyses involving social, political, and economic issues, which comprise multiple dimensions and diverse perspectives [23,24]. In this scenario, Multi-Criteria Decision Analysis (MCDA) methods emerge as suitable tools to support managers [24,25].

MCDA can be applied to compare actions and rank them according to the different incorporations of sustainability and innovation aspects [26]. It provides a synthesis of knowledge that supports decision-making by systematically exploring the advantages and disadvantages of different alternatives [27,28]. Even when decision-makers hold subjective assessments, it is still possible to classify, select, and rank alternatives according to the criteria considered [24]. Decision support systems have increasingly been used in sustainability-related issues to identify the most suitable alternative by integrating factual information from research or modeling with value-based information collected through stakeholder engagement [27].

Based on this and on the gap in the literature highlighted by [18], an opportunity was identified to develop a model using MCDA methods to assist organizations in selecting or ranking innovation projects while considering sustainability aspects. This article therefore aims to propose a model for innovation project portfolio selection oriented toward sustainability, grounded in multi-criteria decision support methods.

2. Materials and Methods

Given that the present research proposal aims at constructing a comparative project evaluation model, Design Science Research (DSR) constitutes the methodological framework best suited to the proposed objective. Since it is consistent with the proposition of a model and problem-solving, DSR grounds and operationalizes the research process when the intended outcome is an artifact or a prescription [29]. [30] highlights that even if the artifact does not seek an optimal result, it primarily aims to achieve a satisfactory outcome for the problem at hand.

In this regard, the research will employ the Design Cycle methodology, as proposed by [31,32,33,34]. This methodology aims to solve a problem—which may be scientific—through the development of a new model, while also generating knowledge that is applicable and useful [35]. Within the Design Cycle, the proposed solution can be generalized by considering the problem class [31]. In general terms, the methodology can be implemented through the execution of the steps illustrated in Figure 1.

As shown in Figure 1, the methodological procedure for conducting this research is divided into five stages: awareness, suggestion, development, evaluation, and conclusion. The five stages, as defined by [33] and to be implemented in this study, are detailed below, as further explained by [31,32].

2.1. Stage 1 – Awareness

This stage concerns understanding the problem at hand, where a specific issue is identified and delineated. The problem situation was detailed and explored through a literature review to support the arguments and highlight a gap regarding project evaluation and selection considering sustainability aspects. Awareness leads primarily to the identification and formalization of the problem to be solved, defining its boundaries and the satisfactory solutions required.

The research opportunity emerged from the identification of scientific gaps, which were subsequently validated. To deepen the understanding of the research context, a bibliometric study was conducted to analyze the main discussions and theoretical approaches to the application of MCDA methods in innovation and sustainability. The objective was to confirm theoretical gaps, understand the state of the art, and structure a theoretical–conceptual framework.

This bibliometric analysis, carried out following the method proposed by [36] and presented in [18], synthesized the reference literature on the application of MCDA in innovation within the sustainability context. It also reviewed the main MCDA approaches that could potentially be applied, thereby strengthening the theoretical foundations for the subsequent stages of the research.

2.2. Stage 2 – Suggestion

This stage involves developing one or more theoretical options of an artifact to address the identified problem, while also defining the premises and requirements for the construction of the model. It is essential to document all development attempts, decisions made, the exclusion of less relevant alternatives with their justifications, and the potential ethical implications of applying the artifact.

In this stage, a systematic literature review (SLR) was carried out following the approach proposed by [37]. The review aimed to identify the state of the art regarding project selection in innovation considering sustainability aspects, as well as the definition of criteria for innovation and sustainability performance. The literature supported the identification of requirements and premises for the model, highlighted the main approaches used in sustainable innovation project portfolio evaluation, and contributed to the synthesis of a theoretical–conceptual framework.

The review also identified and consolidated criteria suggested by the literature to be incorporated into the model, thus ensuring a robust theoretical foundation. This study is presented in [38].

2.3. Stage 3 – Development

This stage corresponds to the design of the model. According to Simon (1996), it is the point at which the internal environment of the artifact is built, and a specific solution is generated, while the external environment and instantiation were characterized in the previous stages of awareness and suggestion. At this stage, the justification for selecting the MCDA method and the criteria to be adopted is required.

The outcome was the first version of the model, with a generalist character. Based on the list of criteria proposed in the SLR [38], the need for new criteria emerged to jointly assess innovation and sustainability, given the scarcity of such integrative measures in literature. These new criteria were consolidated through expert validation, leading to a final structured list that includes each criterion’s definition, typology, related SDG, and evaluation scale. This process of proposition, validation, and consolidation is detailed [39].

In addition, the most appropriate MCDA approaches were selected considering the consolidated criteria, the literature recommendations, and the specific requirements of the model. The final method selection and model development are presented in Section 3 of this article.

2.4. Stage 4 – Evaluation

This stage is characterized by a thorough verification of the artifact’s performance in the environment for which it was designed, assessing its effectiveness against the proposed solutions. The model was tested in two institutions with innovation projects that either incorporate or seek to incorporate sustainability aspects into their organizational strategy. The implementation process and the results generated were discussed with each institution through meetings, allowing the assessment of the model’s contributions, its ability to meet the intended objectives and targets, its level of adherence, and the identification of necessary adjustments.

The information on the tests and the application of the model is presented below. The project selection model was developed in Microsoft Excel, applying MCDA methods to support the comparison, evaluation, and decision-making regarding innovation projects. The model is available in the supplementary material submitted, and details of its construction are presented in Section 3 of this article. Table 1 shows the characteristics defined for the model’s application.

The model was applied in two organizations, hereafter referred to as Company A and Company B. Table 2 and Table 3 present the main characteristics of both organizations.

All responses collected during the application were treated exclusively within the scope of this research. Data confidentiality and the anonymity of respondents and organizations were strictly ensured. The information provided was used solely for the purposes of this study, in accordance with appropriate academic data management policies. No sensitive or strategic information regarding the organizations’ innovation projects or products in development was requested. Detailed information on the model application is presented in Section 4.

The model was also validated through dynamic analysis and structural testing. During and after the application, its performance, critical analyses, and potential improvements were examined, leading to its validation. Results of the validation process are reported in Section 4. Following [31], artifact validation is approached pragmatically, emphasizing the need for assessment of its effectiveness in addressing the identified problem and supporting decision-making processes.

• The primary objective of the artifact is to evaluate innovation projects to assess their performance against sustainability criteria and to support decision-making and portfolio analysis. The defined goals include: (i) applying the model in at least one organization, (ii) evaluating a minimum of two projects within that organization, (iii) analyzing at least four criteria during the application, and (iv) verifying the behavior of the model with at least two innovation and sustainability specialists from the participating organization(s).
• The artifact can be tested through its application in organizations that implement innovation projects and aim to assess their performance considering sustainability parameters.
• The mechanisms used to measure results include: (i) verification of whether the model provided critical analysis to decision-makers by offering new insights on the topic, (ii) sensitivity analysis of the MCDA method to determine how project rankings change with variations in criteria weights, (iii) evaluation of decision-makers’ perceptions and suggestions regarding the model, and (iv) critical analysis of the criteria and their practical application.

2.5. Step 5 – Conclusion

Finally, the overall process was formalized and communicated. According to [31] (p. 757) this step requires: “synthesizing the main learnings across all phases of the project and justifying the contribution of the work to the relevant Class of Problems.” Upon completion of all previous steps, the contributions achieved were consolidated, project challenges were identified, and suggestions for future research were proposed. This step aimed to address specific problems while providing a learning cycle and expanding knowledge. The identified contributions, difficulties encountered, and proposed future studies are summarized in the conclusions of this article.

3. Project Selection Model

3.1. MCDA Method Selection

The multi-criteria decision support methods selected were TODIM and TOPSIS. More than one method was chosen to provide mechanisms for comparison and to analyze their implications. The MCDA approaches identified in the bibliometric study conducted by [18] as the main ones applied in the context of sustainable innovations are: AHP/ANP, ELECTRE, PROMÉTHÉE, DEMATEL, MAUT/MAVT, TOPSIS, TODIM, and MACBETH. To suit the model and considering how the criteria were structured, only methods used for ranking, in which the evaluation of alternatives is performed via a scale, were considered. Approaches based solely on pairwise comparison (AHP, ANP, ELECTRE, PROMÉTHÉE) were therefore deemed non-adherent. Furthermore, hierarchical methods AHP and ANP were discarded because criterion hierarchy was unnecessary.

For each criterion, a scale was developed, as these scales better fit the analysis, provide a global evaluation parameter, and help organizations situate themselves on a recognized scale, avoiding arbitrary values. The DEMATEL technique was discarded because it does not provide partial ranking of alternatives and is thus not suitable. Fuzzy methods were not considered because they represent a type of reasoning rather than an MCDA method. The MAUT/MAVT methods were excluded since assigning criterion weights requires intense interaction among decision-makers. As weights in this study were attributed based on the perception of a small group of decision-makers rather than a broad organizational discussion, this could critically impact results and limit the applicability of MAUT/MAVT.

Microsoft Excel was adopted as the software for model implementation and MCDA processing because it is simple and easily usable by any organization or individual. Considering the previously mentioned factors—type of alternative evaluation, behavior, weights, and software—other aspects were also considered when selecting the MCDA approaches to ensure adherence to the project selection model and its application requirements.

TODIM is based on describing how people actually make decisions under risk and uncertainty [40], which is convenient for project portfolio analysis, especially for projects that have not yet been executed and for which consolidated information on outcomes and operational variables is unavailable [41,42]. Additional advantages of TODIM include minimizing order reversals, accommodating qualitative and quantitative criteria, and handling precise or imprecise, complete or incomplete input data—features aligned with the developed model. Furthermore, this method uses a value function to minimize extreme scale choices and normalize individual opinions [43].

TOPSIS is a simple tool that simultaneously provides the best and worst alternatives and offers a ranking to understand differences among alternatives [44]. This method helps identify differences in ranking positions and how close or distant alternatives are. However, some disadvantages and criticisms exist: it does not consider the relative relevance of distances between the most positive and negative solutions and exhibits internal ranking inconsistency, meaning small changes in the weighting vector can shift the solution. [45] also note that TOPSIS should be measured on ratio scales rather than interval scales, and sensitivity analysis is limited because any change in an alternative’s performance requires adjusting the criterion weight. It is noteworthy that the traditional TOPSIS method, not the behavioral version, was used.

By employing two MCDA methods, the application of TODIM is expected to compensate for TOPSIS’s weaknesses, allowing sensitivity analysis. The choice not to rely solely on TODIM is justified by the belief that using two methods enables comparisons, understanding, and discussion of different behaviors.

3.2. Model Development

The project selection model was built in Microsoft Excel, as previously mentioned, since it is easily accessible and shareable, and it allowed the implementation of the selected multi-criteria methods. The model can be accessed in supplementary material.

The model contains several worksheets, which will be presented below. The first is a menu with a brief explanation of all the other sheets: Presentation, Initial Organization Data, Project List, Criteria Selection, Criteria Weighting, Preferences, and Results.

The "Presentation" sheet aims to provide the main information for using the model, including its objectives, benefits, and requirements, as well as the main steps for its application. Regarding benefits, the organization can access the most recognized selection criteria recently identified and validated in the literature and may identify new tools and models that assist in the selection and decision-making process to choose innovation and sustainability projects. Furthermore, the organization will be able to evaluate its projects considering essential indicators recognized and validated by literature and the market, thus assessing results across different dimensions.

As requirements, it was mentioned that the organization must have at least a list of two or more product innovation projects, already engage with sustainability or have the intention to address this theme in its projects, and have availability of focal points (members of the innovation project portfolio management team) to participate in one or two virtual meetings.

The first step in applying the model consists of filling in the fields with the organization’s initial data. In addition to identification information, details about the responsible persons, focal points, some information about innovation and sustainability activities, and strategic guidelines are requested. This supports understanding of how the company engages with these themes and assists in the application of the model.

The second step aims to identify and list the product innovation projects to be evaluated. The third step is dedicated to selecting the criteria for evaluating the set of selected projects. Currently, the list includes 56 criteria distributed across the dimensions of environmental, social, economic, strategy, technological and operational capabilities, risk, and stakeholders. The criteria must align with the business and with what is intended to be evaluated in the projects and must be mutually independent in terms of preference. The organization can adapt the criteria, scales, or even suggest new ones.

Options of the most recognized selection criteria recently identified and validated by the literature, as well as by renowned specialists in project management, innovation, sustainability, and strategic innovation management in Brazil and Portugal, were made available. The criteria provided are presented in the supplementary material. For all criteria, a scale was proposed based on the propositions identified in [38], the methodology of PNI [46], and research experience. Additionally, each criterion is associated with at least one SDG, helping the organization determine whether projects align with its operational strategy. Even in cases where key SDGs have not yet been mapped, this relationship can assist in identifying them through the selected main criteria.

As presented by [39], the following considerations are indicated for selection: choose at least one criterion from each cluster group: 1 – environmental and social, 2 – economic, strategy, and technological and operational capacities, 3 – risk and stakeholders (with the risk and stakeholder dimension being optional); select between four and nine criteria, as [47] note that excessive information may lead to suboptimal alternative selection, and [48] comments that the human brain is unable to process more than 7 ± 2 items in short-term memory. The list is intended as inspiration and may be adapted according to the organization’s needs.

After selection, in the fourth step, the organization defines weights for each criterion to emphasize those most relevant to the business. Weights range from 1 to 5, with 5 being the most important and 1 the least important. Before proceeding to the next step, any necessary adaptations in the criterion scales are checked.

In the fifth step, each project is evaluated against each criterion, and the most appropriate scale is selected (alternative weighting). Evaluators can perform the assessment individually or conduct a consensus-based evaluation.

The model is already configured for the application of the TOPSIS and TODIM methods. The only required adaptations relate to the number of projects (alternatives) and the number of criteria. After applying the five steps, the pre-configured multi-criteria decision support methods are applied, and the results are provided to the organization.

3.3. Model Composition

The developed model considers the items presented in Figure 2. This figure illustrates the extensive research conducted to construct a broad and generic listing of criteria for evaluating innovation projects considering sustainability aspects.

The model, or conceptual framework, is composed of seven constructs, which are:

• Dimension: the main themes necessary for the evaluation of innovation projects considering sustainable characteristics. These themes go beyond the three proposed by the triple bottom line, as they are required for project portfolio management. Each dimension contains a list of related criteria.
• Criteria: the desirable parameters for evaluating these projects. They integrate the most recognized criteria and the necessary considerations when addressing innovation and sustainability. For each criterion, the following items were developed.
• Definition: a definition was developed for each criterion to assist in understanding it and in selecting the most appropriate criteria to compose the model.
• Typology: determines at which stage of the project development process the criterion is relevant. It is divided into three types: Objectives and Inputs, Development, and Outputs and Effects.
• Scale: universal scales were established for each criterion with qualitative and quantitative options. They range from 3 (three) to 7 (seven) levels, with 1 being the least desirable outcome and the highest level representing the most desirable outcome. It is worth noting that few studies demonstrate the scales of the proposed parameters; thus, it was necessary to develop generic scales to support not only the model but also to serve as inspiration for other applications and studies.
• Relation to SDG: for each criterion, the SDG(s) it supports were established.
• Importance: establishes the level of importance and comparison with other criteria. The levels of importance were determined based on specialists’ indications for some of the synthesized criteria.

4. Application of the Model

The model was applied in two companies that are widely recognized in Brazil for their innovation practices and results. The application was carried out through virtual meetings and information requests. It is worth noting that the model can be used by public or private organizations of any nature that have product innovation projects and wish to evaluate their performance concerning dimensions relevant to innovation from a sustainability perspective.

4.1. Application in Company A

Company A is a medium-sized Brazilian industrial group with over 700 employees, specializing in machining, fastening solutions, automation, and process digitalization. Recognized by the PNI in Sustainability, it has been expanding markets with its own products, artificial intelligence applied to the factory, and analytical services for clients. Its sustainability actions include a 98% reduction in waste, use of renewable energy, carbon inventory, reverse logistics, protection of green areas, and the Semear Program, which combines volunteer work and social benefits for the community. As such, the company integrates technological innovation and socio-environmental responsibility, positioning itself as an example of a future-oriented sustainable industry.

Company A selected four innovation projects to be evaluated, which are presented below. Project 1A aims to develop an intelligent active monitoring device for machining centers in the automotive industry, capable of digitizing hydraulic components into IoT sensors and monitoring critical variables in real time, such as temperature, oil pressure, and vibration. The initiative seeks to increase operational efficiency, reduce maintenance costs, and improve the quality and reliability of components. Project 2A focuses on creating a predictive maintenance and failure prediction platform for CNC machines, integrating artificial intelligence algorithms and data analysis to anticipate mechanical failures, avoid unplanned downtime, and enhance productivity and product quality. Project 3A targets people and process development through the application of Industry 4.0 technologies, digitizing analog equipment and applying artificial intelligence to monitor machine health. This project, conducted in partnership between a business unit of Company A and a leading technology supplier, seeks to balance technological and human advancement, training employees for an increasingly digital market while improving their quality of life in the workplace. Finally, Project 4A involves developing a system that integrates electronics, collaborative robots, and an automated inspection cell, increasing automation and process efficiency.

Company A selected five criteria, which are: “C1 – Environmental / Implementation of Circular Economy”; “C2 – Environmental / Promotion of Conscious Consumption”; “C3 – Economic / Required Investment”; “C4 – Strategy / Alignment with SDGs”; and “C5 – Technological and Operational Capabilities / Degree of Innovation Novelty”, with weights of 5, 4, 5, 4, and 5, respectively, determined by the decision-makers through direct assessment. The company prioritized criteria aligned with its sustainability strategy and that were measurable. It followed the recommended priority groups by dimensions and maintained the indicated number of criteria. Regarding typology, the company selected three criteria for Objectives and Inputs, one for Objectives and Inputs & Development, and one for Outputs and Effects. Although a balance among typologies is recommended, it is believed that this will not affect the analysis of results. The selection of criteria for Company A is considered representative and provided a systematic assessment of the main sustainable aspects relevant to the business and society.

Company A evaluated the four projects according to the proposed scales for the five criteria, and their preferences are presented in Table 4. This evaluation was carried out by the organization’s sustainability specialist and reflected their perceptions.

During the evaluation of the decision-maker’s preferences, it was observed that the projects exhibited similarities across several criteria. For criterion C1, all projects were evaluated identically, as they contribute to extending the equipment’s lifespan, either through sensorization (Projects 1A and 2A) or retrofitting (Projects 3A and 4A). For C2, Projects 3A and 4A were rated higher for promoting internal conscious consumption. Regarding C3, Projects 1A and 2A received greater preference due to higher non-reimbursable investment. For criterion C4, it was noted that all projects are aligned with the company’s SDGs, reinforcing its innovation strategy. In C5, all projects received the same evaluation, as they are incremental innovations with competitive potential. Overall, the preferences were very close, but the weighting exercise was considered useful for generating insights and new perceptions within the organization.

The TOPSIS and TODIM methods were applied following the guidelines of [41,42,44]. As a result, the values obtained from the TOPSIS and TODIM applications are presented in Table 5.

The projects maintained the same ranking for both methods. The company indicated that this ranking is coherent with the organization’s expectations. The first two projects are very similar, as are the last two. Since Projects 1A and 2A involve greater investment and execution effort, it is reasonable that they occupy the top positions. The final ranking did not reveal many surprises, but the evaluation process was important for understanding how the projects are performing.

The sensitivity analysis was performed only using the TODIM method, as this type of analysis is not recommended for the TOPSIS method. Sensitivity analysis can be conducted in different ways, one of the most common being the variation of the weight assigned to the criterion considered most important by the decision-makers, which in this case are “circular economy implementation (C1)”, “necessary investment (C3)”, and “degree of innovation novelty (C5)”. The analysis was conducted by reducing the weight of each criterion by 10%, one at a time. In all three weight adjustments, no change in the ranking occurred. Therefore, the obtained ranking was considered consistent. In this specific application, the decision-makers determined that sensitivity analyses should be conducted only on the reference criterion weight.

4.2. Application in Company B

Company B is a technology-based organization with more than 600 professionals, recognized in sectors such as aerospace, defense, energy, and automotive. Recognized in multiple editions of the PNI, it focuses on the development of complex products and integrated engineering solutions, with over 90% of its contracts associated with innovations. Its notable projects include military aircraft, unmanned aerial vehicles, armored vehicle modernization, training devices, and the pioneering development of a high-resolution space camera launched by SpaceX. Regarding sustainability, the company invests in clean propulsion systems using batteries and hydrogen cells, environmental cameras on satellites such as Amazônia-1, solutions for agribusiness efficiency, and internal practices that reduce resource consumption and increase recycling. By combining cutting-edge technology, continuous innovation, and socio-environmental responsibility, the company establishes itself as a key player in Brazil’s technological and sustainable advancement.

Company B selected seven projects for evaluation, which are as follows: Project 1B involves the development of integrated segments of the eponymous supersonic fighter, an advanced multipurpose trainer that conducted its first flight in April 2023, marking the first construction of rear and central fuselage sections using composite materials. Project 2B involves adapting an aircraft for electronic defense, with comprehensive changes to aeronautical characteristics and mission systems, making the company one of the few non-OEMs worldwide capable of such development, using simulation-based reengineering and virtualization. Project 3B refers to the unprecedented development of a space camera for nanosatellites, capable of capturing high-resolution images of the Earth's surface; launched in April 2023 aboard SpaceX’s Falcon 9, it positioned the company as a leader in the Latin American market and among the few worldwide with this technology. Project 4B developed a camera installed on the Amazônia-1 satellite, focused on environmental monitoring, deforestation detection, fire and soil and water pollution detection, directly contributing to biodiversity preservation. Project 5B aims to advance sustainable propulsion systems, studying UAVs powered by high-performance batteries and hydrogen cells. Project 6B, in two branches, introduced innovations for agribusiness: 6B1 focuses on daily monitoring of beef cattle, generating efficiency gains in feeding, water, and confinement; 6B2 focuses on beekeeping, monitoring hive productivity; additionally, a robust network for forests and crops reduces water consumption and shortens the production cycle. Finally, Project 7B is in the conceptual phase, aiming to develop an aircraft capable of operating above 15 km altitude, functioning as a sensing and communication relay platform (HAPS), providing strategic support for UAVs and offering lower-cost and higher-efficiency alternatives compared to satellites, a factor especially relevant for a large country like Brazil.

The projects are aligned with the organization’s core business, which is the development of complex, technology- and innovation-intensive products for third parties and proprietary products. Many are conducted in partnership but have Company B as the principal developer. It is worth mentioning that the projects were implemented at different times. These projects aim at the development of products and services with significant prominence in the aerospace and defense sectors, demonstrating high competitiveness in terms of incorporated technology and market responsiveness.

Nine criteria were selected, which are: “C1 - Economic – Percentage of sales from key innovations”; “C2 - Stakeholders – Customer satisfaction”; “C3 - Technological and operational capacities – Project pioneering”; “C4 - Technological and operational capacities – Product lifecycle”; “C5 - Social – Social benefits”; “C6 - Social – Social acceptance index”; “C7 - Environmental – Climate change mitigation”; “C8 - Environmental – Resource consumption reduction”; and “C9 - Strategic – Market analysis.” The weights assigned were, respectively: 5, 4, 3, 3, 3, 2, 3, 3, and 5, determined by the decision-makers through direct evaluation. The company prioritized those aligned with its internal innovation strategies, important for advancing sustainability aspects, and measurable.

The company followed the indicated priority groups of dimensions and maintained the recommended number of criteria. Regarding typology, the company selected two criteria for development, seven for outputs and effects, and none for objectives and inputs. As with Company A, it is believed that the lack of balance among typologies will not influence the analysis of the results. The selection of criteria is considered representative, providing a systematic evaluation of the main sustainable aspects of interest to both the business and society.

Company B evaluated the seven projects according to the proposed scales across the nine criteria, and the preference matrix, or decision matrix, is presented in Table 6. This evaluation was conducted by the organization’s RD&I director and reflected their perceptions.

The evaluation of preferences showed that the criteria for climate change mitigation (C7) and resource consumption reduction (C8) are still subjective for the company. While the company recognizes the importance of these themes, it does not yet have consolidated metrics. In the aerospace sector, there is a growing demand for weight reduction and fuel efficiency, although this trend is still in transition. Social benefits (C5) are considered difficult to measure because the impacts are indirect. However, the company acknowledges significant effects, such as the creation of high value-added jobs, attraction of talent and new business opportunities, strengthening of the supply chain, as well as positive impacts on HDI, education, health, and regional infrastructure. The R&D&I director assesses that most projects approach the maximum scale, as they involve cutting-edge technology and generate significant impacts.

The TOPSIS and TODIM methods were applied, and the final values from their application are presented in Table 7.

The analysis showed that the ranking of projects was practically the same in both methods, with the exception of the inversion between projects 5B and 7B. The company considered the result coherent, highlighting Project 4B as the most relevant in terms of sustainability, in addition to Projects 1B, 2B, and 3B as the largest and most strategic. Project 6B, however, was surprising by ranking last, as it was perceived as sustainable but performed poorly on the evaluated criteria. This demonstrates that the use of multi-criteria methods, such as TOPSIS and TODIM, allows a more precise and revealing assessment of project performance in sustainable innovation.

The sensitivity analysis was also conducted considering only the TODIM method, as previously mentioned for Company A. In this case, the weighting of the most important criteria for the decision-makers was varied, which in this instance are “Percentage of sales from main innovations (C1)” and “Market analysis (C9)”. Reducing the weight of C9 by 10% did not change the ranking. When the weight of C1 was also reduced by 10%, a small inversion occurred between two projects in the ranking, namely Project 1B and Project 3B, indicating that, despite the change in weights, the obtained ranking remained consistent. Since the values between these projects are very close, these two changes are expected and reflect minor variations. In this specific application, the decision-makers considered that sensitivity analyses should be performed only on the weight of the reference criterion.

4.3. Model Validation

For the validation of the model, inquiries were conducted during and after its application in order to verify how the model performed, identify opportunities for improvement, and propose future studies. In this way, its comprehensiveness and completeness were assessed. The application carried out in the two aforementioned companies was considered.

Regarding the model's performance, it was verified that it fulfilled its objective, which is to evaluate innovation projects to assess their performance in relation to sustainability criteria and to support decision-making and portfolio analysis. The validation goals defined in the second section of Materials and Methods were achieved, which consist of:

• Applying it in at least 1 (one) organization – the application was carried out in two companies with recognized performance in innovation in Brazil;
• Evaluating at least 2 (two) projects of that organization – Company A evaluated four projects, and Company B evaluated seven projects;
• Analyzing at least 4 (four) criteria within the application – Company A considered five criteria, and Company B considered nine criteria;
• Verifying performance with at least 2 (two) innovation and sustainability specialists from the participating organization(s) – a critical analysis was conducted with two specialists from organizations A and B, one being responsible for the sustainability area of Company A and the other the RD&I Director of Company B.

And the mechanisms for measuring the results, also mentioned in Section 2 (Materials and Methods), were implemented as follows:

• Verification of decision-makers’ perceptions and proposals regarding the model – presented in the sections “General Perceptions of the Model” and “Identified Improvement Opportunities”;
• Verification of whether the model provided a critical analysis for decision-makers in terms of offering new insights on the topic, and critical analysis of the criteria and their applications – presented in the section “Insights and Critical Analysis”. It is worth noting that additional analyses and insights from the authors were included;
• Sensitivity analysis regarding the application of the MCDA method to verify how project rankings change with variations in criterion weights.

4.3.1. General Perceptions of the Model

After the application of the model, participants from Companies A and B were asked about: a) how the process of selection and weighting of the criteria was conducted;

b) whether the generic scales were helpful; c) whether it was difficult to analyze the projects and select the best scales; d) whether the model provided critical analyses for the decision-makers in terms of offering new insights on the topic and critical analysis of the criteria and their applications; e) any suggestions for improvements. Question “e” will be addressed in the next section, “Identified Improvement Opportunities”.

Company A responded that the process of applying the model was very beneficial for the organization, as it served as a process of corporate self-awareness and self-assessment. It generated insights, critical analyses, reflections, and confirmed some of their initial assumptions. The company mentioned that the process of selecting and weighting the criteria, analyzing the projects, and selecting the scales was not difficult, and that some adaptations were made to the scales to fit the company’s reality. They understood that the list of criteria and the model are reference points, and their application requires some adjustments.

Company B stated that, in general, the process of applying the model also provided the organization with new analyses, perceptions, and insights. The company noted that the evaluation process clarified social impacts for the organization and related aspects. Additional analyses and insights commented on by the company and also observed by the authors will be discussed in the section “Insights and Critical Analysis”.

It was noted that, overall, the process of selecting and weighting the criteria, analyzing the projects, and selecting the scales was not difficult. Only for some projects in early stages did the company experience more difficulty in selecting the scales, as these are performance estimates. The company ultimately chose not to adapt the scales; however, during the application, it identified that some criteria with the generic scales did not align with its reality. For instance, the scale for the criterion “product lifecycle (C4)” was distant from the company’s reality, as they develop aircraft that generally last more than 20 years, whereas the scale’s highest value was set at eight years. Even so, the organization believes that this issue did not impact the results.

As previously mentioned, the entire evaluation process—from project selection to criteria weighting and scale selection—was conducted according to the participants’ perceptions. These participants have a comprehensive view of the organization and possess in-depth knowledge and experience regarding the company, its innovation projects, and its sustainability efforts. Therefore, the results and reflections generated reflect the organization’s perspective as conveyed through the participants’ perceptions.

4.3.2. Identified Improvement Opportunities

Both companies identified several issues as opportunities for improvement. Some of the points have already been implemented in the model and will be presented below.

Company A suggested modifying the scale of the criterion “circular economy implementation” to consider the 5Rs of sustainability (reduce, recycle, rethink, reuse, and refuse). It was also suggested to make explicit in the definition that this type of economy involves other production perspectives and business models, such as product-as-a-service, sharing, resource recovery, product life extension, and circular inputs, to improve understanding of the concept.

Additionally, an adaptation was made to the scale of the criterion “alignment with SDGs.” Previously, the scale indicated how many SDGs the innovation project contributed to, with the maximization scale assuming that more was better. The specialist noted that a higher number of SDGs does not necessarily make the project better. For example, a project may be related to only one SDG but be extremely relevant, whereas another may contribute to several SDGs but not align with the organization’s sustainability strategy. It is important to analyze the organization’s strategic perspective. It was suggested to verify the SDGs related to the project and the percentage aligned with the SDGs the company intends to act on or is currently engaging with, which is information requested in the organization’s initial data section. Furthermore, following Company A’s suggestion, an additional intermediate scale was included for the criterion “degree of innovation novelty,” consisting of “2 – disruption in technological and market standards, still without significant competitive advantage.”

Company A also suggested including the criterion “access to funding” to assess the feasibility of projects obtaining external investment funds, either with company co-financing or as grants. Upon further analysis, it was determined that this criterion could be incorporated, with some adjustments, into the criterion “complexity of obtaining external financing.”

Another suggestion from Company A was to make the model available more broadly. The specialist believed that the model could eventually be offered in a web-based format, software, or app to facilitate access for organizations interested in applying it.

Company A also suggested including an explanation of the criteria weights in the model, defining that a weight of 5 (five) is the most important for the organization and 1 (one) is the least important. This improvement has already been implemented. Additionally, the model sometimes used the term “company” and at other times “organization.” Although synonymous, “organization” is broader. Therefore, the term “organization” was adopted, as the model can also be applied in governmental bodies, for example, and in institutions other than companies.

Company B mentioned that it would be useful for some criteria to include a “not applicable” scale, as it is not possible to measure certain criteria for some projects. Accordingly, it is believed that these modifications can be implemented after the criteria selection, as part of the adaptations for specific organizational applications.

4.3.3. Both Companies Identified Several Issues as Opportunities for Improvement. Some of the Points Have Already Been Implemented in the Model and Will be Presented Below.

Company A suggested modifying the scale of the criterion “circular economy implementation” to consider the 5Rs of sustainability (reduce, recycle, rethink, reuse, and refuse). It was also suggested to make explicit in the definition that this type of economy involves other production perspectives and business models, such as product-as-a-service, sharing, resource recovery, product life extension, and circular inputs, to improve understanding of the concept.

Additionally, an adaptation was made to the scale of the criterion “alignment with SDGs.” Previously, the scale indicated how many SDGs the innovation project contributed to, with the maximization scale assuming that more was better. The specialist noted that a higher number of SDGs does not necessarily make the project better. For example, a project may be related to only one SDG but be extremely relevant, whereas another may contribute to several SDGs but not align with the organization’s sustainability strategy. It is important to analyze the organization’s strategic perspective. It was suggested to verify the SDGs related to the project and the percentage aligned with the SDGs the company intends to act on or is currently engaging with, which is information requested in the organization’s initial data section. Furthermore, following Company A’s suggestion, an additional intermediate scale was included for the criterion “degree of innovation novelty,” consisting of “2 – disruption in technological and market standards, still without significant competitive advantage.”

Company A also suggested including the criterion “access to funding” to assess the feasibility of projects obtaining external investment funds, either with company co-financing or as grants. Upon further analysis, it was determined that this criterion could be incorporated, with some adjustments, into the criterion “complexity of obtaining external financing.”

Another suggestion from Company A was to make the model available more broadly. The specialist believed that the model could eventually be offered in a web-based format, software, or app to facilitate access for organizations interested in applying it.

Company A also suggested including an explanation of the criteria weights in the model, defining that a weight of 5 (five) is the most important for the organization and 1 (one) is the least important. This improvement has already been implemented. Additionally, the model sometimes used the term “company” and at other times “organization.” Although synonymous, “organization” is broader. Therefore, the term “organization” was adopted, as the model can also be applied in governmental bodies, for example, and in institutions other than companies.

Company B mentioned that it would be useful for some criteria to include a “not applicable” scale, as it is not possible to measure certain criteria for some projects. Accordingly, it is believed that these modifications can be implemented after the criteria selection, as part of the adaptations for specific organizational applications.

4.3.4.

The insights and analyses were contributed by the two specialists (decision-makers) from the companies used in the application, as well as by the authors’ considerations. In both applications, the companies preferred to use the model primarily to evaluate projects that were already in progress or completed, with the aim of assessing their performance according to sustainable parameters. As highlighted earlier, the model does not have a restriction to analyze only projects that have not yet started. It can evaluate projects at different statuses in order to support decision-making and analysis. This decision showed that, specifically in these cases, although it may indicate a preference, it could be more interesting for organizations to evaluate projects after the decision of whether they will actually be part of the organization’s portfolio. The model thus helps not only in the innovation funnel—that is, in deciding which projects to pursue—but also in a deeper analysis of how projects are performing concerning sustainability aspects.

With this strategy, it was observed that mixing projects already in execution with others not yet started or recently initiated introduces some degree of complexity to the analysis. Projects in execution often have more precise data, while those to be initiated or recently started work with estimates, which can bias or even make it impossible to identify scales. However, these issues did not prevent analyses considering distinct projects.

The model was also used to evaluate business process innovation projects. It was verified that the model performed adequately in this scenario without loss or misalignment. Some adaptation of certain criteria may be necessary. Therefore, it is possible to include process innovation projects in addition to product innovation projects within the model.

It was also observed that for some criteria of the “outputs and effects” typology, it makes more sense—or is more appropriate—to evaluate projects that are already implemented. An example is the evaluation of the criterion “expected customer satisfaction.” For projects not yet executed, this estimate becomes very imprecise and biased by the company’s perception rather than what is intended to be evaluated. These criteria can be selected to examine such projects, but this bias must be considered in the process of analyzing their performance and the resulting ranking.

Another aspect identified, specifically regarding the selection of criteria, was that Company A initially began the criteria selection by analyzing the related SDGs, highlighting those related to the organization’s priority SDGs. The company concluded that this analysis did not help in identifying the criteria, and those highlighted were not a priority at the moment. Ultimately, it selected the criteria without considering this factor. Therefore, it was considered that the SDG relationships provide interesting information but are not crucial for the initial criteria selection. This analysis could, however, assist in tie-breaking but not in the initial selection.

Another recommendation for criteria selection was to balance the number of criteria considering the three identified typologies. This was not followed in the two applications, yet it did not influence the selection or analyses. Therefore, it is believed that this recommendation can be disregarded, and this information was removed from the model.

It was observed that criteria selection is very particular, considering the organization’s strategy, what makes more sense to evaluate, and what is measurable. As highlighted by the specialist from Company A, the model serves as a reference, but its application requires necessary adaptations to the organization’s reality. According to the authors’ and specialists’ perceptions, these adaptations are generally neither complex nor costly.

The specialist from Company A was concerned about not being able to select at least one social criterion. He made a critical consideration during the application, noting that the company still does not have significant engagement in the social dimension. He mentioned that many of the organization’s actions resemble charitable initiatives rather than systematic and strategic social contributions from its projects. He could not visualize how his projects operate in this sense to generate benefits and reduce negative impacts. This situation is very common in the Brazilian small- and medium-sized business scenario. When companies of this size were not created for social innovation, their actions are sporadic and lack strategic involvement.

Consequently, it is likely that most Brazilian innovative companies focus heavily on environmental aspects when it comes to sustainability. This is not only a reality for small and medium-sized enterprises but also for large national companies. Previous research highlighted a greater importance for economic aspects, but recently specialists have recognized that environmental and social aspects are increasingly important within sustainable analysis. In Brazil, however, environmental aspects likely overshadow social aspects. This may be because environmental regulations have been in place for longer, while similar requirements do not exist for social aspects in the country.

Additionally, Company B mentioned difficulty during the model’s application in assessing environmental and social criteria, with the latter being the most subjective. It was observed that when projects are not inherently social—that is, not created for this purpose—it is difficult to measure their social benefits. Considering all dimensions and criteria used during the applications, environmental and social criteria were the most labor-intensive to measure and required more investigation.

Another fact observed and reported was that national B2B companies (companies selling products and services to other businesses) still exhibit a lower level of sustainability maturity compared to B2C companies (selling directly to consumers). B2B companies are generally driven by market-imposed requirements, such as legislation or demands from the government and/or supply chain. This situation is likely common in developing countries. For example, Company A, which operates under a B2B model, noted that one of its major clients recently requested the company to conduct a study to analyze its greenhouse gas emissions in Scope 1 and 2. This measurement will help identify the client’s Scope 3 emissions. This initiative is recent, and the company is still determining how to carry it out.

5. Conclusions

This research aimed to advance the frontier of knowledge on the selection of innovation projects with an emphasis on sustainability, although it faced methodological and operational challenges. The application of the model in companies presented difficulties because, despite interest, the availability of teams was limited, which reduced the number of cases tested. Moreover, the evaluation scales were largely developed by the authors, which may have introduced imprecise estimates, although they were adaptable to organizational needs. The assessment of projects that had not yet started also imposed restrictions, as results and impacts at this stage are more difficult to measure.

Despite these limitations, the research provided relevant contributions. The developed model demonstrated potential to support organizations in the selection and management of project portfolios, reducing subjectivity and offering strategic insights. The research successfully developed a customizable and unprecedented model capable of evaluating, selecting, and supporting the management of innovation projects within a sustainable context, as well as making the process of selecting and managing innovation project portfolios under sustainable perspectives less subjective. The model provides a unique tool that captures the organization’s main strategic perspectives, translates them into evaluation criteria under sustainability aspects, and ultimately offers insights to guide the management of the innovation project pipeline. Its application in Brazilian companies allowed for the identification of social and environmental impacts that are still little explored, in addition to generating useful reflections for improving innovation evaluation methodologies in the country.

As future propositions, it is recommended to expand the application of the model across different sectors and organizational sizes, explore specific adaptations by area of activity, integrate sustainability criteria into ESG goals, develop digital application tools, and deepen analyses of the social and environmental impacts of innovations. These initiatives can contribute to theoretical and practical advancements, as well as to the design of public policies focused on sustainability and innovation.

6. Patents

The project selection model presented in this study was registered as a computer program, registration number: BR512024001798-4, registration date: 12/21/2023, title: "INOVA_SUST," and registering institution: INPI – National Institute of Industrial Property - Brazil.