Assessing Resource Management in Higher Education Sustainability Projects: A Bootstrap Dea Approach

1. Introduction

The increasing frequency of extreme weather events worldwide underscores the urgent need to address climate change and promote sustainability [1]. This urgency has driven a global movement toward sustainable development, a concept popularized by the Brundtland Report, which defines it as “a process that meets the needs of the present without compromising the ability of future generations to meet their own needs” [2]. Among the United Nations’ 17 Sustainable Development Goals (SDGs), combating climate change remains a critical priority.

Given this context, Brazil has assumed a pivotal role in advancing global sustainability efforts through its commitments to the Paris Agreement and the 2030 Agenda. Notably, conserving the Amazon rainforest is a main concern, as it is one of the world’s most significant natural carbon sinks, capturing more greenhouse gases than it emits and playing a crucial part in mitigating climate change [3].

However, despite various sustainable development initiatives, the country faces significant challenges in advancing its goals. That scenario is especially evidenced by the [4], which indicates that only 7.7% of the SDGs are being met satisfactorily. This highlights the urgent need to make government initiatives more efficient, in order to maximize the value generated for society and improve public welfare.

Efficiency is key to ensuring that actions produce positive social and environmental impacts while optimally utilizing available resources and reducing waste [5]. Research shows that efficient projects not only reduce environmental costs but also promote social progress, such as greater inclusion, quality of life, and sustainable economic development [6]. It is crucial to focus on solutions that integrate effectiveness, innovation, and collaboration, creating public and private partnerships that enhance results and ensure the implementation of policies that truly transform the social and environmental landscape [7].

Educational institutions play a vital role in shaping society and setting new standards through their socioeconomic, technoscientific and cultural influence in the environment in which they are situated [8]. That role remains instrumental in both shaping and implementing sustainability policies and best practices.

In light of this, the present work aims to systematically evaluate the efficiency of sustainability-focused STEM (Science, Technology, Engineering, and Mathematics) initiatives at the University of Brasilia (UnB), a leading national institution in social and environmental sustainability [9]. By identifying the optimal allocation of resources, the research seeks to enhance the effectiveness of these projects while providing benchmarking insights to support decision making in sustainability-related efforts at other institutions. Ultimately, this study addresses the central question: How efficient have UnB’s internal STEM initiatives been in promoting sustainability?

2. Related Works

A bibliometric approach was conducted to review precedents in the use of Data Envelopment Analysis (DEA) for measuring the efficiency of sustainability-oriented projects. In order to do that, a three-year period was selected, and documents were collected from Web of Science (WOS). The search was restricted to journal articles and conference papers within the field of Engineering and Environmental Sciences to ensure a comprehensive and focused cross-analysis in the STEM field. The query was designed around three key dimensions, combined using the AND operator: DEA (DEA OR "Data Envelopment Analysis"), Sustainability (Sustainab* OR Environ* OR Green OR "Socio-environ*"), and Projects (Project OR Initiative).

The query yielded 129 articles, which were used to conduct co-citation and bibliographic coupling analyses to respectively identify relevant thematic clusters and shared theoretical foundations. A summary of the articles most closely aligned with this research was also elaborated. Figure 1 presents the results obtained from the co-citation analysis using all the documents retrieved.

Four major clusters were identified from the search, each associated with a specific contribution. The red cluster brings together fundamental studies on DEA. [10] introduce the CCR model, establishing it as a method for measuring efficiency in units with multiple inputs and outputs. [11] improves this approach with the slacks-based measure (SBM), which directly addresses inefficiencies in inputs and outputs. Complementarily, [12] propose a three-stage model that adjusts inputs or outputs to account for environmental effects and statistical noise.

Moreover, the purple cluster includes articles on productivity measurement with a focus on Malmquist indexes. [13] develop a theoretical foundation for comparing inputs, outputs, and productivity under different production structures, linking Malmquist and Törnqvist indexes, while [14] apply the Malmquist index to analyze productivity growth in 17 Organisation for Economic Co-operation and Development (OECD) countries, decomposing it into technical change and efficiency change.

The green cluster presents contributions on the use of DEA for evaluating sustainability in specific sectors. [15] propose a DEA model for intermediate networks, focusing on efficiency and technological inequality, in order to evaluate three stages of the sustainability system: economic growth, environmental protection and health promotion in Chinese provinces. In contrast, [16] apply DEA to analyze the performance of energy industries, addressing the challenge of balancing economic development with environmental protection.

Lastly, the yellow cluster explores the historical evolution of DEA in different areas. [17] review 40 years of DEA development, discussing its popular models, advantages, limitations and applications, as well as conducting a bibliometric analysis of publication trends. On the other hand, [18] review the literature from 2017 to 2020, emphasizing the growing use of DEA to measure sustainability, but pointing out the underrepresentation of the social dimension of sustainability and the use of proxy indices as incomplete substitutes for a multidimensional sustainability assessment.

Based on these observations, the obtained clusters suggest that the main research lines have been built upon classical pioneering works in efficiency and productivity analysis. Current work lies on the application of DEA to specific sectors, with ongoing efforts to refine the original methods through the integration of new tools and frameworks.

Furthermore, Figure 2 displays the results derived from the bibliographic coupling analysis of the selected articles.

Through that analysis, six major clusters were detected. The yellow cluster includes studies on the use of DEA to analyze sustainable performance in specific areas. [19] investigate how environmental regulations influence the performance of the circular economy in China, showcasing that regulation mainly promotes performance through a catch-up effect. [20] apply DEA to measure eco-efficiency in the tourism sector in Gansu, revealing that hotels are the largest contributors to carbon emissions. Meanwhile, [21] use DEA to assess the efficiency of industrial pollution control in Chinese provinces. The authors highlight regional differences while also suggesting improvements in efficiency.

Building on that, the green cluster focuses on efficiency in sustainable technologies and green logistics. [22] propose an integrated DEA-SBM approach with projection analysis to assess the performance of green technology R&D in China. [23] develop a model to analyze efficient locations for renewable energy installations in Vietnam. Meanwhile, [24] evaluate green logistics efficiency in Northwest China, highlighting regional differences and proposing strategies to support carbon emission reductions.

The purple cluster seeks to evaluate the impact of different policies and technologies on efficiency and sustainability across various contexts. [25] analyze regional differences in traffic restriction policies in China, suggesting dynamic adjustments towards a sustainable economic development. In contrast, [26] focus on strategies for reusing water and substrates in Mediterranean greenhouses, aiming to promote eco-efficiency and agricultural profitability.

Moving further, the red cluster focuses on evaluating efficiency in key sectors such as energy, tourism, and sustainable financing. [27] investigate the financing efficiency in China’s energy and environmental protection industry, focusing mainly on digital transformation and technological innovation perspectives. [28] analyze tourism efficiency in China, identifying patents and government control as key efficiency factors, while [29] propose a model to assess the performance of the Chinese energy supply chain, aligned with the carbon neutrality and peak emission goals.

Complementarily, the brown cluster focuses on assessing efficiency in sectors related to agriculture and industry, with an emphasis on sustainable practices and its impacts. [30] analyze the effect of off-farm employment on agricultural production efficiency in China, pointing out to the negative impacts of self-employment, especially at lower levels of off-farm employment. On the other hand, [31] examine environmental efficiency in the salmon industry in Chile, observing an improvement in environmental efficiency after the implementation of stricter regulations following the ISA virus outbreak.

Finally, in the orange cluster, [32] analyze the cost efficiency in rapeseed production in Hunan, China, using DEA. The study shows that all technical, allocative and cost efficiencies have room for improvement. The authors point out that expanding farmers’ operations may increase efficiency, but it would also lead to negative environmental impacts.

The clusters from coupling highlight how DEA and sustainability have been combined in recent years. The studies predominantly assess various facets of the sustainability domain applied to different sectors, such as energy, tourism and agriculture. Additionally, other studies focus on determining the impact of isolated factors on improving specific indicators and/or productivity, as well as enhancing decision-making through the combination of different multi-criteria methods.

Finally, a summary of the found previous work most closely aligned with this study was organized in Table 1.

Although the analyzed studies advance the application of DEA to assess efficiency and resource allocation towards sustainability, some limitations persist. The model proposed by [35] relies on some subjectively measured variables, which may introduce potential biases in its evaluation. Similarly, the approach of [36], despite being robust, employs 15 distinct variables, which compromises its generalizability. Meanwhile, [39] is limited by the scope of its sample by using the project’s covered area as one of the selected inputs, whereas [22,34] prioritize metrics such as publications, patents, and energy consumption, neglecting direct social impacts.

In light of these limitations, this study seeks to differentiate itself by proposing an objective approach with few parameters, focused on optimizing the allocation of managerial resources. This characteristic facilitates its application in different contexts, enabling the extraction of practical decision making insights through its replication.

3. Methodology

This is an applied research study with a quantitative approach that utilizes data collected from project reports available on UnB’s central online platform, along with supplementary information gathered through interviews with project coordinators. The obtained data underwent a rigorous processing phase, including an iterative outlier removal and normalization procedure to ensure comparability across different scales and mitigate potential biases. Additionally, linear correlation analyses were conducted to examine the interrelationships among the variables.

With the processed data, the model’s return to scale and orientation were adjusted to align with the study’s objectives and the specific characteristics of the dataset. Finally, the bootstrap technique was applied to enhance the reliability and robustness of the results.

3.1. Model Selection

A technical efficiency approach on the topic was chosen, considering the use of DEA. This method allows a suitable comparison of the studied initiatives based on their inputs and outputs, as it has been a extensively used solution for combining economic, environmental and social indicators with different units within the sustainable domain [40,41].

DEA was chosen for this work because it is particularly well-suited to evaluating the relative efficiency of Decision-Making Units (DMUs) in resource utilization and outcome maximization without the need of predefined weights. It also does not require a prior identification of any type of relationship between the variables nor any specific statistical distribution for the data [18].

3.2. Variables Selection

Six complementary variables were selected for the analysis based on their availability, accuracy and relevance to the study. Team size (TS), project budget (B), and workload (W) were considered as inputs variables, while the amount of published papers (PP) and the number of beneficiaries (NB) were treated as outputs variables. Table 2 summarizes the variables selected.

The decision to lean the input variables toward a managerial perspective was adopted as an alternative to allow its application to projects with different focuses. Concurrently, the output variables were selected to assess both the social impact of the initiatives, measured by the number of people that were benefited, and the scientific contribution, taking into account the academic context in which the sampled projects are situated.

3.3. Data Gathering

The study population comprises all STEM projects associated with sustainability at UnB. To ensure the consistency and comparability of the selected projects, specific inclusion and exclusion criteria were established. Only projects completed between 2020 and 2023 were included, while those associated with junior enterprises, athletic organizations, student chapters, research groups and competition teams were excluded.

The data for this study was collected through the Integrated System for Academic Activity Management (SIGAA), UnB’s central platform for managing academic processes such as enrollment, grade submission, and communication between students and professors. By applying the filters "Engineering”, “Project" and "Sustainability," a detailed list of relevant projects was generated. This list included data on team size, workload, and project budget. Additional information, such as the number of published papers and the number of beneficiaries, was obtained through interviews with project coordinators, as these details were not available in SIGAA reports.

3.4. Outlier Detection

Initially introduced by [42] for ranking efficient DMUs, the super-efficiency DEA model allows for efficiency scores greater than 1, where each DMU cannot use itself as a reference. This variation enables more precise observation among the more efficient DMUs, while inefficient DMUs maintain the same scores as in the standard DEA model [43].

In that way, this model presents a promising alternative for identifying and removing outliers, considering the potential for greater variation in the scale of observed results [44]. Following these principles, an iterative process was conducted to verify the relative super-efficiencies of the sample projects, leading to the elimination of those classified as outliers according to the Interquartile Range (IQR) Method. This procedure spanned a total of three iterations, resulting in a final sample of 20 projects. The complete original sample can be found in Appendix A, Table A1.

3.5. Return to Scale

[45] emphasize the importance of understanding the nature of returns to scale for the correct application of the DEA model. Significant distortions in results can arise from incorrect assumptions about returns to scale, potentially leading to losses in statistical efficiency. In this context, the authors propose calculating scale efficiency as:

${\widehat{\theta }}_E={\displaystyle \frac{{\sum }_{p=1}^r{\widehat{\theta }}_{r,\mathrm{CRS}}}{{\sum }_{p=1}^r{\widehat{\theta }}_{r,\mathrm{VRS}}}}$, where ${\widehat{\theta }}_E\le 1$, ${\widehat{\theta }}_{\mathrm{CRS}}$ is the constant return to scale efficiency, and ${\widehat{\theta }}_{\mathrm{VRS}}$ is the variable return to scale efficiency.

In scenarios where ${\widehat{\theta }}_E$ is approximately equal to 1, it is understood that constant returns to scale (CRS) are most appropriate for the data. Conversely, a value significantly lower than 1 may suggest the behavior of variable returns to scale (VRS). For the analyzed sample, ${\widehat{\theta }}_E\approx 0.69$ for the original data and ${\widehat{\theta }}_E\approx 0.77$ after the removal of outliers. Given the value being relatively close to 1, the constant returns to scale (CRS) model was adopted.

3.6. Model Orientation

Furthermore, the study explored input orientation through the lens of an isoquant curve, which illustrates all combinations of productive factors that yield the same level of output. By considering the minimum combination of resources on this curve to achieve a specific level of system output, it becomes possible to delineate the frontier defining the efficient set of combinations [46]. From this perspective, the focus shifts to reducing resource usage across projects while maintaining the same output level.

3.7. Bootstrap

To enhance the reliability and acceptance of the DEA model’s efficiency scores, bootstrapping was employed. This technique simulates a sample by applying the original estimator, ensuring that the simulation results replicate the original sample through a Data Generating Process (DGP). This process involves repeated resampling, conducted multiple times [47]. The results were derived from 1,000 resamplings, with a significance level ($\alpha$) set at 0.05.

4. Results

This section begins with an evaluation of the project profiles within the sample, emphasizing the relationships between the chosen variables and their characteristics. Finally, the application of the DEA model to the data using bootstrap is introduced.

4.1. Projects Profile

To identify the priorities of action within the evaluated projects, a count of mentions for the different SDGs was conducted. Figure 3 illustrates the findings of this analysis.

Complementarily, the Pearson correlation between the selected inputs and outputs for the model was calculated. In that way, Figure 4 illustrates the degree of linear relationships among the variables. It is important to note that the acronym TS represents team size, B denotes the budget, W indicates workload, PP refers to the number of published papers, and NB signifies the number of beneficiaries.

After gaining an initial understanding of the project profiles and the selected variables, the research moved on to the application of the DEA model.

4.2. Bootstrap DEA Application

Table 3 presents the efficiency scores for each evaluated DMU, which have been anonymized for privacy reasons. It includes both bias-corrected efficiency scores generated by the bootstrapping technique and the corresponding 95% confidence intervals for each project’s efficiency score. The following abbreviations were used for the departments within the sample: Department of Electronic Engineering (EEL), Department of Civil Engineering (ENC), Department of Energy Engineering (EEN), Department of Mechanical Engineering (ENM), Department of Electrical Engineering (ENE), Department of Production Engineering (EPR), Department of Automotive Engineering (EAU), Department of Mathematics (MAT) and Department of Collective Health (DSC).

Linked to the results presented above, it was possible to compare the performance of the different university departments present in the sample based on their distributions. Figure 5 utilizes density plots to demonstrate the results for departments that contain more than one project in total.

Lastly, an analysis of the slacks across the DMUs was conducted. The optimal objective value (B*) can indicate potential excess inputs or shortages of outputs [48]. Table 4 presents the percentage slack for each variable and DMU, considering its distance to the efficiency frontier. The absolute final goal values that correspond to the necessary reductions or increases in the variables can be found in Appendix B, Table A2.

The following section offers a more comprehensive analysis of these findings, delving deeper into their significance and broader implications for the efficiency of the examined projects.

5. Discussions

Based on the obtained results, according to Figure 3, SDG 4 (Quality Education) appeared as the most popular goal among the sampled projects, with a total of 19 citations. Given that the projects originate from a higher education institution, it is reasonable to highlight the emphasis on education, particularly regarding the experiences offered to project participants. Following closely is SDG 11 (Sustainable Cities and Communities), with 15 mentions, which focuses on making cities and communities more inclusive, resilient, and sustainable. Third, SDG 3 (Good Health and Well-Being) is highlighted, emphasizing access to quality healthcare and the promotion of overall well-being. Conversely, the least addressed SDGs were SDG 16 (Peace, Justice, and Strong Institutions), SDG 14 (Life Below Water), and SDG 2 (Zero Hunger and Sustainable Agriculture), which received 1, 3, and 3 mentions, respectively.

The results also indicate that projects with higher volumes of publications are generally those with larger budgets, as it was shown in Figure 4. These two variables showed a strong positive correlation, with $r=0.78$. However, the same cannot be said regarding the number of beneficiaries. The calculations demonstrated a weak negative correlation between the budget and number of beneficiaries, with $r=-0.22$. Overall, no significant relationships were found between the defined inputs and the number of beneficiaries. On the other hand, the relationship between the two measured outputs presented a weak negative correlation ($r=-0.33$), suggesting that these outputs do not tend to increase simultaneously; in other words, one may be prioritized at the expense of the other.

Moreover, team size and workload showed a moderate positive correlation ($r=0.5$), which is evidently explained by the direct relationship between the time spent on a project and the number of allocated members.

Following that, Table 3 and Figure 5 revealed that the EEL, ENM, and ENE departments achieve their highest frequencies at efficiency scores around 0.5, while ENC and MAT detain the top peaks among all departments, close to 0.75. In contrast, EEN stands out with the lowest results, peaking around 0.25. The DSC, EPR, and EAU departments, with only one project each, recorded approximate scores of 0.76, 0.72, and 0.57, respectively, as also summarized in Table 3.

Finally, based on the slack evaluation presented in Table 4, DMU 16 exhibited the most significant need for input reduction to reach the efficiency frontier. Specifically, it requires a 53.9% reduction in team size (TS) and a 7.5% increase in the number of beneficiaries (NB). For DMU 9, the recommendations include an 8.1% reduction in TS and a 21.7% decrease in workload (W). DMU 18 is advised to reduce W by 11.8%, while for DMU 10, a decrease of 9.4% in TS is deemed optimal. These were the projects that required the most significant adjustments.

In practical terms, the findings provide a deeper understanding of the context faced by sustainability projects within higher education, while also proposing concrete actions to objectively enhance their efficiency, ultimately supporting progress toward the SDGs. The results pointed to a direct correlation between a project’s budget and the number of publications, meaning that greater financial resources may indeed lead to a broader dissemination of the actions taken in favor of the aimed SDG.

For academia, this study advances the knowledge frontier by addressing a flexible resource management approach combined with Bootstrap DEA to assess sustainability-focused initiatives. Furthermore, the results serve as a valuable benchmark reference for other universities and institutions seeking more informed decisions regarding resource allocation towards sustainability.

6. Final Considerations

The study successfully achieved its primary goal of evaluating the efficiency of sustainability-focused STEM initiatives at the University of Brasilia. By assessing the relative efficiency of the selected projects, the study identified strategies for optimal resource allocation, providing valuable benchmarking insights for other higher education institutions.

To address the research question, a robust methodology was employed, incorporating an extensive literature review, definition of key variables, data collection from ongoing initiatives and the application of Bootstrap DEA. This comprehensive approach enabled an objective and adaptable evaluation of the initiatives, highlighting areas for improvement and offering actionable recommendations to enhance their performance.

Research limitations include the unavailability of data from 2024 up to the time the current work was written, as well as the low sample of projects linked to specific departments, making it difficult to observe patterns or to detect tendencies.

For future research, extending this analysis to encompass additional universities and academic institutions would offer a more comprehensive view of sustainability practices in higher education, facilitating cross-institutional comparisons. Such a broader perspective could significantly enhance institutional sustainability efforts and contribute to advancing global initiatives for sustainable development.