Beyond Traditional Metrics: A Novel Framework for Managing R&D Project Performance in the Petroleum Industry

1. Introduction

Conventional project performance models rely on “iron triangle” indicators (time, cost, scope) or on narrow scientific outputs (patents, publications, prototypes). These metrics are insufficient for R&D, where uncertainty, learning, and adaptability are intrinsic.

R&D activities are inherently uncertain, resource-intensive, and strategically critical. Classical project management techniques often adopt efficiency-based metrics—budget adherence, milestone achievement, patents, or publications—as proxies for performance. Such measures neglect intangible yet vital contributions such as knowledge spillovers, capability building, adaptability, and portfolio synergies.

Recent scholarship has called for more holistic perspectives on innovation performance, integrating organizational learning, dynamic alignment, and sustainability considerations. This paper responds to this gap by developing a structured framework for evaluating R&D projects across multiple dimensions. The framework is designed to enhance strategic decision-making and strengthen organizational resilience by equipping firms to navigate technological discontinuities and volatile market conditions with greater foresight and adaptability.

We propose a Beyond Traditional Metrics (BTM) Framework, a novel, multi-dimensional approach that evaluates R&D projects not only on outputs but also on knowledge, capability building, adaptability, ecosystem readiness, and stakeholder value creation.

This paper introduces a novel structured management framework for evaluating and optimizing the performance of R&D projects. Traditional KPIs such as ROI or time-to-market are insufficient to capture the multidimensional value of R&D in the petroleum industry. Our framework integrates five dimensions: Knowledge Creation, Innovation Velocity, Dynamic Strategic Alignment, Team & Organizational Health, and Resilience & Robustness.

Yet, focusing solely on publications and patents can underestimate the industrial and operational relevance of R&D. To complement academic and intellectual property indicators, the framework introduces the “Industrial Application Rate,” which measures the proportion of R&D outputs effectively deployed at scale. This indicator links research activities to tangible industrial value creation and operational efficiency (Coccia, 2021).

2. Evolution from Traditional to Novel Metrics: Shift from Output-Focused to Multi-Dimensional Metrics

Research reveals that R&D project performance management frameworks are shifting from traditional output metrics to holistic, multi-dimensional approaches that include process orientation, strategic alignment, innovation potential, and cross-departmental integration.

It is worth noting that Process orientation within R&D project management reflects a systematic focus on how innovative capabilities are developed, shared, and institutionalized within the organization. A process-oriented view prioritizes continuous learning, cross-functional collaboration, and structured feedback loops between projects. By monitoring workflow efficiency and knowledge reuse, process-oriented KPIs help evaluate not only outcomes but also the learning mechanisms that sustain innovation capacity (Cooper & Sommer, 2020).

2.1. Knowledge Diffusion and Spillover Metrics

Research on R&D evaluation increasingly shows a move away from relying solely on publications, patents, and short-term sales to frameworks that also capture processes, strategic alignment, innovation potential, knowledge spillovers, and ecognizing ze capabilities.

In this context, knowledge diffusion and spillovers require conceptual precision. Knowledge diffusion refers to the internal dissemination of research insights within an organization, whereas knowledge spillovers capture the unintended external transfer of know-how to other entities. These phenomena differ from direct R&D outputs such as patents or publications, as they represent broader pathways of learning and capability building (Fleming et al., 2019).

• Hauser (1998) critiques the limitations of market-outcome metrics for all R&D types, advocating tiered metrics and “research tourism” to encourage external idea exploration.
• Kristiansen and Ritala (2018) show that traditional KPIs are ineffective for radical innovation, proposing process-based measures such as market orientation, learning, and resource dedication.
• Albuquerque et al. (2024) and Klessova et al. (2021) introduce frameworks assessing innovation maturity and upstream potential rather than output alone.
• Lin, Shih, & Chang (2024) and Autant-Bernard et al. (2013) highlight knowledge diffusion from academia to industry and across European regions as a critical but under-measured aspect of innovation performance.
• Su, Yu, & Tao (2018) develop metrics for knowledge diffusion efficiency in R&D networks, complementing traditional productivity measures.

2.2. Innovation Velocity as a Strategic Metric

Recent work has also isolated the dimension of “innovation speed” or “innovation velocity” as a distinct performance category.

• Jing (2024) reviews how innovation speed varies across industries and time periods.
• Kessler & Chakrabarti (1996) provide a conceptual model of the antecedents and outcomes of innovation speed.
• Langerak et al. (2004) link market orientation, product advantage, and launch proficiency to faster time-to-market and better performance.

2.3. Multi-Dimensional Performance Assessment Frameworks: From Single Indicators to Integrated Dashboards

Several authors propose frameworks that combine technological, market, process, and ecognizing ze metrics into single systems:

• Bitman & Sharif (2008) present a project-ranking system evaluating reasonableness, attractiveness, responsiveness, competitiveness, and innovativeness.
• Mallick & Carlson (2003) emphasise integrated frameworks ecognizing the relationships between design, manufacturing, and marketing metrics.
• Meyer et al. (1997) advocate multi-product, multi-period planning and cross-departmental data integration to assess technological and market leverage.

These approaches reflect a broader trend toward holistic, system-level assessment of R&D performance.

2.4. Strategic Alignment of R&D Portfolios: Strategic Alignment as a Dimension of R&D Performance

While strategic alignment ensures coherence between R&D initiatives and corporate priorities, it is not in itself a sufficient indicator of performance.

In its traditional form, it functions as a binary conformity check indicating whether a project fits current strategic priorities rather than a measurable indicator of value creation or capability development. To be meaningful, strategic alignment must be reframed as a SMART metric, capturing the degree to which R&D initiatives build future strategic options, strengthen technological capabilities, and enhance innovation ecosystems.

A strong stream of research addresses how to align R&D projects and portfolios with corporate strategy:

• Santiago & Soares (2020) show how “bucket” approaches can support strategic portfolio alignment.
• Martinsuo & Anttila (2022) and Martinsuo, Poskela & Killen (2022) discuss practices of strategic alignment within and between innovation portfolios.
• Cooper, Edgett & Kleinschmidt (2001) and Meskendahl (2010) demonstrate that portfolio-level selection and ecognizing ze frameworks can enhance success when explicitly tied to business strategy.

2.5. Context-Specific Performance Management Systems: Readiness and Adaptability Frameworks

Frameworks must be adapted to ecognizing ze setting, sector, and collaboration mode:

• Larsen & Lindquist (2016) develop an activity-level framework for a manufacturing R&D department with emphasis on transparency and sustainability.
• Fayek & Golabchi (2021) focus on collaborative R&D in the construction sector, aligning metrics with industry needs.
• Lee & Lee (2023) introduce a system ecognizing indicators by observability and frequency, with a progress-priority matrix for a Korean research institute
• Holden (2022), Van Cauwenbergh et al. (2021), and Yfanti & Sakkas (2024) expand Technology Readiness Levels (TRL) into broader “institutional readiness” frameworks for sustainable and co-created innovations.

2.6. Organisational and Team Dimensions of Performance: Team, ecognizing ze Health and Resilience Metrics

To enhance evaluation realism, the framework considers psychological and organizational factors influencing team effectiveness. Although psychological safety, resilience, and collective learning are increasingly recognized as foundational drivers of innovation performance in R&D settings. Yet, their qualitative and tacit nature makes them difficult to translate into standardized performance metrics. To mitigate subjectivity, mixed-method approaches combining qualitative surveys and proxy metrics (e.g., idea-sharing frequency, turnover rate, cross-team collaboration events, participation in learning activities) can offer valid insights into team dynamics (Newman et al., 2022).

There is growing recognition that team resilience, psychological safety, and ecognizing ze learning directly influence R&D performance:

• Edmondson (1999) introduces the concept of psychological safety and learning behaviour in work teams.
• Atkinson et al. (2020, 2021) and Hülsheger et al. (2009) link team-level factors to innovation outcomes.
• Varajão, Fernandes & Amaral (2023) show how information systems team resilience improves project success.
• Liu et al. (2024), Naderpajouh et al. (2020), and Koilakonda (2023) address R&D network resilience under risk propagation and strategies for overcoming challenges.
• Woods (2015) outlines four concepts of resilience relevant to innovation project management.

3. Conceptual Framework

3.1. Framework Principle

The proposed framework evaluates R&D project performance using a weighted scoring system across five interrelated performance dimensions: Knowledge Creation & Diffusion (KC), Innovation Velocity (IV), Dynamic Strategic Alignment (DA), Team & Organizational Health (TH) and Resilience & Robustness (RR)(Table1). Each dimension is scored on a 0–10 scale. Weighted sums provide a composite score for portfolio-level decision making.

To better support portfolio governance, the BTM framework incorporates Multi-Criteria Decision-Making (MCDM) techniques such as AHP and TOPSIS. These methods enable systematic handling of subjective judgments, uncertainty, and interdependencies across criteria (Santiago & Soares, 2021).

Table 1. Performance dimension and description of the suggested BTM Framework.

Performance dimension	Description
Knowledge Creation and Diffusion	Measurement: patents, publications, datasets, methods, and evidence of reuse across other projects., stronger human-centric KPIs such as staff mobility across projects, number of mentoring sessions or internal workshops conducted (mentoring frequency), integration rate of research outcomes into operational workflows (adoption rate of new R&D methods), contribution to professional standards and Sonatrach-Academia linkages (Martinsuo et al., 2022; Nonaka & Takeuchi, 2019). Rationale: R&D value extends beyond immediate deliverables to the creation of knowledge assets with long-term impact.
Innovation Velocity and Learning	Measurement: iteration cycle time, capability accretion index, experiment-to-insight ratio, rate of adoption of lessons across teams. Innovation performance should not be conflated with velocity. While shorter cycle times may improve responsiveness, they do not necessarily indicate deeper learning or technological advancement. Genuine innovation requires iterative experimentation, reflective learning, and the ability to transform failures into insight (Pisano, 2019). Therefore, the measurement system privileges evidence of validated learning, capability accumulation, and knowledge diffusion across the organization. Rationale: Projects that accelerate organizational learning create competitive advantage, regardless of immediate outcomes.
Dynamic Strategic Alignment	Measurement: knowledge reuse rates, adaptive behavior during disruptions, and team recovery after project setbacks. Strategic alignment represents the systematic coherence between organizational objectives, strategic development domains, operational processes, and the resources mobilized to achieve them. In R&D contexts, effective alignment entails translating corporate strategy into actionable portfolio decisions through mechanisms such as thematic road-mapping, adaptive prioritization, and iterative alignment reviews that account for shifting strategic, regulatory, and market trends. Recognizing the inherent challenges of quantifying strategic fit, the proposed BTM framework adopts a mixed-methods triangulation approach integrating expert judgment, quantitative indicators, and qualitative insights to enhance evaluative robustness. Its operationalization relies on measurable proxies such as knowledge reuse rates, adaptive responses to technological or market disruptions, and team recovery following project setbacks, which collectively capture the dynamic and nonlinear nature of learning and resilience within R&D ecosystems (Gibson & Birkinshaw 2020). Rationale: Performance should reflect not only internal goals but responsiveness to external dynamics.
Team and Organizational Health	Measurement: psychological safety, interdisciplinary collaboration, and network centrality of knowledge exchange. To reduce subjectivity in KPI interpretation, the framework employs structured expert elicitation methods, including the Delphi technique and inter-rater reliability assessment. These procedures ensure consistency across evaluators and improve transparency and replicability (Santiago & Soares, 2021). Psychological safety within R&D teams is increasingly recognized as a key driver of innovation, knowledge sharing, and problem-solving. To move from conceptual understanding to operational management, psychological safety can be implemented through a structured set of SMART indicators that reflect both behavioral dynamics and tangible outcomes. Quantifiable measures include team stability, project success indicators, the number of cross-team problem resolutions, innovation output per member, and engagement and participation indices. Behavioral signals (engagement, collaboration) reveal the presence of a psychologically safe environment, while outcome-oriented indicators (patents, project successes) demonstrate its concrete impact on organizational performance. By integrating these measures into R&D management systems, organizations can not only monitor team dynamics but also design targeted interventions to foster trust, inclusion, and sustainable innovation. Rationale: High-performing R&D projects rely on healthy team dynamics that foster creativity and innovation.
Resilience and Robustness	Measurement: adaptability to shocks (budget cuts, competitor advances) and robustness of outputs under uncertainty, obsolescence index. Finally, measuring resilience and responsiveness to competitor advances requires multidimensional indicators. The revised model incorporates an ‘Adaptability Index,’ capturing project continuity under disruption, and comparative benchmarking against emerging technologies to gauge robustness relative to industry evolution (de la Torre et al., 2023). Rationale: Resilience is an overlooked determinant of long-term R&D value.

Table 1. Performance dimension and description of the suggested BTM Framework.

Performance dimension	Description
Knowledge Creation and Diffusion	Measurement: patents, publications, datasets, methods, and evidence of reuse across other projects., stronger human-centric KPIs such as staff mobility across projects, number of mentoring sessions or internal workshops conducted (mentoring frequency), integration rate of research outcomes into operational workflows (adoption rate of new R&D methods), contribution to professional standards and Sonatrach-Academia linkages (Martinsuo et al., 2022; Nonaka & Takeuchi, 2019). Rationale: R&D value extends beyond immediate deliverables to the creation of knowledge assets with long-term impact.
Innovation Velocity and Learning	Measurement: iteration cycle time, capability accretion index, experiment-to-insight ratio, rate of adoption of lessons across teams. Innovation performance should not be conflated with velocity. While shorter cycle times may improve responsiveness, they do not necessarily indicate deeper learning or technological advancement. Genuine innovation requires iterative experimentation, reflective learning, and the ability to transform failures into insight (Pisano, 2019). Therefore, the measurement system privileges evidence of validated learning, capability accumulation, and knowledge diffusion across the organization. Rationale: Projects that accelerate organizational learning create competitive advantage, regardless of immediate outcomes.
Dynamic Strategic Alignment	Measurement: knowledge reuse rates, adaptive behavior during disruptions, and team recovery after project setbacks. Strategic alignment represents the systematic coherence between organizational objectives, strategic development domains, operational processes, and the resources mobilized to achieve them. In R&D contexts, effective alignment entails translating corporate strategy into actionable portfolio decisions through mechanisms such as thematic road-mapping, adaptive prioritization, and iterative alignment reviews that account for shifting strategic, regulatory, and market trends. Recognizing the inherent challenges of quantifying strategic fit, the proposed BTM framework adopts a mixed-methods triangulation approach integrating expert judgment, quantitative indicators, and qualitative insights to enhance evaluative robustness. Its operationalization relies on measurable proxies such as knowledge reuse rates, adaptive responses to technological or market disruptions, and team recovery following project setbacks, which collectively capture the dynamic and nonlinear nature of learning and resilience within R&D ecosystems (Gibson & Birkinshaw 2020). Rationale: Performance should reflect not only internal goals but responsiveness to external dynamics.
Team and Organizational Health	Measurement: psychological safety, interdisciplinary collaboration, and network centrality of knowledge exchange. To reduce subjectivity in KPI interpretation, the framework employs structured expert elicitation methods, including the Delphi technique and inter-rater reliability assessment. These procedures ensure consistency across evaluators and improve transparency and replicability (Santiago & Soares, 2021). Psychological safety within R&D teams is increasingly recognized as a key driver of innovation, knowledge sharing, and problem-solving. To move from conceptual understanding to operational management, psychological safety can be implemented through a structured set of SMART indicators that reflect both behavioral dynamics and tangible outcomes. Quantifiable measures include team stability, project success indicators, the number of cross-team problem resolutions, innovation output per member, and engagement and participation indices. Behavioral signals (engagement, collaboration) reveal the presence of a psychologically safe environment, while outcome-oriented indicators (patents, project successes) demonstrate its concrete impact on organizational performance. By integrating these measures into R&D management systems, organizations can not only monitor team dynamics but also design targeted interventions to foster trust, inclusion, and sustainable innovation. Rationale: High-performing R&D projects rely on healthy team dynamics that foster creativity and innovation.
Resilience and Robustness	Measurement: adaptability to shocks (budget cuts, competitor advances) and robustness of outputs under uncertainty, obsolescence index. Finally, measuring resilience and responsiveness to competitor advances requires multidimensional indicators. The revised model incorporates an ‘Adaptability Index,’ capturing project continuity under disruption, and comparative benchmarking against emerging technologies to gauge robustness relative to industry evolution (de la Torre et al., 2023). Rationale: Resilience is an overlooked determinant of long-term R&D value.

In fact, each individual BTM dimension (KC, IV, DA, TH, RR) is supported by prior research (knowledge diffusion, TRL/innovation readiness, strategic portfolio alignment, team health, project resilience), and prior frameworks often include 2–3 of these aspects (e.g., TRL + strategic fit, or resilience + risk), so none of the five dimensions alone are fully novel (Table 2):

• Knowledge Creation & Diffusion (KC) — large literatures on knowledge spillovers, bibliometrics, patent-to-industry diffusion and R&D networks. Useful reviews and empirical work show how to measure diffusion (citations, patent citations, network centrality) (Jaffe et al., 1993; Autant-Bernard et al., 2013 ; Audretsch, & Feldmanm 2004;. Lina et al. 2024)
• Innovation Velocity (IV) — the concept (sometimes called innovation speed, experiment velocity, or innovation velocity) is increasingly studied; research groups (e.g., Cambridge IFM’s InnVel work) and recent reviews explore composite measures (cycle time, experiment throughput, conversion rates). It’s an emerging metric space rather than a mature single index (Kessler and Chakrabarti, 1996; Langerak et al. 2004, Jing, 2024)
• Dynamic Strategic Alignment (DA) — strategic alignment for R&D and portfolios (including adaptive alignment across stages) has active empirical work showing firms use continuous alignment practices and bucket methods to keep R&D in sync with strategy (Cooper et al., 2001: Meskendahl, 2010; Santiago et al., 2020; Martinsuo and Anttila, 2022, Martinsuo et al. 2022).
• Team & Organizational Health (TH) — organizational and team health metrics (psychological safety, resilience, cross-disciplinary coordination) are increasingly linked to innovation outcomes in healthcare and IS projects; there is growing empirical evidence that team health predicts project success and resilience. (Edmondson, 1999 ; Hülsheger et al., 2009 ; Atkinson et al. 2020 ;. Atkinson and Singer, 2021 ; Varajão et al., 2023).
• Resilience & Robustness (RR) — an expanding literature applies resilience concepts to projects and R&D (frameworks, case studies, meta-theoretical essays). Resilience as a Where BTM can legitimately claim novelty (publishable claims) (Ahern et al., 2014; Woods, 2015; Naderpajouh et al. 2020, Koilakonda, 2023).

3.2. Framework Originality

The originality of the BTM framework lies in its integration of five R&D performance dimensions into a decision-support architecture grounded in Multi-Criteria Decision-Making (MCDM) and dynamic capability theory. Unlike traditional KPI dashboards, BTM enables predictive governance and interdimensional analysis, enhancing both transparency and decision quality (Martinsuo et al., 2022; Roy, 1996).

This integrated approach allows R&D managers to interpret project outcomes beyond isolated indicators, promoting systemic insights into governance and value creation (Table 3):

• Integration & Equal-Status Dimensions — many studies focus on single dimensions or treat some (e.g., team health) as secondary. Presenting all five as co-equal pillars in a single, operational management framework is a clear synthetic contribution.
• Explicit Stage-Weighted Scoring + Decision Rules — formalizing how weights change with maturity stage (rules, piecewise functions, or ML-driven reweighting) is rarely formalized in literature; operationalizing this is novel.
• Innovation Velocity as a Core Dimension — treating IV as a first-class performance dimension (with a defined composite index) rather than a loose “speed” concept is an opportunity for original methodological contribution.
• Team & Organizational Health embedded into scoring — including TH with measurable KPIs (psychological safety, cross-functional throughput, mentoring) directly in project scoring (not just HR dashboards) is under-explored academically.
• Value-of-Failure operationalization inside RR (measuring retained assets, abandonment reuse index) — explicit KPIs for “value when abandoned” are scarce and will strengthen originality.

4. Methods and Validation Plan for the BTM Framework

4.1. Methods

The BTM Framework evaluates R&D project performance across five dimensions: Knowledge Creation & Diffusion (KC), Innovation Velocity (IV), Dynamic Strategic Alignment (DA), Team & Organisational Health (TH), Resilience & Robustness (RR)

Each dimension is assessed through predefined KPIs and a weighted scoring system to generate an overall performance index (Table 4).

To prevent a purely checklist-based assessment, the BTM framework embeds Multi-Criteria Decision-Making (MCDM) methods to account for subjective and interdependent variables. Predictive analytics support stage-weighted rebalancing by forecasting project outcomes based on maturity and performance trends, thus ensuring data-driven portfolio optimization (Santiago & Soares, 2021; Roy, 1996).

Moreover, although analytically distinct, the five BTM dimensions—Knowledge Creation, Innovation Velocity, Dynamic Alignment, Team Health, and Resilience & Robustness—are interdependent and mutually reinforcing. Correlation analysis reveals that higher Knowledge Creation often enhances Innovation Velocity and Dynamic Alignment, reflecting a systemic interaction among dimensions (Martinsuo et al., 2022).

To enhance interpretability, equal weighting (20% per dimension) is initially applied for baseline comparability. However, scenario-specific and stage-based re-weighting using expert-driven Multi-Criteria Decision Analysis (MCDA) ensures adaptability and empirical validation. Sensitivity analyses confirm the robustness of results under varying weight configurations (Roy, 1996; Santiago & Soares, 2021).

Furthermore, an increase in workshops or prototype iterations does not automatically imply effective knowledge transfer. The framework therefore integrates impact-based metrics—such as adoption rate of solutions, reuse of methodologies, and measurable improvements in project success—to ensure qualitative learning outcomes are captured alongside activity counts (Nonaka & Takeuchi, 2019).

4.2. Scoring & Index Calculation

In this framework, each Key Performance Indicator (KPI) is assessed on a 0–10 scale, where 0 indicates no achievement and 10 represents excellent performance. For each performance dimension, the Dimension Score is calculated by multiplying the average KPI score by its assigned weight. The Overall Performance Index (OPI) corresponds to the sum of all Dimension Scores, with a maximum possible score of 10.0 (100 %). To interpret results, scores between 8.0 and 10.0 are classified as Outstanding, 6.0 to 7.9 as Strong, 4.0 to 5.9 as Acceptable, and below 4.0 as Needs Improvement.

To improve the robustness of project evaluation, scoring thresholds and weighting systems were recalibrated.The threshold values (8.0, 6.0, 4.0) are used as preliminary classification levels derived from pilot data. Future iterations will calibrate these thresholds using statistical benchmarking, historical R&D portfolio data, and external performance standards to enhance empirical validity (Coccia, 2021).

The framework is ecognizing zed through a mixed-methods design that combines qualitative and quantitative approaches.

• Operationalisation of Dimensions: Literature review to define measurable indicators for each dimension.
• Expert Consultation: Workshops with R&D managers and policymakers to refine KPIs and scoring weights.
• Pilot Testing: Apply the framework to a representative portfolio of R&D projects.
• Multi-Criteria Decision Analysis (MCDA): Balance trade-offs between dimensions and assign weights.
• Visualisation: Present results through radar plots, heatmaps, and portfolio dashboards for intuitive interpretation.

Additionally, to mitigate subjectivity from averaging expert inputs, the revised framework employs Delphi-based consensus rounds and Analytical Hierarchy Process (AHP) consistency ratios. These methodological enhancements ensure structured convergence of expert judgments and improve reliability (Santiago & Soares, 2021).

4.3. Processes and Tools for Implementation

To integrate the BTM framework into actual practice, three phases are planned:

• Planning Phase:

  o

  Define multi-layered KPIs combining traditional and novel metrics.

  o

  Establish a “performance contract” ecognizing both success and “intelligent failure.”

Failure management is a critical yet underexplored aspect of R&D performance.

The framework distinguishes between ‘intelligent’ and ‘wasteful’ failures. Intelligent failures generate transferable insights, reusable models, or validated datasets even when objectives are unmet. A ‘Lessons Learned Index’ quantifies the extent to which terminated projects contribute to future learning and innovation (Cannon & Edmondson, 2005).

• Execution Phase:

  o

  Use real-time dashboards and AI-assisted early warning systems to detect weak signals of underperformance (scope drift, communication stress).

  o

  Conduct quarterly dynamic alignment reviews to enable pivots.

• Evaluation Phase:

  o

  Employ a multi-dimensional scorecard including Knowledge Half-Life (duration of knowledge relevance) and Portfolio Contribution Index (cross-project spillover effects).

Knowledge half-life is approximated through bibliometric indicators such as citation decay rates and reuse frequency of project deliverables in subsequent studies. Spillover effects are measured through co-authorship, co-patenting, and cross-project knowledge reuse, acknowledging their temporal and causal limitations (Fleming et al., 2019).

4.4. Validation Plan

Traditional approaches to R&D performance have emphasized measurable outputs while neglecting softer dimensions such as adaptability and resilience.Traditional R&D performance models focus narrowly on cost, time, and tangible outputs. The BTM framework addresses these limitations by integrating learning dynamics, adaptability, and team resilience. This multidimensional view enhances predictive power and governance insight beyond traditional evaluation methods (Martinsuo et al., 2022).

4.5. Expected Outcomes

To maintain usability, the revised BTM framework uses a tiered dashboard separating core KPIs for executive decision-making from diagnostic indicators for project teams. This structure ensures simplicity while preserving analytical depth (Bican & Brem, 2020).

4.6. BTM Benefits

This framework advances current practice by:

• Reframing R&D performance as ecosystemic rather than project-isolated.
• Capturing hidden value from projects that “fail” traditionally but succeed in knowledge or capability building.
• Incorporating resilience and adaptability, critical for innovation under uncertainty.
• Offering a balanced mix of quantitative and qualitative indicators applicable across sectors.

Key challenges include reliable measurement of intangible assets, cultural acceptance of “intelligent failure,” and integrating AI-based monitoring into existing project management systems.

5. A Case Study

This section presents a numerical and decision-oriented application of the BTM framework applied to 10 strategically diverse R&D projects in the Oil & Gas sector. Scores are normalized on a 0–10 scale, with equal weights (20% per dimension) (Table 5). The Overall Performance Index (OPI) is calculated as a weighted aggregate synthesizing the five BTM dimensions into a single managerial indicator.

Classification thresholds used are:

• 8.0 – 10.0 = Outstanding (excellence frontier)

• 6.0 – 7.9 = Strong (strategically robust)

• 4.0 – 5.9 = Acceptable (requires selective improvement)

• < 4.0 = Needs Improvement (critical zone)

Classification counts are:

Outstanding: 0 projects (capability gap identified)

Strong: 6 projects (dominant cluster)

Acceptable: 4 projects (stagnation cluster)

Needs Improvement: 0 projects (no critical failures detected)

To enhance interpretability, radar, heatmaps and bubble plots, which provide richer multi-dimensional insights, have been established (Figure 1, Figure 2 and Figure 3).

• Heatmaps offer diagnostic granularity, highlighting high/low-performing dimensions at a glance.

• Project action matrices provide a strategic portfolio map (de la Torre et al., 2023) (Figure 4).

This shift improves evaluability and makes the visual analytics actionable for executive decision-making.

5.1. Analysis of the Results

To ensure direct managerial usability, a ‘BTM Project Action Matrix’ has been incorporated to connect diagnostic insights with governance decisions. Each cell of the matrix links project performance profiles to recommended actions such as resource reallocation, training, or methodological adaptation, facilitating evidence-based management (Santiago & Soares, 2021) (Figure 4).

5.1.1. Strategic Portfolio Analysis

A. OPI vs. KC (Knowledge Creation)

Objective: Assess the project’s contribution to organizational “IQ” and Knowledge Creation.

• Scale & Continue: P1, P2, P3, P6, P9. These drive both revenue and internal expertise.
• Pivot or Re-scope: P5. High performer but “Knowledge-light.” Re-scope to ensure technical insights and IP are better captured.
• Partner & Support: P8. High knowledge potential but execution is failing. Needs a support partner to turn knowledge into results.
• Abandon or Exit: P4, P7, P10. Projects providing neither actionable results nor strategic learning.

B. OPI vs. IV (Innovation Value)

Objective: Determine if the project provides a competitive edge (Innovation Value) or just maintains the status quo.

• Scale & Continue: P3, P5. True innovators. They deliver high current performance and break new ground for the future.
• Pivot or Re-scope: P1, P2, P6, P9. Efficient “Cash Cows.” High performance but low novelty. Action: Inject R&D to prevent them from becoming obsolete in the next cycle.
• Partner & Support: None.
• Abandon or Exit: P4, P7, P8, P10. Stagnant projects with no future competitive value.

C. OPI vs. DA (Strategic Alignment)

Objective: Measure Strategic Alignment—how directly the project supports core business goals.

• Scale & Continue: P1, P2, P3, P9 are core assets. They drive the business forward and fit the mission perfectly.
• Pivot or Re-scope: P5, P6. Successful but “maverick” projects. They perform well but are drifting from the strategy. Action: Align scope to serve the main mission.
• Partner & Support: None.
• Abandon or Exit: P4, P7, P8, P10. Low performance and no strategic fit; these are “distraction” projects.

D. OPI vs. TH (Team and Organizational Health)

Objective: Measure the impact of the project on Team and Organizational Health.

• Scale & Continue: P1, P2, P5, P6, P9. These projects are successful and contribute to a healthy, sustainable work environment.
• Pivot or Re-scope: P3. A top performer sitting exactly on the health threshold (6.0). Action: Investigate potential team burnout or cultural friction before scaling further.
• Partner & Support: None. No projects show high team health with low performance.
• Abandon or Exit: P4, P7, P8, P10. These projects are underperforming and are likely toxic to team morale or organizational culture.

E. OPI vs. RR (Resilience & Robustness)

Objective: Evaluate if projects are built on a solid foundation to withstand disruptions (Resilience & Robustness).

• Scale & Continue (High OPI/High RR): P1, P2, P3, P5, P9. These are “weather-proof” assets that deliver high performance.
• Pivot or Re-scope (High OPI/Low RR): None. High performance is currently backed by sufficient robustness.
• Partner & Support (Low OPI/High RR): P10. A stable project but underperforming. Seek external efficiency to boost output without altering its solid structure.
• Abandon or Exit (Low OPI/Low RR): P4, P7, P8. Fragile projects with low performance; they represent a high risk of failure under stress.

5.1.2. Overall Portfolio Performance

• The Overall Performance Index (OPI) across the portfolio ranges from 4.2 (42%) to 7.4 (74%), demonstrating moderate dispersion.
• Six projects (P1, P2, P3, P5, P6, P9) are “Strong” (≥6.0 OPI, ≥60%).
• Four projects (P4, P7, P8, P10) remain “Acceptable”, indicating latent improvement potential.
• No project reached the “Outstanding” benchmark (≥8.0), but none fell below 4.0 (critical zone), revealing a strategic opportunity to elevate excellence.
• No project is in the critical failure territory (<4.0), but none are in critical failure territory (<4.0 OPI), indicating baseline operational stability.

5.1.3. High-Performing Projects (“Strong”)

• P2_CCS_Field_Pilot (OPI 7.4) leads the portfolio, excelling in Resilience & Robustness (RR = 9) and Team Health (TH = 8), demonstrating high adaptability and organizational maturity.
• P9_Reservoir_Simulation_Platform (OPI 7.0) shows balanced performance across all dimensions, reflecting a well-structured and strategically coherent project.
• Other “Strong” projects (P1, P3, P5, P6) perform well on IV and KC, but display opportunities to enhance resilience, risk anticipation, and cross-functional alignment.

Implication: These projects form the portfolio’s learning backbone and should be used as internal benchmarks.

5.1.4. Medium-Performing Projects (“Acceptable”)

• P4 (OPI 5.2) and P10 (OPI 5.4) show reasonable performance but lag in Innovation Velocity and Knowledge Creation, suggesting execution bottlenecks and low intellectual capital generation.
• P7 (OPI 4.2) is the weakest performer with consistently low scores, indicating a need for urgent strategic reassessment.
• P8 (OPI 5.6) remains constrained by weak strategic fit and modest resilience, limiting its long-term impact.

Implication: These projects require targeted capability development, scope clarification, and enhanced governance monitoring.

5.1.5. Dimension-Level Insights

• Knowledge Creation & Diffusion (KC): Scores vary from 4 to 8. Projects with low KC could adopt stronger publication, patenting, and cross-project knowledge-transfer policies.
• Innovation Velocity (IV): Mixed scores (3–8). Low IV in some projects highlights prototype delays, weak milestone tracking, or insufficient technical agility.
• Dynamic Strategic Alignment (DA): Scores range 5–8; lower scores indicate projects may be drifting from corporate or market priorities.
• Team & Organizational Health (TH): Scores mostly 5–8; weaker scores may stem from turnover, communication gaps, or insufficient inter-team cohesion.
• Resilience & Robustness (RR): Scores vary 4–9; the best performers excel here, but others risk disruption (e.g., technology disruption) without stronger risk management.

The Resilience & Robustness dimension now incorporates a ‘Technology Disruption Readiness’ indicator assessing how effectively R&D projects anticipate, adapt, and respond to emerging technological shifts that could affect their relevance, by measuring foresight, flexibility, and agility (Coccia, 2021).

5.1.6. Portfolio-Level Patterns

• Dimension imbalance: Many projects show high Innovation Velocity but relatively lower Resilience & Robustness high output speed with potential fragility under uncertainty.(e.g., technology disruption).
• Strategic drift: Some “Acceptable” projects have lower Dynamic Strategic Alignment, suggesting misfit with corporate priorities or market/technology trends.
• Knowledge diffusion gaps: Lower KC in several projects implies insufficient cross-project knowledge transfer, limiting learning spillovers across the portfolio.

5.1.7. Portfolio-Level Recommendations

• Prioritize Support for Acceptable Projects: Reassess scope, ensure strategic alignment, and enhance collaboration tools for P4, P7, P8, P10.
• Replicate Best Practices: Use the governance, team practices, and risk-management strategies of P2 and P9 as models.
• Set “Outstanding” Targets: Raise benchmarks and incentive structures to push top projects beyond OPI 8.0.
• Balance Risk/Reward: Keep a healthy mix of high-risk exploratory projects (like P7) and mature impact-driven, strategically aligned projects (like P2 and P9).

5.1.8. Benchmarking & Sensitivity

• Benchmarking: Compare high-performing P2 and P9 against existing models like Balanced Scorecard or TRL-based assessments to quantify incremental value of the BTM framework.
• Sensitivity analysis: Adjust KPI weights (e.g., raise weight of Resilience to 30% for high-risk fields) to test ranking robustness under alternative strategic scenarios.
• Trendline: The BTM results align with literature calling for multi-dimensional performance metrics and can serve as a baseline for longitudinal monitoring of petroleum R&D portfolios.

5.1.9. Actionable Recommendations (Table 6)

• Scale best practices: Replicate governance, knowledge-sharing, and risk-management processes of P2 and P9 in weaker projects.
• Raise OPI targets: Set “Outstanding” goalposts (≥8.0) and tie incentives to balanced performance across all five dimensions.
• Strategic re-scope: Reposition projects with low DA to better align with emerging energy transitions or market needs.
• Embed resilience: Strengthen scenario planning, redundancy architecture, and early-warning systems for projects with low RR scores.
• Knowledge platforms: Implement structured cross-project seminars or digital platforms to enhance KC metrics portfolio-wide.

6. Conclusion

The Beyond Traditional Metrics (BTM) framework offers a more comprehensive, theoretically grounded, and empirically robust model for evaluating R&D project performance. It unifies quantitative and qualitative dimensions, balancing efficiency, innovation, and adaptability.

The strengthened theoretical linkages (through MCDM and dynamic capability theory), refined KPI definitions, and inclusion of learning- and resilience-based metrics collectively transform BTM from a static performance dashboard into a dynamic decision-support system for strategic R&D management.

Applied to the petroleum industry, the portfolio analysis reveals a solid core of high-performing projects alongside a group of ‘acceptable’ initiatives that could be enhanced or repositioned to increase overall impact. By acting on these insights, managers can improve portfolio balance, strategic alignment, and resource allocation, while transparently optimizing investments in emerging energy-transition technologies such as CCS and hydrogen.

Integrating novel dimensions—knowledge creation, innovation velocity, dynamic alignment, team health, and resilience—the framework moves beyond efficiency-driven metrics to provide a fuller picture of project contributions. Its application enables greater learning, agility, and long-term value creation, ultimately strengthening innovation ecosystems. Future work should focus on empirical calibration of weights and broader benchmarking to refine its relevance across sectors.