Submitted:
01 December 2025
Posted:
29 December 2025
You are already at the latest version
Abstract

Keywords:
1. Introduction
2. Background
2.1. Gaps in Policy and Evaluation Frameworks
2.2. Limits of Innovation Metrics and Organisational Approaches
2.3. Complexity and Network-Based Approaches to Innovation
3. Research Problem
3.1. Aim and Research Question
3.2. Research Strategy
3.3. Application of the Method
- i.
- Should scientometrics be complemented with measures describing how scientific- evolves into technical- innovation?
- ii.
- To which extent should researchers decide on intellectual property and on business models that exploit the innovation they authored in first-place?
- iii.
- How could innovation pathways be valued outside of the network of stakeholders, their affiliated partners or related fields?
3.4. Data Collection
3.5. Participants
3.6. Data Analysis Methods
3.6.1. Data Preparation
3.6.2. Thematic Analysis
3.6.3. Network Analysis
3.7. Research Ethics
4. Results
4.1. Thematic and Network Analysis of Primary Data
- i.
- Opportunities and disadvantages of tracing intermediate innovation-steps,
- ii.
- insights about the potential to unlock value embedded in intermediate innovation steps, with respect to innovation metrics used in EU-funded projects, and
- iii.
- some conditions for unlocking that value.



4.2. Advantages
- Failure as innovation: Failure is perceived as a form of innovation in itself (R1:28). Failure provides information on what assets did not produce actionable results and helps avoid faulty repetitions (R1:31;R1:75;R2:84).
- Integration of heterogeneous data: Researchers expressed interest in services that allow integration of diverse datasets to forecast the outcome of experiments by adjusting asset configurations, typically in the form of different bodies of materials and methods (R1:81-84; R2:41; R2:89; R2:135):
“So an output I would expect of a service for supporting research is in helping us generate hypotheses and experimental design [..] to first construct a functional prototype and then improve performance on manyfolds aspects of the KPIs we’re using.” (R2:41)
- Extension to intangible assets: These insights may enrich existing literature [17] by extending process-asset mapping models to include intangible assets such as experimental settings, material usage, required knowledge on procedures and methods from scientific literature and/or patents, required knowledge from prior tested procedures and in general any source of information that is used in the process of ideation (R1:25; R1:41-R1:45; R2:41; R3:10). Any intermediate steps, also ones not producing immediate actionable results, contributes information towards subsequent steps (R1:86; R1:91; R2:41). The last innovation-step represents an embodiment of all previous pathways that led to it.
- Recommended timescale for tracing observations: Respondents supported regular monitoring of innovation steps, ideally every two weeks but no less than once per month (R0:123; R1:133; R2:111; R3:30).
- Need for new metrics: A respondent introduced the metaphor of two innovation “geometries”, that we can associate to the concepts of innovation and invention previously discussed:
This motivates the development of new key performance indicators (KPIs) based on in-itinere analysis of prior steps, rather than relying solely on ex-ante or ex-post metrics on bibliometrics. Such metrics could lift on the historical path of prior results towards new options for valorising intermediate results that were not planned (R3:90). KPIs based on innovation paths could be integrated in IT-frameworks for supporting decisions towards the next R&D process (R1:78). For example, logging innovation-steps and querying innovation-paths backward would help to summarise prior knowledge towards the next experimental configuration (R1:27 : R1:29; R1:41), and capitalise on time invested on not-yet successful results. For instance, trials can last months, and in between there are many innovation-steps with many “failures”.“The horizontal one, where innovation is generated by connecting distant fields; the vertical one, where innovation emerges suddenly by an unexpected event.” (R3:16–R3:18)
- All respondents agreed that communication style must depend on the type of audience.
- For marketing (R0:96) activities, disclosing visual representations of intermediate innovation-steps can be functional to engage in open science (R3:35-R3:37; R1:54) or suited for sharing results to non-technical audiences (R3:163; R1:61).
- For evaluating patentable results of innovation steps (R0:129 : R0131; R0:133; R0:139; R2:71;R2:74-R2:75), prior innovation pathways could be useful to identify what has been tested before (R1:78) and to extract quantitative insights as alternatives to scientometric indicators (R3:25), such as the possibility of forecasting of a technological saturation of patents associated to candidate innovation-steps [35]. A fortnightly update pace was suggested to support periodic patentability assessments (R0:122–R0:129).
-
Tracing failures can provide:
- -
- Knowledge of previously tested approaches
- -
- Know-how for successful replication
- -
- Basis for generating new hypotheses (R2:41)
- -
- Complement to literature that omits failure cases or practices to avoid
-
Tracing innovation steps can support:
- -
- Resource planning and experimental output forecasting
- -
- Discovery of new options to valorize an output, beyond traditional scientometrics or surveys (R1:17; R1:20; R1:27–R1:31; R1:75; R1:84; R2:85; R0:49; R0:63)
- Asset categories should include intangible resources, (e.g., knowledge, know-how, experimental settings
- Tracing should record the purpose and expectations for asset use
- The input interface should use minimalist design to reduce friction
- Pathways should be visualised through graphs and charts
- Integration with existing organisational databases is crucial for mapping and comparing use of resources (for example, compare the chemical properties of materials used in experiments)
- Communication tools should adapt to audiences, interactive maps may help to schematise innovation (knowledge graphs for explaining what an innovation is and what are the outcomes and how they are produced ) and interactive charts may help to summarizing prior pathways at desire time-scale (all prior steps to a given point in time). Traditional media, especially short-videos for social network dissemination, are envisioned as most effective for engaging in open science
4.3. Disadvantages
- Intellectual property concerns: Some respondents questioned current intellectual property rules, where the EU supports research with public funding but property rights of any invention are only ascribed to the private companies of the consortium (R1:155–R1:158; R3:151; R0:152).
- Surveys’ weakness: The reliability of surveys as method to report results to the EU was critiqued because the way that questions are posed or rephrased in function of target stakeholders, could yield different answers.
- Authorship and ownership: Respondents were challenged if tracing innovation pathways should also trace authorship and grant a legal value to authors for utilizing the intermediate-steps they contributed for future innovation outside of the original project (R3:56; R3:113; R1:165; R0:152). There was a mixed view if an author should be free to carry-on development over unused innovation-steps on an independent basis, but a general agreement if done in a context of business and entrepreneurship such as with affiliated spin-off companies.
- Resource burden: Introducing a new workload for granular tracing of innovation-steps is expected to increase time-costs and might even call for full-time duties (R1:35; R3:101; R2:150–151). However, if attuned to current practices and a plausible periodicity (fortnightly), time-cost was perceived as an investment (R1:130; R1:148; R3:101).
- Limited applicability: Tracing innovation paths could only be relevant for internal use or dissemination, as TRL metrics must use consolidated approaches (i.e. bi-monthly surveys).
-
Perception bias:
- -
- A tendency to discuss opportunities more than drawbacks, possibly due to participants’ roles in research space may be inclined to discussing options more than conservative management roles.
- -
- Unclear if tools for tracing innovation pathways would be up-taken by small organisations or organisation with tight schedules.
5. Discussion
5.1. Design Implications for Prototyping Tools to Assess Innovation Emerging from Knowledge-Networks
- Inputs: Resources used by the process (materials, data about experimental conditions, procedures extracted from scientific literatures and/or patents).
- Processes: Conceptually representing any process (experiments, procedures) as a function with arity equal to the number of inputs which return at least one output.
- Outputs: Results, or outputs, of the process, including both successful and failed attempts.



5.2. Policies Implications of Non-Linear Innovation Emergence
- Accelerate innovation by identifying critical turning points and bottlenecks within projects.
- Allocate funding not only based on expected outcomes, but also on the diversity and recombination potential of exploratory steps.
- Encourage the diffusion of negative or null results, thus reducing systemic inefficiencies.
- Recognise and valorize contributions beyond the traditional patent or publication models.
5.3. Theoretical Implications of Latent Value of Exploration in Innovation Networks
- Innovation emerges from temporal networks that encode the transformation of resource sets into new ones.
- In scientific research, failed or inconclusive steps have intrinsic value by informing what does not work.
- In entrepreneurial systems, the value of innovation only depends on predefined metrics or on outcomes that can be rendered proprietary.
- This asymmetry leads to under-utilisation of public investments and a loss of knowledge for recombining resources, in future market or in alternate domains (e.g. industrial fields with less technical saturation).
- If contributions, whether successful or not, are encoded as structured data in a network of innovation steps, their potential to recombine resources into novelty can be traced, quantified, and valorized.
- Graph representations of economic landscapes can help to objectify the value of innovation in function of its topology, thus beyond proprietary outputs accrued by a set of nodes, and support new evaluation metrics integrated with policy design that can gauge the trade-off between impact contributed to the system (i.e. the graph) and impact accrued from nodes who hold proprietary or exploitative rights.
- (corollary) Graph representation methods can be directly integrated with machine and representation learning to model innovation dynamics, enabling the mapping of contributions and their authorship to the emergent properties of the network.
5.4. Expected Impact and Follow-Ups
6. Conclusions
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
| AI | Artificial Intelligence |
| LLMs | Large Language Models |
| RAG | Retrieval Augmented Generation |
| TRLs | Technology Readiness Levels |
Appendix A
| TRL Level | Objective | Description | Main Activities |
|---|---|---|---|
| TRL 1 | Basic observations and reports | Scientific research begins and results are translated into future R&D. | Scientific reviews |
| TRL 2 | Technology concept | Basic principles have been studied, and practical applications are considered. | Generation of ideas, hypotheses, candidate parameters, and initial testing |
| TRL 3 | Experimental functions in proof-of-concept | Active research and design begin, including construction of proof-of-concept models. | Hypothesis testing |
| TRL 4 | In-laboratory validation (components) | Proof-of-concept technology is ready; multiple components tested together. | Proof-of-concept demonstration |
| TRL 5 | On-field validation (components) | Breadboard technology tested in near-real environments. | Pilot deployment |
| TRL 6 | On-field validation (prototype) | Fully functional prototype developed and tested. | Prototype testing (e.g., safety) |
| TRL 7 | On-field demonstration (working model) | Prototype demonstrated in real environment. | Prototype testing (e.g., efficacy) |
| TRL 8 | On-field testing (working model) | Technology tested and qualified for implementation. | Patenting, registration |
| TRL 9 | Deployment and industrialisation (product) | Technology approved (regulatory compliant). | Legal compliance for market distribution |
| TRL | Common Assessment Methods | Example Metrics |
|---|---|---|
| TRL 1 | Literature review, expert interviews | Number of supporting publications, novelty score |
| TRL 2 | Feasibility studies, concept design documents | Completeness of design spec, clarity of use case |
| TRL 3 | Lab experiments, bench tests | Prototype functionality %, performance vs. spec |
| TRL 4 | Controlled environment trials | Reliability %, reproducibility rate |
| TRL 5 | Simulated real-world tests, stakeholder observation | Performance stability, stakeholder satisfaction score |
| TRL 6 | Pilot projects, field trials | Operational uptime %, defect rate, feedback loops implemented |
| TRL 7 | Full-scale trial in operational setting | System interoperability score, safety incidents logged |
| TRL 8 | Certification tests, QA audits | Compliance % to standards, mean time between failures |
| TRL 9 | Post-deployment monitoring | ROI, adoption rate, market penetration % |
| Dimension | Purpose |
|---|---|
| TRL (Technology) | Tracks the technical maturity of the solution (as already established) |
| SRL (Societal) | Measures societal awareness, acceptance, and adoption of the solution |
| ORL (Organizational) | Measures the public administration’s ability to integrate and sustain the solution |
| LRL (Legal) | Measures the solution’s compliance with existing laws and regulatory alignment |
| 1 | A problem that in the private sector is mitigated by stock-option incentives, while might suffer from limited applicability to the public sector, and that may, as Mazzucato notes, be abused by top management to extract value rather than create it. |
References
- De Weerd-Nederhof, P. Innovation Management, Strategy and Implementation Using the Pentathlon Framework. R&D Management 2007, 37, 90–92. [Google Scholar] [CrossRef]
- Burroughs Wellcome Fund.; Howard Hughes Medical Institute. Making the Right Moves. https://www.hhmi.org/science-education/programs/resources/making-right-moves, 2006. Accessed: 2022-08-21.
- de Vaan, M. Etymological Dictionary of Latin: and the other Italic Languages; Brill: Leiden, The Netherlands, 2008.
- Mallory, J.P.; Adams, D.Q. The Oxford Introduction to Proto-Indo-European and the Proto-Indo-European World; Oxford University Press, 1997. [CrossRef]
- Cicero, M.T. De Inventione; Number 386 in Loeb Classical Library, Harvard University Press: Cambridge, MA, 1949. Original Latin text, written ca. 55 BCE.
- Wiener, N. Cybernetics or Control and Communication in the Animal and the Machine; The MIT Press, 2019; [https://direct.mit.edu/book-pdf/2254528/book_ 9780262355902.pdf]. [CrossRef]
- von Foerster, H. Understanding Understanding: Essays on Cybernetics and Cognition; Springer: New York, 2003.
- Schumpeter, J.A. The Theory of Economic Development: An Inquiry into Profits, Capital, Credit, Interest, and the Business Cycle; Harvard University Press: Cambridge, MA, 1934. First English edition translated from the 1911 German original Theorie der wirtschaftlichen Entwicklung.
- Yolles, M.; Chiolerio, A. Quantum-Informed Cybernetics for Collective Intelligence in IoT Systems. Preprints 2025. [Google Scholar] [CrossRef]
- Mazzucato, M. Mission-oriented innovation policies: challenges and opportunities. Industrial and Corporate Change 2018, 27, 803–815. [Google Scholar] [CrossRef]
- Johnston, L.; Fenwick, J. New development: Public service innovation. Public Money & Management 2025, 45, 151–156. [Google Scholar]
- Mazzucato, M. From Market Fixing to Market Creating: A New Framework for Innovation Policy*. In Innovation Systems, Policy and Management; Niosi, J., Ed.; Cambridge University Press: Cambridge, 2018; pp. 79–103. Niosi, J. (Ed.). [CrossRef]
- Meyer, M. Tracing knowledge flows in innovation systems. Scientometrics 2002, 54, 193–212. [Google Scholar] [CrossRef]
- Bruno, I.; Lobo, G.; Covino, B.V.; Donarelli, A.; Marchetti, V.; Panni, A.S.; Molinari, F. Technology readiness revisited: a proposal for extending the scope of impact assessment of European public services. In Proceedings of the Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance, Athens Greece, sep 2020; pp. 369–380. [CrossRef]
- Townsend, W.R. Innovation and the value of failure. International journal of management and marketing research 2010, 3, 75–84. [Google Scholar]
- Hartmann-Sonntag, I.; Scharnhorst, A.; Ebeling, W. Modelling selforganization and innovation processes in networks. arXiv preprint cond-mat/0406425 2004. [Google Scholar] [CrossRef]
- Bider, I.; Perjons, E.; Elias, M.; Johannesson, P. A fractal enterprise model and its application for business development. Software and Systems Modeling 2017, 16, 663–689. [Google Scholar] [CrossRef]
- Hensen, J.L.; Loonen, R.; Archontiki, M.; Kanellis, M. Using Building Simulation for Moving Innovations across the ‘Valley of Death.’. REHVA Journal 2015, 52, 58–62. [Google Scholar]
- Sachs, J.D.; Schmidt-Traub, G.; Mazzucato, M.; Messner, D.; Nakicenovic, N.; Rockström, J. Six Transformations to achieve the Sustainable Development Goals. Nature Sustainability 2019, 2, 805–814. Publisher: Nature Publishing Group. [CrossRef]
- Assom, L.; Codello, A. Multi-partite graph database. US-20150169758-A1, 2013.
- Robinson, D.K.R.; Huang, L.; Guo, Y.; Porter, A.L. Forecasting Innovation Pathways (FIP) for new and emerging science and technologies. Technological Forecasting and Social Change 2013, 80, 267–285. [Google Scholar] [CrossRef]
- Robinson, D.K.; Huang, L.; Guo, Y.; Porter, A.L. Forecasting Innovation Pathways (FIP) for new and emerging science and technologies. Technological Forecasting and Social Change 2013, 80, 267–285. [Google Scholar] [CrossRef]
- Shi, Y.; Yang, A.J.; Deng, S. Entropy in Science of Science. arXiv 2025, arXiv:2503.20543. [Google Scholar] [CrossRef]
- Haar, P.t. Measuring innovation: A state of the science review of existing approaches. Intangible Capital 2018, 14, 409–428. [Google Scholar] [CrossRef]
- Alkan, A.K.; Sourav, S.; Jablonska, M.; Astarita, S.; Chakrabarty, R.; Garuda, N.; Khetarpal, P.; Pióro, M.; Tanoglidis, D.; Iyer, K.G.; et al. A Survey on Hypothesis Generation for Scientific Discovery in the Era of Large Language Models. arXiv 2025, arXiv:2504.05496. [Google Scholar] [CrossRef]
- Hülsheger, U.; Anderson, N.; Salgado, J. Team-level predictors of innovation at work: a comprehensive meta-analysis spanning three decades of research. Journal of Applied Psychology 2009, 94, 1128–1145. [Google Scholar] [CrossRef] [PubMed]
- Stieler, M.; Henike, T. Innovation nudging—A novel approach to foster innovation engagement in an incumbent company. Creativity and Innovation Management 2022, 31, 35–48. [Google Scholar] [CrossRef]
- Banua, G.S. Measuring innovation using key performance indicators. In Proceedings of the Procedia Manufacturing, 2017. 11th International Conference Interdisciplinarity in Engineering, INTER-ENG 2017, 5-6 October 2017, Tirgu Mures, Romania.
- Al-Baik, O.; Miller, J. The kanban approach, between agility and leanness: a systematic review. Empirical Software Engineering 2015, 20, 1861–1897. [Google Scholar] [CrossRef]
- Barjolle, D.; Midmore, P.; Schmid, O. Tracing the Pathways from Research to Innovation: Evidence from Case Studies. EuroChoices 2018, 17, 11–18. [Google Scholar] [CrossRef]
- Katz, J.S. What is a complex innovation system? PloS one 2016, 11, e0156150. [Google Scholar] [CrossRef]
- Chiolerio, A. Liquid Cybernetic Systems: The Fourth-Order Cybernetics. Advanced Intelligent Systems 2020, 2, 2000120. [Google Scholar] [CrossRef]
- Hausmann, R.; Yildirim, M.A.; Chacua, C.; Hartog, M.; Matha, S.G. Innovation policies under economic complexity. World Intellectual Property Organization (WIPO) Economic Research Working Paper Series 2024.
- Trappey, A.J.; Liang, C.P.; Lin, H.J. Using machine learning language models to generate innovation knowledge graphs for patent mining. Applied Sciences 2022, 12, 9818. [Google Scholar] [CrossRef]
- Aletti, G.; Crimaldi, I.; Ghiglietti, A.; Nutarelli, F. Modeling Innovation Ecosystem Dynamics through Interacting Reinforced Bernoulli Processes. arXiv 2025, arXiv:2505.13364. [Google Scholar] [CrossRef]
- Gao, X.; Zhang, Z.; Xie, M.; Liu, T.; Fu, Y. Graph of AI Ideas: Leveraging Knowledge Graphs and LLMs for AI Research Idea Generation. CoRR 2025.
- Dong, X.L. Generations of knowledge graphs: The crazy ideas and the business impact. arXiv 2023, arXiv:2308.14217. [Google Scholar] [CrossRef]
- Maria, A.; Puente, D.; María, J.; Young, S. D5.1 Report on the State of the Art of innovation impact assessment.
- Eisenhardt, K.M.; Graebner, M.E. Theory building from cases: Opportunities and challenges. Academy of management journal 2007, 50, 25–32. [Google Scholar] [CrossRef]
- Gregor, S.; Hevner, A.R. Positioning and presenting design science research for maximum impact. MIS quarterly 2013, 337–355. [Google Scholar] [CrossRef]
- COgITOR Project Consortium. COgITOR Project: Cybernetic Organic Intelligent Technologies for Open Regeneration. https://www.cogitor-project.eu, 2025. Accessed: 28 October 2025.
- Zhou, X.; Huang, L.; Porter, A.; Vicente-Gomila, J.M. Tracing the system transformations and innovation pathways of an emerging technology: Solid lipid nanoparticles. Technological Forecasting and Social Change 2019, 146, 785–794. [Google Scholar] [CrossRef]
- Kader, G.D.; Perry, M. Variability for categorical variables. Journal of statistics education 2007, 15. [Google Scholar] [CrossRef]



| Phase | Interpretation | ||
|---|---|---|---|
| Invention | System explores new configurations: variety increases, structure decreases. | ||
| Innovation | System consolidates coherent local patterns within a rich global state. | ||
| Stagnation | The network evolves to informational homeostasis. |
| Physics Term | Interpretation for Innovation Studies |
|---|---|
| (Shannon entropy) | Global diversity of information distributed across nodes. Describe the variety of ideas. |
| (Fisher information) | Local coherence. Structural coherence of collaboration or specialization in a field. |
| (innovation functional) | Balanced innovation objective: Trade-off between exploration of novel ideas and consolidation of existing ones. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
