From Trustworthy AI to Superintelligence Governance: Lessons from the ALTAI Framework and SHAPES Pilots

1. Introduction

Artificial intelligence is undergoing a profound transition. Systems that were once narrowly optimized for specific tasks are increasingly exhibiting characteristics associated with higher levels of autonomy, adaptability, and generality. This trajectory toward artificial general intelligence (AGI), and potentially superintelligence, challenges not only existing technical paradigms but also the ethical, organizational, and societal frameworks through which AI systems are governed. As AI capabilities expand, ensuring trustworthiness, accountability, and meaningful human oversight becomes a central concern rather than a peripheral one.

To date, most ethical and governance frameworks for AI have been developed in the context of domain-specific or narrowly scoped systems. Concepts such as transparency, human agency, robustness, and accountability have been operationalized through practical instruments intended to support developers, deployers, and organizations in assessing and managing AI-related risks. One of the most widely recognized examples is the Assessment List for Trustworthy Artificial Intelligence (ALTAI), developed under the auspices of the European Commission’s High-Level Expert Group on AI. ALTAI translates high-level ethical principles into a structured self-assessment framework, aiming to make Trustworthy AI actionable in real-world development and deployment contexts [1].

While ALTAI and similar frameworks have proven valuable in raising awareness and structuring ethical reflection, their long-term relevance in the face of increasingly autonomous and adaptive AI systems remains an open question. Governance approaches based on static, organization-centric self-assessment may encounter fundamental limitations when applied to systems that evolve over time, exhibit emergent behavior, or operate within complex multi-agent environments. These challenges are particularly salient in discussions of AGI and superintelligence, where traditional assumptions about controllability, predictability, and bounded impact may no longer hold.

The goal of this paper is to assess how well the ALTAI fits the governance context of increasingly advanced AI systems, including AGI and prospective superintelligence. To achieve this goal, the paper provides an analytical and conceptual review of ALTAI as a governance instrument, examining its underlying assumptions, strengths, and structural limitations beyond its original role as a Trustworthy AI self-assessment framework. Empirical insights from the SHAPES project pilots [2], where ALTAI was applied to evaluate AI-based technologies in real-world healthcare and ageing-related settings, are used as an evidence-informed starting point. These experiences provide practical lessons on how Trustworthy AI frameworks function in complex socio-technical environments, offering a basis for evaluating their relevance under increasing levels of autonomy and complexity. The central argument of the paper is that ALTAI should be understood as an early governance prototype rather than a comprehensive solution for advanced AI oversight. While it successfully operationalizes key ethical principles such as human oversight, transparency, and accountability, it also reveals important limitations related to scalability, dynamism, and long-term risk management. By critically examining these strengths and limitations, the paper contributes to ongoing discussions on how existing Trustworthy AI approaches can be extended and integrated into more robust governance frameworks suitable for AGI and superintelligence.

The remainder of the paper is structured as follows. Following this introduction, the rest of Section 1 reviews the foundations of Trustworthy AI and introduces ALTAI within the broader landscape of AI governance and safety frameworks. Section 2 describes the study’s analytical approach and the role of empirical insights derived from the SHAPES project pilots. Section 3 presents findings from the SHAPES pilots and evaluates ALTAI’s strengths, limitations, and scalability in the context of increasingly advanced AI systems, including AGI and prospective superintelligence. Section 4 explores how existing Trustworthy AI governance approaches could be extended to address the challenges of AGI and superintelligence through the proposed ALTAI+ governance stack. Section 5 discusses the implications of the findings for governance research, organizational practice, and policymaking, and considers the limitations of the study together with directions for future research. Finally, Section 6 concludes the paper by reflecting on the role of ALTAI as an early governance prototype and on the evolution of AI governance toward more advanced intelligence systems.

1.1. Background: Trustworthy AI and Governance Frameworks

The concept of Trustworthy Artificial Intelligence emerged in response to growing concerns about the societal, ethical, and legal implications of AI systems [3,4,5]. In the European context, this concept was formally articulated in the Ethics Guidelines for Trustworthy AI published by the European Commission’s High-Level Expert Group on Artificial Intelligence (AI HLEG) [6]. Its Ethics Guidelines define Trustworthy AI as AI that is lawful, ethical, and robust, emphasizing that all three dimensions must be fulfilled simultaneously throughout the AI system lifecycle [7].

To operationalize these high-level ethical imperatives, the Guidelines introduce seven concrete requirements: human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental well-being; and accountability [7]. Importantly, these requirements are not presented as abstract values but as governance-relevant dimensions that must be addressed through organizational processes, technical design choices, and oversight mechanisms. This framing reflects an explicit shift from purely normative ethics toward governance-oriented ethics, where responsibility is distributed across human actors, technical systems, and institutional arrangements.

Within this framework, governance is understood broadly as the set of structures, processes, and practices through which AI systems are designed, deployed, monitored, and controlled. Rather than focusing solely on algorithmic performance or technical safety, Trustworthy AI emphasizes lifecycle-wide accountability, documentation, and the capacity for meaningful human intervention. This perspective is particularly relevant for advanced AI systems, whose behavior may evolve and whose impacts extend beyond narrowly defined application contexts.

1.2. The Assessment List for Trustworthy Artificial Intelligence (ALTAI)

Building on the Ethics Guidelines, ALTAI [1] was released in 2020 as a voluntary self-assessment tool intended to make Trustworthy AI practically actionable. Developed by the AI HLEG after an extensive piloting phase involving a broad range of stakeholders, ALTAI translates the seven Trustworthy AI requirements into a structured set of questions and prompts that organizations can use to assess their AI systems and surrounding governance practices.

ALTAI is explicitly designed as a non-binding and non-certifying instrument [1]. Its purpose is not to determine regulatory compliance, but to support internal reflection, risk identification, and organizational learning. The tool targets developers, deployers, and procurers of AI systems, encouraging them to consider ethical and governance issues across the entire lifecycle of an AI system, including design, training, deployment, operation, and monitoring. In this sense, ALTAI functions as a governance scaffold rather than a technical control mechanism.

A defining characteristic of ALTAI is its emphasis on self-assessment. Organizations are expected to interpret the questions considering their specific context, sector, and use case, and to document how relevant risks are identified and mitigated. This flexibility has been identified as both a strength and a limitation. On the one hand, it allows ALTAI to be applied across diverse domains and organizational settings. On the other hand, it places significant responsibility on organizations to possess sufficient expertise, incentives, and resources to conduct meaningful assessments, raising questions about consistency, comparability, and long-term effectiveness.

1.3. ALTAI in the Broader Landscape of AI Governance and Safety

ALTAI occupies a distinctive position within the broader landscape of AI governance and safety frameworks. Unlike technical AI safety approaches that focus on robustness, alignment, or formal verification [8,9], ALTAI primarily addresses organizational and socio-technical dimensions of trustworthiness. It complements, rather than replaces, technical safety measures by emphasizing documentation, oversight, and accountability structures that surround AI systems.

Prior research has examined ALTAI’s strengths and limitations, noting its value in bridging the gap between high-level ethical principles and operational practices, while also highlighting challenges related to adoption, interpretability, and enforcement. These challenges become increasingly pronounced as AI systems grow more complex, adaptive, and autonomous. In such contexts, governance frameworks based on static, periodic self-assessment may struggle to capture dynamic risks, emergent behavior, and long-term societal impacts [10].

This tension is particularly salient when considering the transition from narrow AI systems to AGI and, potentially, superintelligence. Advanced systems may operate across multiple domains, interact with other autonomous agents, and modify their behavior in ways that are not fully foreseeable at design time. These characteristics challenge core assumptions underlying many existing governance tools, including ALTAI—such as bounded system behavior, stable organizational control, and clearly identifiable points of human oversight.

Against this backdrop, ALTAI can be interpreted as an early governance prototype: a tool that reflects current understandings of Trustworthy AI, while also revealing important gaps that must be addressed as AI capabilities advance. Understanding these gaps, and the lessons that can be drawn from real-world applications of ALTAI, is essential for informing the development of governance frameworks capable of addressing the risks and responsibilities associated with AGI and superintelligence.

2. Materials and Methods

2.1. Research Approach

This study adopts a conceptual and analytical approach informed by empirical insights derived from the SHAPES project pilots [2]. The study builds upon previously reported experiences of applying the Assessment List for Trustworthy Artificial Intelligence (ALTAI) to evaluate AI-based technologies in real-world healthcare and ageing-related contexts [11]. Rather than generating new empirical data or conducting a systematic literature review, the paper uses these pilot experiences as an evidence-informed basis for examining the suitability of ALTAI as a governance instrument for increasingly advanced AI systems.

The study is motivated by the observation that many contemporary discussions on Artificial General Intelligence (AGI) and superintelligence governance rely primarily on theoretical or speculative perspectives. In contrast, this paper seeks to build on practical governance experience by analyzing how an existing Trustworthy AI framework performs when confronted with governance challenges that are expected to become more significant as AI systems increase in autonomy, adaptability, and societal impact.

The analysis is guided by three research questions:

• What strengths does ALTAI demonstrate as a practical governance instrument in real-world deployment contexts?
• What limitations become visible when ALTAI is applied to complex socio-technical systems?
• Which elements of ALTAI remain relevant, and which require extension, in the context of AGI and prospective superintelligence governance?

2.2. Empirical Basis: Insights from SHAPES Pilots

The empirical foundation of this study is derived from experiences reported in the SHAPES project [11], where ALTAI was used as a voluntary self-assessment instrument to evaluate AI-enabled solutions deployed in real-world healthcare and ageing-related environments. The pilot activities involved different technological solutions, organizational settings, and stakeholder groups across multiple European countries, providing a diverse context for examining the practical application of Trustworthy AI governance principles.

Within SHAPES, ALTAI was used primarily as a tool for ethical reflection, governance awareness, and organizational learning rather than as a compliance or certification mechanism. Assessments were conducted by multidisciplinary teams that included technical experts, domain specialists, and organizational stakeholders. This setting reflects a realistic governance environment in which AI systems interact with users, institutional processes, data infrastructures, and regulatory requirements.

The present study does not reproduce or reanalyze the original SHAPES empirical data. Instead, it draws upon previously reported observations and lessons learned from the pilots to identify governance-related strengths, limitations, and challenges associated with ALTAI. These observations are used as a starting point for broader conceptual reflection on governance requirements for increasingly advanced AI systems.

2.3. Analytical Framework

The analytical process consists of interpreting the lessons derived from the SHAPES pilots through the lens of contemporary AI governance discussions. Particular attention is given to how the assumptions embedded within ALTAI—such as human oversight, organizational accountability, transparency, and lifecycle governance—would perform under conditions of increasing AI capability and autonomy.

The analysis is structured around three dimensions:

• Strengths, examining which governance functions ALTAI performs effectively in practice.
• Limitations, identifying governance challenges revealed through practical application.
• Scalability, assessing which elements of ALTAI could remain relevant and which would require modification as AI systems evolve toward AGI and potentially superintelligence.

This framework enables the paper to move beyond evaluating ALTAI solely as a Trustworthy AI assessment tool and to consider its broader significance as an early governance prototype. Rather than asking whether ALTAI itself is sufficient for governing AGI or superintelligence, the analysis explores what its practical use reveals about the governance capabilities, institutional structures, and oversight mechanisms that may be required for future generations of AI systems.

2.4. Use of AI

AI-assisted language model, Microsoft 365 Copilot (GPT-5-based AI assistant), supported the structuring and linguistic refinement of the manuscript. The AI tool was used as a writing aid and did not contribute to the generation of empirical data, the interpretation of results, or the formulation of scientific conclusions. All content was reviewed and validated by the authors, who take full responsibility for the manuscript.

3. Results

The results will begin by looking into the SHAPES pilots and assessing how ALTAI performed in practice. This is followed by moving into assessing ALTAI in the Context of AGI and Superintelligence.

3.1. Empirical Insights from SHAPES Pilots

The empirical insights from SHAPES pilots will be covered by the observed strengths, practical limitations and the relevance for advanced AI governments.

3.1.1. Observed Strengths of ALTAI in Practice

One of the most consistent observations from the SHAPES pilots was ALTAI’s effectiveness as a structuring device for ethical and governance discussions. The assessment questions prompted organizations to articulate responsibilities, oversight mechanisms, and risk mitigation strategies that were often implicit or undocumented prior to the assessment. In this sense, ALTAI functioned as a catalyst for organizational learning rather than as a technical evaluation tool.

ALTAI also proved valuable in fostering cross-disciplinary dialogue. Because the assessment is not framed in purely technical terms, it enabled meaningful participation from non-technical stakeholders, including healthcare professionals and project managers. This contributed to a more holistic understanding of trustworthiness that encompassed technical robustness, data governance, human oversight, and societal impact.

Another observed strength was ALTAI’s flexibility. The self-assessment format allowed organizations to interpret questions considering their specific use cases and constraints. This adaptability made it possible to apply the framework across diverse pilot settings without extensive customization. From a governance perspective, this flexibility supports broad adoption, particularly in early-stage or experimental deployments where rigid assessment criteria may be impractical.

3.1.2. Practical Limitations Identified in the Pilots

At the same time, the SHAPES pilots revealed several limitations inherent in ALTAI’s design. A recurring challenge concerned the static nature of self-assessment. ALTAI assessments were typically conducted at specific points in time, whereas the AI systems under evaluation continued to evolve during pilot operation. This temporal mismatch made it difficult to account for changes in system behavior, data use, or operational context that emerged after the initial assessment.

Another limitation related to interpretive variability. Because ALTAI relies on organizations to judge the relevance and adequacy of governance measures, assessment outcomes depended heavily on the expertise, risk awareness, and incentives of the assessing team. While this flexibility is a strength in terms of applicability, it also introduces inconsistencies that complicate comparison across systems or organizations.

The pilots further highlighted challenges in addressing system-level and emergent risks. ALTAI is primarily oriented toward individual AI systems and clearly defined use cases. In practice, however, several SHAPES solutions interacted with broader digital ecosystems, including external data sources, legacy systems, and human decision-making processes. Capturing the ethical and governance implications of such interactions proved difficult within the confines of a system-centric self-assessment framework.

3.1.3. Relevance of SHAPES Insights for Advanced AI Governance

Although the SHAPES pilots focused on domain-specific AI applications, the empirical insights derived from these experiences have broader relevance for discussions on AGI and superintelligence governance. In particular, the observed limitations of static, organization-centric self-assessment become more pronounced as AI systems increase in autonomy, adaptability, and scope of operation.

The pilots illustrate how governance challenges can arise even in relatively constrained AI systems when system behavior evolves over time or when responsibilities are distributed across organizational boundaries. These dynamics foreshadow governance issues that are expected to intensify in AGI-level systems, where emergent behavior, multi-agent interaction, and long-term societal impact are central concerns.

Taken together, the SHAPES pilots suggest that while ALTAI provides a valuable foundation for Trustworthy AI governance, its practical application exposes structural assumptions that may not hold in more advanced AI scenarios. These empirical insights therefore serve as an evidence-informed basis for the analytical evaluation presented in the following section, where ALTAI is examined explicitly in relation to the governance requirements of AGI and superintelligence.

3.2. ALTAI in the Context of AGI and Superintelligence

Considering ALTAI in the context of AGI and superintelligence, we will cover the analysis from three angles: strengths, limitations and scalability. Table 1 presents an overview of the analysis.

3.2.1. ALTAI as a Governance Prototype: What It Operationalizes Well

A central value of ALTAI lies in its ability to translate normative Trustworthy AI principles into governance-relevant questions that organizations can act upon. ALTAI is explicitly positioned as a voluntary self-assessment tool designed to help developers, deployers, and procurers evaluate whether an AI system and its surrounding processes align with seven Trustworthy AI requirements (human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental well-being; accountability).

First, ALTAI supports organizational sensemaking and responsibility allocation. In practice, the framework prompts teams to identify “who is responsible for what” and to document existing controls, oversight practices, and escalation mechanisms. This is consistent with the tool’s own guidance that it is best completed by a multidisciplinary team with competencies spanning the seven requirements. In SHAPES-related materials, the emphasis on multidisciplinary completion appears as a practical recommendation: involving diverse expertise is treated as a prerequisite for meaningful assessment.

Second, ALTAI is particularly effective at eliciting transparency- and process-oriented governance improvements. Internal SHAPES synthesis slides explicitly note that, when combining available self-assessment results across pilot themes, the best result was obtained in “transparency”. This aligns with ALTAI’s design: it contains structured prompts that encourage traceability, explainability/communication, and documentation of system capabilities and limitations. Even when technical system internals remain complex, a governance tool can still produce tangible improvements by demanding documentation, communication plans, and accountability mapping—areas where organizations can often act immediately.

Third, ALTAI tends to surface equity- and inclusion-related gaps that are otherwise under-addressed in engineering-led assessments. In the same SHAPES synthesis, the requirement cluster “Diversity, Non-discrimination and Fairness” generated the greatest number of recommendations, indicating that ALTAI systematically identifies this domain as improvement-heavy in real deployments. This is a meaningful governance signal: ALTAI’s question sets operate as a “gap detector” for socio-technical risk areas that may not be prioritized by performance metrics alone.

Fourth, ALTAI provides a reusable governance workflow that can be generalized across domains. The SHAPES pilot experiences indicate that ALTAI offers more than a static checklist; it provides a structured process for identifying governance issues, documenting responsibilities, generating recommendations, and supporting organizational learning. This process-oriented character allows Trustworthy AI considerations to be integrated into lifecycle governance activities and multidisciplinary decision-making. While such a workflow is not sufficient for governing AGI or superintelligence, it represents a governance capability that can be adapted and extended to more advanced AI contexts. In this sense, ALTAI contributes to governance a repeatable governance routine that remains valuable even as AI systems become increasingly autonomous and complex.

From the standpoint of AGI and superintelligence governance, these strengths matter because early governance architectures must often begin with organizational capacity building: documentation, stakeholder involvement, oversight planning, and explicit accountability. ALTAI is not a superintelligence safety solution, but it is a credible institutional starting point that can raise baseline governance maturity.

3.2.2. Structural Limitations: Where a Static, Self-Assessment Tool Strains

The same real-world evidence that demonstrates ALTAI’s strengths also points to several structural limits that are likely to become sharper as systems move toward AGI-level autonomy and complexity.

Firstly, technical robustness and safety are persistently hard to substantiate through self-assessment alone. The SHAPES synthesis indicates that the worst overall result across combined pilot themes was “technical robustness and safety.” This is not surprising: robustness and safety require evidence from engineering validation, security testing, reliability monitoring, and in some cases third-party evaluation. A checklist can prompt these activities, but it cannot generate the underlying evidence. As a result, ALTAI may function more as an indicator of “known unknowns” than as a mechanism that closes robustness gaps.

Secondly, the self-assessment format introduces interpretive variability and comparability problems. ALTAI is intentionally flexible: organizations are expected to tailor answers to their context. But this flexibility also makes outputs difficult to compare across systems, organizations, and sectors, since different teams may apply different standards of evidence and different thresholds for “adequacy.” A broader review of ALTAI has similarly highlighted issues around adoption and practical utility in industry, underscoring the gap between principles and on-the-ground implementation. In AGI/superintelligence contexts, where external trust and high-stakes assurance will likely be required, purely organization-internal self-assessment becomes a weak foundation unless complemented by more formal assurance mechanisms.

Thirdly, tooling and data portability limitations constrain longitudinal and system-level analysis. Your internal material notes that, in SHAPES pilot assessment, ALTAI provides recommendations per pilot, but “data transfer to other analysis programs is difficult.” This is not a minor usability issue: governance for advanced AI systems will increasingly require continuous monitoring, aggregation across deployments, and integration into broader risk management tooling. If assessment artifacts are not portable or machine-actionable, organizations struggle to move from episodic self-assessment to “governance as a living system.”

Fourthly, ALTAI is system- and organization-centric, thus, advanced AI risks are often ecosystem- and interaction-centric. ALTAI assumes a relatively bounded system with identifiable owners, scope, and lifecycle decisions. Yet even in SHAPES-related environments, systems operate within broader digital ecosystems (data exchange, interoperability constraints, cross-border considerations). Your earlier SHAPES-related MDPI paper summarizes concerns including interoperability, data exchange procedures, and security. These are precisely the kinds of risks that grow when AI systems become more general, interact with other agents, or are embedded in large-scale socio-technical infrastructures.

Finally, ALTAI is primarily a governance “front-end”; it lacks binding enforcement or independent verification. The European Commission frames ALTAI as a practical tool to translate principles into steps for self-assessment; it is not a binding instrument. This is appropriate for early adoption and learning, but superintelligence governance discussions often emphasize mechanisms with stronger accountability properties (independent audits, continuous oversight, external reporting, and enforceable constraints). ALTAI can support preparedness, but it cannot by itself create the institutional leverage required for frontier risks.

3.2.3. Scalability to AGI and Superintelligence: What Must Change (and What Can Be Reused)

ALTAI’s suitability for AGI and superintelligence governance should be assessed not in binary terms (“fit” vs. “not fit”), but as a question of which components scale and which require conceptual extension.

Several ALTAI elements appear scalable as governance primitives:

• Lifecycle thinking and documentation discipline. ALTAI encourages organizations to address risks across the system lifecycle and to document rationales and controls. Even if AGI systems require new technical safeguards, they will still require “audit trails” of design decisions, data governance choices, and oversight mechanisms.
• Human agency and oversight as a core requirement. The Ethics Guidelines and ALTAI emphasize that systems should support human agency and that oversight mechanisms (human-in-the-loop / human-on-the-loop / human-in-command) are necessary. In AGI contexts, the details of oversight may change (e.g., layered oversight, institutional oversight), but the principle remains essential.
• Stakeholder engagement as a governance practice. Internal guidance [12] stresses stakeholder involvement and transparency of results, reinforcing ALTAI’s role as a participatory governance scaffold. For AGI and superintelligence, stakeholder sets broaden (public authorities, cross-border actors, civil society), but the need for structured engagement persists.

Other aspects do not scale well without additional mechanisms:

• Static assessments vs. dynamic systems. Advanced AI systems may change behavior through updates, retraining, and adaptation. A periodic checklist approach will lag behind system evolution. The SHAPES synthesis itself calls for combining missing evaluations and then developing risk reduction recommendations based on aggregated results—implicitly signaling a need for ongoing, integrated assessment rather than one-off completion.
• Assurance for robustness and safety. The observed weakness in “technical robustness and safety” indicates that organizations need deeper technical assurance pipelines (testing, security validation, incident response, monitoring) that extend beyond self-reported compliance with checklist items.
• Cross-system and multi-agent governance. As AI systems are deployed at scale, risk emerges from interactions across components and institutions. ALTAI’s system-centric framing needs to be complemented with ecosystem-level methods (portfolio risk governance, supply-chain assurance, interoperability governance, and third-party oversight).

Based on the empirical lessons and the structural analysis above, a plausible extension path is to treat ALTAI as a front-end governance instrument that must be integrated into a broader governance stack (“ALTAI+”), where:

• Self-assessment is coupled with evidence requirements (e.g., documented tests, monitoring evidence, incident logs) for high-risk requirements such as robustness and safety.
• Assessments become continuous and machine-actionable, addressing the portability problem noted in SHAPES materials (difficulty transferring ALTAI outputs to analysis programs).
• Independent review layers are introduced for higher capability systems, addressing the non-binding nature of ALTAI by adding auditability and external accountability mechanisms that align with the accountability requirement but exceed internal self-assessment.
• Ecosystem governance is explicitly incorporated, so that interoperability, cross-organizational data exchange, and system-of-systems effects are addressed as first-class governance objects (echoing SHAPES concerns around interoperability, data exchange, and security).

In this framing, ALTAI remains valuable—not as a final governance solution for AGI/superintelligence, but as a standardized entry point into governance practice. Its greatest contribution is establishing a shared vocabulary and set of governance dimensions that can be expanded into more rigorous assurance regimes.

4. Toward Superintelligence Governance: Extending ALTAI-Type Approaches

The preceding analysis suggests that ALTAI should not be evaluated solely in terms of whether it can “govern” AGI or superintelligence in its current form. Rather, its value lies in revealing which governance elements already exist in practical Trustworthy AI tools, and which elements require fundamental extension as AI capabilities increase.

Current discussions on superintelligence governance emphasize that future systems may exhibit levels of autonomy, generality, and impact that exceed the assumptions underlying most present-day AI governance frameworks. As noted in the broader literature, governance of superintelligence cannot rely exclusively on reactive measures or ad hoc controls; instead, it requires anticipatory, multi-layered, and coordinated approaches that combine organizational practices, technical safeguards, and institutional oversight. Within this landscape, ALTAI can be interpreted as an early-stage governance instrument that addresses some—but not all—of these requirements [13].

The transition from Trustworthy AI to superintelligence governance therefore involves a conceptual shift: from ensuring that individual systems comply with ethical principles, toward managing capability growth, system interactions, and long-term societal risk. ALTAI’s contribution to this shift lies less in its specific checklist items than in the governance logic it embodies.

4.1. Preserving Governance Primitives That Scale

Several governance “primitives” embedded in ALTAI appear robust enough to be retained as AI systems become more capable.

First, lifecycle-oriented governance is likely to remain essential. ALTAI’s emphasis on assessing risks across design, deployment, operation, and monitoring phases anticipates the need for continuous oversight. While static self-assessment is insufficient for advanced systems, the underlying principle—that governance must span the entire system lifecycle—remains valid for AGI and superintelligence contexts.

Second, human agency and oversight constitute a foundational requirement that persists even as the form of oversight evolves. The Ethics Guidelines for Trustworthy AI and ALTAI both frame oversight as a spectrum, ranging from human-in-the-loop to human-on-the-loop and human-in-command. In superintelligence governance, this concept is likely to expand beyond individual operators to include organizational, regulatory, and possibly international oversight bodies, but the core idea of meaningful human control remains central [13].

Third, accountability through documentation and transparency is a governance asset that scales conceptually, even if its implementation becomes more demanding. ALTAI’s insistence on documenting responsibilities, decision rationales, and mitigation measures provides a governance baseline that can support future auditability and external review—both of which are frequently cited as prerequisites for high-capability AI governance.

4.2. Governance Elements That Require Extension

At the same time, the analysis indicates that several aspects of ALTAI do not scale to superintelligence contexts without substantial modification. A central limitation is the static and organization-centric nature of self-assessment. Superintelligent systems are often discussed in terms of dynamic capability growth, emergent behavior, and interactions across multiple technical and institutional boundaries. Governance mechanisms for such systems must therefore be capable of continuous assessment, real-time monitoring, and adaptive intervention, rather than episodic self-reflection.

Another key gap concerns independent verification and coordination. ALTAI is intentionally non-binding and internally focused, which supports early adoption but limits its effectiveness in high-stakes scenarios. Superintelligence governance proposals frequently emphasize the need for independent evaluation, shared safety standards, and coordination among leading development efforts to mitigate competitive pressures and systemic risk. These functions cannot be fulfilled by self-assessment alone [13].

Finally, ecosystem-level governance becomes increasingly important as AI systems operate within networks of other systems, organizations, and infrastructures. ALTAI primarily addresses individual AI deployments, whereas superintelligence governance must account for system-of-systems effects, cross-border impacts, and long-term societal consequences—dimensions that exceed the scope of most current Trustworthy AI tools.

4.3. Toward an “ALTAI+” Governance Stack

Taken together, these observations point toward the need for an extended governance model that builds on ALTAI rather than replacing it. One way to conceptualize this extension is as an “ALTAI+” governance stack, in which ALTAI functions as an entry-level or baseline component within a broader governance architecture.

In such a stack:

• ALTAI-type self-assessment provides internal awareness, documentation, and initial risk identification.
• Continuous monitoring and technical assurance mechanisms address dynamic behavior, robustness, and safety beyond what checklists can capture.
• Independent oversight and audit structures introduce external accountability for high-capability systems.
• Coordination mechanisms at organizational or international levels help manage shared risks and reduce incentive structures that favor speed over safety.

This layered approach aligns with emerging views in the literature that superintelligence governance must combine technical, organizational, and institutional controls, rather than relying on any single mechanism [13,14].

4.4. Interim Implications for Governance Research and Practice

From a research perspective, the analysis underscores the importance of studying how existing Trustworthy AI tools perform under increasing capability stress, rather than treating AGI and superintelligence governance as an entirely separate domain. Empirical experiences with tools like ALTAI provide valuable insight into governance bottlenecks, organizational behavior, and practical limitations that are likely to persist—or intensify—as AI systems advance.

For practitioners and policymakers, the findings suggest that early investment in governance capacity—documentation practices, stakeholder engagement, and accountability structures—can create a foundation upon which more advanced oversight mechanisms can be built. ALTAI alone is not sufficient for superintelligence governance, but it can play a meaningful role in preparing organizations for the governance demands that lie ahead.

Table 2 summarizes this argument by mapping the seven ALTAI requirements to typical AGI and superintelligence level stressors and the corresponding governance extensions required to address them. The table illustrates how ALTAI’s core governance dimensions remain conceptually relevant, while also making it explicit where static, organization centric self-assessment must be complemented by dynamic monitoring, independent oversight, and ecosystem level coordination. In this sense, the table operationalizes the proposed ALTAI+ approach, showing how existing Trustworthy AI practices can be extended—rather than replaced—to meet the governance demands of increasingly autonomous and high impact AI systems.

5. Discussion

This study aimed to examine ALTAI not merely as an ethical checklist, but as a governance instrument whose underlying assumptions and practical performance can inform discussions on AGI and superintelligence governance. By combining conceptual analysis with evidence-informed insights from the SHAPES pilots, the paper highlights both the enduring value and the structural limits of current Trustworthy AI approaches when confronted with increasing AI capability.

5.1. Positioning ALTAI within the Trustworthy AI Literature

Within the broader Trustworthy AI literature, ALTAI represents one of the most mature attempts to operationalize ethical principles into actionable governance practices [1]. Unlike abstract ethical codes, it provides organizations with concrete prompts that encourage documentation, responsibility allocation, and stakeholder engagement. The findings of this study are consistent with earlier observations that ALTAI is particularly effective at supporting organizational reflection and cross-disciplinary dialogue, especially in areas such as transparency, accountability, and fairness [10].

At the same time, this analysis reinforces a key tension identified in prior research: the gap between ethical principles and technical assurance [3,10]. While ALTAI successfully structures ethical discussion, it remains dependent on the availability and quality of underlying technical evidence, particularly with respect to robustness and safety. This tension is not a flaw unique to ALTAI, but rather a structural feature of governance tools that rely on self-assessment and organizational reporting. As AI systems become more complex and adaptive, this gap becomes increasingly consequential.

5.2. Implications for AGI and Superintelligence Governance Debates

Discussions on AGI and superintelligence governance often span a wide spectrum, ranging from long-term scenarios involving highly capable future AI systems to immediate questions concerning the governance, oversight, and regulation of contemporary AI technologies [3,13]. The findings of this paper suggest that tools like ALTAI occupy a productive middle ground: they are grounded in current practice yet reveal governance challenges that are likely to intensify as AI capabilities grow.

One key implication is that superintelligence governance should not be approached as a complete rupture from existing AI governance, but neither can it be addressed through incremental extension alone. The empirical insights from SHAPES demonstrate that even relatively bounded AI systems already strain static, organization-centric governance models. Issues such as evolving system behavior, distributed responsibility, and ecosystem-level interactions appear early and foreshadow the governance demands of more advanced systems.

From this perspective, ALTAI functions as a diagnostic instrument. Its strengths indicate which governance primitives—lifecycle thinking, human oversight, accountability through documentation—are likely to remain essential. Its limitations, in turn, identify where new mechanisms are required, such as continuous monitoring, independent verification, and coordination across organizational and national boundaries. Rather than asking whether ALTAI ‘solves’ superintelligence governance, a more productive question is how insights from its application can inform the design of layered governance architectures.

5.3. Governance as a Socio-Technical Capacity, Not a Single Tool

A recurring theme across the analysis is that governance should be understood as a socio-technical capacity, not as a single framework or checklist. ALTAI’s effectiveness in practice depends heavily on organizational maturity, multidisciplinary collaboration, and the willingness to act on identified gaps. These conditions are not guaranteed, and they become more demanding as AI systems scale in autonomy and impact.

For AGI and superintelligence, this suggests that governance effectiveness will hinge less on the existence of any particular tool and more on the integration of multiple mechanisms: internal self-assessment, technical assurance pipelines, external audits, and institutional oversight. In this sense, ALTAI’s greatest contribution may be pedagogical and preparatory. It familiarizes organizations with governance thinking, ethical risk identification, and accountability practices that are prerequisites for more stringent oversight regimes.

5.4. Balancing Pragmatism and Long-Term Risk Awareness

The discussion also highlights the importance of balancing pragmatic governance measures with awareness of long-term risks. While some critiques argue that focusing on AGI or superintelligence can distract from immediate governance challenges, the findings here suggest the opposite: long-term perspectives can sharpen attention to present-day weaknesses. The difficulties encountered in applying ALTAI to real systems—such as handling system evolution or cross-organizational dependencies—are not speculative problems. They are observable today and are likely to grow more severe with increasing capability.

Thus, integrating long-term governance thinking into Trustworthy AI practice does not require speculative assumptions about future superintelligence. It requires systematic attention to how current governance tools perform under stress and how they can be extended to manage uncertainty, scale, and interaction effects.

5.5. Implications

The analysis presented in this paper has implications that extend beyond the specific case of ALTAI and the SHAPES pilots. By examining a widely adopted Trustworthy AI tool under increasing capability stress, the study contributes to broader discussions on how AI governance should evolve as systems approach AGI-level generality and, potentially, superintelligence. These implications are relevant for researchers, practitioners, and policymakers alike.

For the research community, the findings underscore the importance of studying governance tools as socio-technical artifacts, rather than evaluating them solely at the level of principles or formal design. ALTAI illustrates how governance frameworks embed assumptions about organizational capacity, system boundaries, and the stability of AI behavior—assumptions that deserve explicit scrutiny as AI capabilities expand.

For practitioners—developers, deployers, and organizations responsible for AI systems—the results suggest that governance should be approached as a capability that must mature alongside technical capability. ALTAI demonstrates that even relatively simple governance instruments can deliver value by structuring ethical reflection, clarifying responsibilities, and fostering interdisciplinary dialogue. However, the limitations identified in the SHAPES pilots also indicate that such tools cannot substitute for deeper technical assurance or ongoing oversight.

Organizations planning to deploy increasingly autonomous or adaptive AI systems should therefore view ALTAI-type self-assessment as a baseline, not an endpoint. Practical governance strategies should combine internal self-assessment with continuous monitoring, technical validation, and escalation mechanisms that can respond to unexpected system behavior. Importantly, these practices should be institutionalized rather than treated as one-off exercises tied to specific projects.

The findings also point to the importance of organizational learning and culture. ALTAI is most effective when used by multidisciplinary teams that are willing to critically reflect on identified gaps and act upon recommendations. For advanced AI systems, cultivating such governance competence early can reduce future friction when more stringent oversight requirements emerge.

From a policy perspective, the analysis cautions against viewing Trustworthy AI tools as either sufficient solutions or irrelevant stopgaps. Instead, they should be recognized as foundational components of a layered governance ecosystem. ALTAI’s voluntary and non-binding nature supports experimentation and early adoption, but it also highlights the limits of self-regulation in high-stakes contexts.

Policymakers and standard-setting bodies can draw two key lessons. First, governance frameworks for advanced AI should be graduated and capability-sensitive, with oversight requirements that intensify as system autonomy, scale, and impact increase. Self-assessment tools may be appropriate at lower capability levels, while higher-risk systems require independent evaluation, reporting obligations, and coordinated oversight mechanisms.

Second, the analysis supports the argument that governance infrastructure must be developed proactively, rather than in response to crises. Experiences from ALTAI deployment show that governance challenges arise well before systems reach AGI-level capability. Addressing these challenges early can inform the design of institutions and processes that are better equipped to manage long-term risks associated with superintelligence.

5.6. Limitations and Future Research

Several limitations should be acknowledged. First, the study uses an analytical and conceptual review approach and does not claim comprehensive coverage of all governance frameworks or all superintelligence governance proposals. Second, the SHAPES-related insights are used as contextual evidence rather than as new empirical results, meaning that conclusions are interpretive and intended to inform governance design rather than to establish predictive claims. Third, because ALTAI is explicitly a voluntary self-assessment tool, some limitations identified here reflect the inherent constraints of self-assessment as a governance method rather than deficiencies in ALTAI specifically.

Future work can pursue several research directions to strengthen the pathway from Trustworthy AI governance to governance appropriate for AGI and superintelligence:

• Longitudinal governance studies and “governance drift” measurement. Future research should examine how governance assessments evolve over time as systems are updated, retrained, and repurposed, and how changes in organizational context affect Trustworthy AI practices. The SHAPES synthesis points toward the need to combine missing evaluations and develop risk-reduction recommendations based on aggregated results, suggesting that longitudinal integration is a practical requirement, not merely a theoretical preference.
• Machine-actionable assessment artifacts and interoperability for governance analytics. The practical difficulty of exporting ALTAI data for analysis indicates a need for standardized representations (e.g., structured exports) that enable aggregation across deployments, continuous monitoring, and integration with other risk management tools. Research here could focus on governance data models and minimum metadata standards that preserve ALTAI’s flexibility while improving portability.
• Evidence-based assurance layers for robustness and safety. The persistent weakness in “technical robustness and safety” in SHAPES-related results motivates research on how self-assessment prompts can be coupled to concrete evidence requirements (e.g., test artifacts, monitoring logs, incident response exercises) that support robust assurance.
• Ecosystem-level governance models for system-of-systems effects. Future work should address governance beyond single deployments, including cross-organizational data exchange, dependency chains, and multi-agent interaction effects. This direction aligns with the observation that governance challenges in practice often arise from integration and interoperability constraints rather than isolated model behavior [14].
• Layered oversight and coordination mechanisms for high-capability systems. Superintelligence governance discussions commonly emphasize coordination among leading efforts and independent oversight for systems above capability or resource thresholds. Future research can explore how ALTAI-like governance primitives (documentation, accountability mapping, stakeholder engagement) can be embedded into multi-level oversight structures, including organizational committees, third-party auditors, and potentially international coordination mechanisms (as explored in some governance framework proposals).
• Governance capacity building and education as a precursor to advanced oversight. Internal materials also frame ALTAI as a structured process that emphasizes stakeholder engagement and continuous monitoring, suggesting its usefulness as a governance “training” mechanism within organizations. Future research could develop and evaluate governance training interventions and maturity models that prepare organizations for more stringent oversight regimes associated with advanced AI.

6. Conclusions

This paper examined ALTAI as a governance instrument and considered what its practical application can teach us about governance needs as AI systems evolve toward AGI and, potentially, superintelligence. ALTAI was designed as a voluntary, non-binding self-assessment tool that operationalizes the seven Trustworthy AI requirements (e.g., human oversight, robustness and safety, privacy and data governance, transparency, fairness, societal well-being, and accountability) into actionable prompts for organizations. Within this framing, the core contribution of the paper is not to claim that ALTAI ‘governs’ advanced intelligence, but to show how an established Trustworthy AI tool behaves when confronted with capability- and complexity-related stressors that are likely to intensify on the path toward AGI and superintelligence.

First, ALTAI is most valuable as an early-stage governance prototype that builds organizational governance capacity. Both the underlying EU framing and practical experience emphasize that ALTAI is intended to support structured reflection, documentation, and stakeholder-informed risk identification rather than formal certification. In internal SHAPES synthesis materials, ALTAI use is associated with structured self-evaluations and the consolidation of pilot-level results into an overview that supports subsequent risk-reduction recommendations. This supports the interpretation of ALTAI as a “governance scaffold” that helps organizations clarify responsibilities and make governance discussions concrete, especially in multidisciplinary settings.

Second, real-world use highlights that ALTAI tends to produce stronger outcomes in process-oriented domains than in evidence-heavy technical assurance domains. In the SHAPES synthesis, the combined self-assessment results show the strongest performance in “transparency,” while “technical robustness and safety” emerges as the weakest area. This pattern is consistent with the basic logic of self-assessment tools: they can effectively prompt documentation, communication, and governance improvements, but they cannot substitute for technical testing, monitoring, and independent assurance required to substantiate robustness and safety claims.

Third, ALTAI systematically surfaces socio-ethical gap areas that can otherwise remain under-addressed. The SHAPES synthesis indicates that the requirement cluster “Diversity, Non-discrimination and Fairness” generated the largest number of recommendations. This is a governance-relevant outcome: it suggests ALTAI can act as a structured “gap detector” for fairness and inclusion issues across deployments, even when technical evaluation is the primary organizational focus.

Fourth, the limits observed in pilot practice anticipate governance requirements that are frequently discussed in superintelligence governance. Internal SHAPES-related notes identify practical constraints such as difficulties in transferring ALTAI data into other analysis programs, which undermines longitudinal monitoring and aggregated governance analytics. Such limitations are important because many superintelligence governance discussions emphasize coordination, independent evaluation, and more formal oversight structures for high-capability systems—requirements that go beyond internal, static self-assessment. The implication is that ALTAI remains useful as a baseline layer, but additional mechanisms are required to support continuous assurance, ecosystem-level governance, and credible external accountability.

In conclusion, ALTAI is best understood as a practical Trustworthy AI governance prototype that can raise governance maturity and systematically surface ethical and socio-technical gaps, but it is not sufficient on its own for the governance demands associated with AGI and superintelligence. The pathway forward is not to discard such tools, but to extend them into layered governance stacks that incorporate continuous assurance, ecosystem-level risk management, and independent oversight—thereby connecting present-day governance practice to the long-term challenges of governing advanced intelligence systems.