Assurance of Safety-Critical AI-Enabled Capabilities: Integrating Human, Software, and Machine Learning Assurance Paradigms

Artificial Intelligence (AI)-enabled systems are now increasingly deployed across safety-critical, mission-critical, and socio-technical domains. Existing engineering disciplines offer mature approaches to human, organisational safety and software assurance; however, AI-enabled systems can exhibit characteristics such as probabilistic behaviour, data dependency, limited explainability, and adaptation that challenge traditional assurance methods. Our research examines assurance principles across three domains that arguably dominated assurance reforms in capability development during different periods: 1) human-human organisational assurance, circa 1980 to 2000, 2) software assurance, circa 2000 to 2020, and 3) assurance of AI-enabled systems, emerging since 2020. We synthesise established literature and frameworks from each paradigm, including the NIST AI Risk Management Framework, while recognising that AI assurance remains an evolving discipline. We contribute a unified set of 20 assurance precepts for safety-critical AI-enabled capabilities. These precepts are mapped across four quadrants of assurance activities, responsibilities and critical questions, and organized within a novel Dual Assurance Spiral for AI-Enabled Capabilities (DAS4AIC). Through comparative analysis, we demonstrate how classical safety principles extend from human-dominant assurance through software-dominant assurance to the assurance of AI-enabled systems. Importantly, some human-dominant assurance precepts map more directly to AI-enabled system than through software assurance. This finding may help explain operational concerns about the erosion of accountabilities, ethical responsibility and effective human oversight when AI is introduced into safety-critical capabilities. The proposed mapping was evaluated through a face-validity workshop involving an experienced and diverse group of assurance practitioners. The workshop identified critical assurance gaps, particularly in data governance, explainability, and human-autonomy teaming. We discuss the implications for organizational governance and argue that effective governance provides the foundation for auditing, training, and validating and monitoring AI-enabled systems in safety critical operational environments. To our knowledge, this is the first study to systematically align merging AI-assurance principles with the earlier and still overlapping, human–dominant and software-dominant assurance paradigms.