Submitted:
01 June 2026
Posted:
02 June 2026
You are already at the latest version
Abstract

Keywords:
1. Introduction
- SQ1. What does each of the four leading reference frameworks specify at each phase of the AI system lifecycle, and where do they converge, diverge, or leave gaps? (Section 6.)
- SQ3. How does the resulting model align with existing UK institutional anchors, and what would be required to test that alignment in practice? (Section 7.)
2. Related Work
2.1. Responsible AI Principles and Their Operationalisation
2.2. AI Auditing and Assurance
2.3. Lifecycle, Post-Market Monitoring and Continuous Accountability
2.4. MLOps, Model Monitoring, Drift and Incident Reporting
2.5. Public Sector AI: Adoption, Accountability and Administrative Law
3. Conceptual Framework
3.1. Lifecycle Thinking for AI Assurance
- Pre-Deployment. The period before a system enters live service. Assurance activity here is anticipatory: what might go wrong, and what is being done to prevent it.
- Model Activation. The transition into live service. Assurance activity here is baseline-setting: what does 'normal' look like for this system in production conditions, such that future divergence can be detected. Without activation, monitoring becomes observation without calibration.
- Operational Response. The period in live service. Assurance activity here is detective and reactive: noticing divergence from the activation baseline, escalating when required, and intervening.
- Closed-Loop Learning. The mechanism by which operational findings feed back into design, procurement, and governance. Assurance activity here is institutional: making sure lessons are captured, shared, and acted on across departments.
3.2. Theoretical Anchor: Value Sensitive Design
3.3. An Illustrative Example: AI-Assisted Document Triage
3.4. What the Model Is Not
4. Research Approach and Methodology
4.1. Design Science Research
4.2. Stage One: Comparative Document Analysis
Analytical Safeguards
4.3. Stage Two: Doctoral Research Development
4.4. Stage Three: Expert-Informed Artefact Refinement
4.5. Methodological Limitations
5. The Four-Pillar Reference Model
5.1. Four Areas for Augmentation
5.2. Pillar 1: Pre-Deployment
5.3. Pillar 2: Model Activation
5.4. Pillar 3: Operational Response
5.5. Pillar 4: Closed-Loop Learning
5.6. The Pillars as a Structure, Summarised
6. Mapping the Four Pillars Against Four Reference Frameworks
6.1. Why These Four Frameworks
- The NIST AI Risk Management Framework (NIST, 2023) is the most widely cited cross-sectoral framework for trustworthy AI in technical and assurance discourse. It defines seven characteristics of trustworthy AI (valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; fair with harmful biases managed) and four functions (Govern, Map, Measure, Manage) that organise the actions required to address those characteristics. UK departmental practice does not formally adopt NIST AI RMF, but several of its constructs appear in departmental governance materials.
- The OECD AI Principles (OECD, 2024) are the first intergovernmental standard for trustworthy AI and are now adhered to by 47 jurisdictions, including the European Union. They set out five values-based principles: inclusive growth, sustainable development and well-being; respect for the rule of law, human rights and democratic values; transparency and explainability; robustness, security and safety; and accountability. The 2024 update relocated the provisions on traceability and systematic, ongoing-lifecycle risk management to the accountability principle, which strengthens the principles' lifecycle relevance for the present argument.
- The EU AI Act (European Union, 2024) is, of the four, the only one with regulatory force. Although the United Kingdom is not bound by the Act, UK suppliers placing AI systems on the EU market are, and the Act's structure (notably the Chapter III obligations on high-risk systems and the Chapter IX provisions on post-market monitoring and market surveillance) is becoming an important reference for what comprehensive lifecycle assurance looks like in the regulated case. Article 72 is among the more explicit and prescriptive provisions across the four frameworks for continuous post-deployment activity (Schuett, 2023; Mökander et al., 2023), although a definitive comparative ranking depends on how 'requirement' is operationalised across frameworks of different legal force.
- The UK AI Playbook for Government (Government Digital Service, 2025) is the central operational guidance for AI use within UK central government. It sets out ten principles covering knowledge of AI capabilities and limitations, lawful and ethical use, secure deployment, meaningful human control, lifecycle management, fitness of tool to task, openness, commercial engagement, skills, and assurance, and a six-stage implementation pathway from problem definition through to decommissioning. Of the four frameworks, the Playbook is the one departments are most directly accountable to in practice.
6.2. The Mapping
| Pillar | NIST AI RMF | OECD AI Principles | EU AI Act | UK AI Playbook |
|---|---|---|---|---|
| 1. Pre-Deployment | Dense | Partial | Dense | Dense |
| 2. Model Activation | Implicit | Implicit | Partial | Implicit |
| 3. Operational Response | Partial | Partial | Dense | Dense |
| 4. Closed-Loop Learning | Implicit | Implicit | Partial | Partial |
Coding Rationale per Cell
6.3. Reading the Mapping
6.4. What the Mapping Is for
7. Institutional Fit Analysis
7.1. DSIT Responsible AI Unit
7.2. Departmental Service Assurance Functions
7.3. Crown Commercial Service and Procurement
7.4. What Would Be Required to Test Institutional Fit
8. Discussion
8.1. Contribution to Scholarship
8.2. Platform Independence
8.3. Conditions for Empirical Evaluation
8.4. Future Empirical Evaluation Design
- Time to detection. The interval between onset of an AI-specific issue (drift breaching a threshold, an output anomaly cluster, a fairness proxy excursion) and the first recorded internal flag. Explicit baseline-setting at Pillar 2 and AI-specific monitoring at Pillar 3 should shorten this interval relative to general service management alone.
- Time to escalation. The interval between detection and SRO-level awareness or formal incident declaration. The stakeholder specification at Pillars 2 and 3 should shorten this interval through explicit escalation paths.
- Number of repeated incidents. The frequency, within and across departments, with which an incident class observed in one system recurs in a subsequent system. Pillar 4 should reduce this frequency over time.
- Quality of assurance artefacts. Completeness, traceability, and downstream usability of documents produced at each pillar, assessed against a structured rubric agreed in advance with participating departments, drawing on existing rubrics for ML production readiness (Breck et al., 2017).
- Procurement changes triggered by operational evidence. The number and significance of changes to procurement specifications, contract terms, or supplier requirements traceable to evidence collected under Pillar 3 and processed under Pillar 4. This metric most directly tests whether the closed loop is closing.
8.5. Limitations
8.6. Positioning, Supervisory Review and Conflict of Interest
9. Conclusion
Acknowledgments
Conflicts of Interest
Appendix A. Coding Audit Trail
| Pillar | Framework | Source reference | Summary of relevant text | Code | Rationale |
|---|---|---|---|---|---|
| 1. Pre-Deployment | NIST AI RMF | Map function (1.1 to 5.1); Govern function (1.1 to 6.2); Measure function (1.1 to 4.3) | Map establishes context, identifies and analyses risks and impacts before deployment. Govern provides cross-cutting accountability policy. Measure 1 to 2 covers initial metric specification. | Dense | Activity is explicitly named, operational form is articulated through sub-categories and outcomes, and responsibilities are specified across the four functions. |
| 1. Pre-Deployment | OECD AI Principles | Principles 1.4, 1.5; Accountability principle (post-2024 amendment) | Calls for risk assessment, documentation and impact analysis before deployment. | Partial | Activity is named but operational form is not specified; the principles are values-based rather than procedural. |
| 1. Pre-Deployment | EU AI Act | Chapter III, Section 2, Articles 8 to 15; Article 27 (FRIA); Article 43 (conformity assessment) | Comprehensive pre-market obligations on high-risk systems: risk management, data governance, technical documentation, transparency, human oversight, accuracy and robustness. Fundamental Rights Impact Assessment for public-sector high-risk deployments. | Dense | Activity is explicitly required, operational form is prescribed at article level, and the conformity assessment process specifies responsibilities. |
| 1. Pre-Deployment | UK AI Playbook | Principles 1, 2, 3, 5, 6; Stages 0 to 3 of the implementation pathway; ATRS submission | Knowing AI's limitations, lawful use, secure use, lifecycle management, fitness of tool. Pathway from problem definition through development and testing. | Dense | Activity is named, operational form is articulated through the staged pathway, and ATRS provides an evidence anchor. |
| 2. Model Activation | NIST AI RMF | Measure function (1.1 to 4.3) | Measure applies at activation but the framework does not separate baseline-setting from continuous measurement. | Implicit | Coverage can be inferred from the Measure function's application across the lifecycle, but activation is not addressed as a distinct phase with operational form. |
| 2. Model Activation | OECD AI Principles | Principle 1.5 (lifecycle phases, on an ongoing basis) | Risk management at each phase of the AI system lifecycle on an ongoing basis. No distinct activation specification. | Implicit | Activation is implicit within 'each phase of the lifecycle' but not addressed as a distinct activity. |
| 2. Model Activation | EU AI Act | Article 72(3) (post-market monitoring plan as part of Annex IV technical documentation); Article 19 (logs); Article 26 (deployer obligations) | Requires the post-market monitoring plan to be part of technical documentation, which arguably requires baseline specification at activation. Supporting structure via logs and deployer obligations. | Partial | Activity is named through the post-market monitoring plan requirement but operational form for the activation phase specifically is not prescribed; the plan is required but its activation content is not. |
| 2. Model Activation | UK AI Playbook | Stage 4 (deployment with monitoring and incident plans); Principle 4 (meaningful human control) | Stage 4 covers deployment but does not separate baseline-setting from continuous monitoring. | Implicit | Coverage can be inferred from stage 4 but baseline-setting is not addressed as a distinct activity with operational form. |
| 3. Operational Response | NIST AI RMF | Manage function (1.1 to 4.4); Measure function applied continuously; Map function re-applied as context evolves | Treat risks, respond to incidents, allocate resources. Continuous measurement. Re-mapping as context, capabilities, risks and impacts evolve. | Partial | Activity is named across functions and operational form is partially articulated, but prescriptiveness is lower than EU Act Article 72. |
| 3. Operational Response | OECD AI Principles | Principle 1.5 (systematic risk management approach on an ongoing basis) | Systematic risk management approach to each phase of the AI system lifecycle on an ongoing basis. | Partial | Activity is named but operational form is not specified; values-based rather than procedural. |
| 3. Operational Response | EU AI Act | Article 72 (post-market monitoring by providers); Article 73 (reporting of serious incidents); Chapter IX, Section 4 (market surveillance and corrective action) | Among the most prescriptive provisions across the framework set for continuous post-deployment activity. Serious incident reporting and market surveillance. | Dense | Activity is explicitly required, operational form is prescribed at article level, responsibilities are specified for providers and surveillance authorities. |
| 3. Operational Response | UK AI Playbook | Continuous monitoring guidance (dashboards, drift detection, logging, periodic re-validation, triggered re-assessment); Stage 4; Principles 4 and 5 | Continuous monitoring with explicit guidance on dashboards, drift detection, logging, re-validation. | Dense | Activity is explicitly named and operational form is articulated through stage 4 guidance. |
| 4. Closed-Loop Learning | NIST AI RMF | Manage function (incident response, organisational learning) | Gestures at incident response and organisational learning. | Implicit | Coverage can be inferred from Manage function but cross-organisational coordination is not specified, and a learning loop architecture is not articulated. |
| 4. Closed-Loop Learning | OECD AI Principles | Principle 1.5 (accountability, ongoing) | 'Ongoing' risk management implies feedback. No cross-organisational learning mechanism specified. | Implicit | Feedback is implied through the ongoing requirement but the mechanism by which lessons travel is not specified. |
| 4. Closed-Loop Learning | EU AI Act | Article 73 (serious incident reporting); Article 72(2) (evaluate continuous compliance with Chapter III, Section 2); Chapter VII (European AI Office) | Serious incident reporting to authorities; continuous compliance evaluation; Office as potential cross-deployer channel. | Partial | Activity is named through incident reporting and compliance evaluation, but a structured cross-organisational learning mechanism is not established; the AI Office's role in this is not yet operationalised. |
| 4. Closed-Loop Learning | UK AI Playbook | Principle 7 (openness and collaboration); Stage 5 (decommissioning and knowledge capture) | Central collation of lessons learned is referenced; mechanism is not mandated. | Partial | Activity is named through principle 7 and stage 5 but the mechanism is not mandated and operational form is not specified. |
References
- Ada Lovelace Institute. Regulating AI in the UK; Ada Lovelace Institute: London, 2023; Available online: https://www.adalovelaceinstitute.org/report/regulating-ai-in-the-uk/ (accessed on 5 May 2026).
- Ada Lovelace Institute and DataKind UK. Examining the Black Box: Tools for assessing algorithmic systems; Ada Lovelace Institute: London, 2020; Available online: https://www.adalovelaceinstitute.org/report/examining-the-black-box-tools-for-assessing-algorithmic-systems/ (accessed on 5 May 2026).
- Barzelay, M. Learning from second-hand experience: methodology for extrapolation-oriented case research. Governance 2007, 20(3), 521 to 543. [Google Scholar] [CrossRef]
- Black, J.; Murray, A.D. Regulating AI and machine learning: setting the regulatory agenda. Eur. J. Law. Technol. 2019, 10(3). [Google Scholar]
- Bowen, G.A. Document analysis as a qualitative research method. Qual. Res. J. 2009, 9(2), 27 to 40. [Google Scholar] [CrossRef]
- Breck, E.; Cai, S.; Nielsen, E.; Salib, M.; Sculley, D. 'The ML test score: a rubric for ML production readiness and technical debt reduction'. In Proceedings of the 2017 IEEE International Conference on Big Data; IEEE: Piscataway, NJ, 2017; p. 1123 to 1132. [Google Scholar] [CrossRef]
- Brown, S.; Davidovic, J.; Hasan, A. The algorithm audit: scoring the algorithms that score us. Big Data Soc. 2021, 8(1). [Google Scholar] [CrossRef]
- Brundage, M.; et al. Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. arXiv 2020, arXiv:2004.07213. [Google Scholar] [CrossRef]
- Bullock, J.B. Artificial intelligence, discretion, and bureaucracy. Am. Rev. Public Adm. 2019, 49(7), 751 to 761. [Google Scholar] [CrossRef]
- Cardno, C. 'Policy document analysis: a practical educational leadership tool and a qualitative research method'. Educ. Adm. Theory Pract. 2018, 24(4), 623 to 640. [Google Scholar] [CrossRef]
- Cobbe, J. Administrative law and the machines of government: judicial review of automated public-sector decision-making. Leg. Stud. 2019, 39(4), 636 to 655. [Google Scholar] [CrossRef]
- Cobbe, J.; Singh, J. 'Artificial intelligence as a service: legal responsibilities, liabilities, and policy challenges'. Comput. Law. Secur. Rev. 2021, 42. [Google Scholar] [CrossRef]
- Cobbe, J.; Veale, M.; Singh, J. 'Understanding accountability in algorithmic supply chains'. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23); ACM: New York, 2023; p. 1186 to 1197. [Google Scholar] [CrossRef]
- Costanza-Chock, S.; Raji, I.D.; Buolamwini, J. 'Who audits the auditors? Recommendations from a field scan of the algorithmic auditing ecosystem'. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22); ACM: New York, 2022; p. 1571 to 1583. [Google Scholar] [CrossRef]
- Cross, N. 'Designerly ways of knowing: design discipline versus design science'. Des. Issues 2001, 17(3), 49 to 55. [Google Scholar] [CrossRef]
- Department for Science; Innovation and Technology. Introduction to AI Assurance. DSIT: London, 2024. Available online: https://www.gov.uk/government/publications/introduction-to-ai-assurance (accessed on 5 May 2026).
- Engin, Z.; Treleaven, P. 'Algorithmic government: automating public services and supporting civil servants in using data science technologies'. Comput. J. 2019, 62(3), 448 to 460. [Google Scholar] [CrossRef]
- European Union (2024) Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Off. J. Eur. Union 2024, L series.
- Fjeld, J.; Achten, N.; Hilligoss, H.; Nagy, A.; Srikumar, M. Principled Artificial Intelligence: Mapping Consensus in Ethical and Rights-Based Approaches to Principles for AI. In Berkman Klein Center Research Publication; 2020; Volume No. 2020-1. [Google Scholar] [CrossRef]
- Floridi, L.; Cowls, J.; Beltrametti, M.; Chatila, R.; Chazerand, P.; Dignum, V.; Luetge, C.; Madelin, R.; Pagallo, U.; Rossi, F.; Schafer, B.; Valcke, P.; Vayena, E. AI4People: an ethical framework for a good AI society: opportunities, risks, principles, and recommendations. Minds Mach. 2018, 28(4), 689 to 707. [Google Scholar] [CrossRef]
- Friedman, B. 'Value-sensitive design'. interactions 1996, 3(6), 16 to 23. [Google Scholar] [CrossRef]
- Friedman, B.; Hendry, D.G. Value Sensitive Design: Shaping Technology with Moral Imagination; MIT Press: Cambridge, MA, 2019. [Google Scholar]
- Friedman, B.; Kahn, P.H.; Borning, A. 'Value sensitive design and information systems'. In Human-Computer Interaction and Management Information Systems: Foundations; Zhang, P., Galletta, D., Eds.; M.E. Sharpe: Armonk, NY, 2006; p. 348 to 372. [Google Scholar]
- Gama, J.; Zliobaite, I.; Bifet, A.; Pechenizkiy, M.; Bouchachia, A. A survey on concept drift adaptation'. ACM Comput. Surv. 2014, 46(4)(Article 44), 1 to 37. [Google Scholar] [CrossRef] [PubMed]
- Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J.W.; Wallach, H.; Daume, H., III; Crawford, K. 'Datasheets for datasets'. Commun. ACM 2021, 64(12), 86 to 92. [Google Scholar] [CrossRef]
- Government Digital Service (2025) Artificial Intelligence Playbook for the UK Government. Department for Science, Innovation and Technology: London. Available online: https://www.gov.uk/government/publications/ai-playbook-for-the-uk-government (accessed on 5 May 2026).
- Greene, D.; Hoffmann, A.L.; Stark, L. 'Better, nicer, clearer, fairer: a critical assessment of the movement for ethical artificial intelligence and machine learning'. In Proceedings of the 52nd Hawaii International Conference on System Sciences, 2019. [Google Scholar] [CrossRef]
- Hevner, A.R.; March, S.T.; Park, J.; Ram, S. 'Design science in information systems research'. MIS Q. 2004, 28(1), 75 to 105. [Google Scholar] [CrossRef]
- Holstein, K.; Wortman Vaughan, J.; Daume, H., III; Dudik, M.; Wallach, H. 'Improving fairness in machine learning systems: what do industry practitioners need?'. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems; ACM: New York, 2019; p. 1 to 16. [Google Scholar] [CrossRef]
- International Organization for Standardization. ISO/IEC 42001:2023; Information technology - Artificial intelligence - Management system. ISO: Geneva, 2023a.
- International Organization for Standardization. ISO/IEC 23894:2023; Information technology - Artificial intelligence - Guidance on risk management. ISO: Geneva, 2023b.
- Jobin, A.; Ienca, M.; Vayena, E. 'The global landscape of AI ethics guidelines'. Nat. Mach. Intell. 2019, 1(9), 389 to 399. [Google Scholar] [CrossRef]
- Klaise, J.; Van Looveren, A.; Cox, C.; Vacanti, G.; Coca, A. 'Monitoring and explainability of models in production'. arXiv 2020, arXiv:2007.06299. [Google Scholar] [CrossRef]
- Kop, M. 'EU Artificial Intelligence Act: the European approach to AI'. In Stanford-Vienna Transatlantic Technology Law Forum, Transatlantic Antitrust and IPR Developments; Stanford University, 2021; Volume Issue No. 2/2021. [Google Scholar]
- Kreuzberger, D.; Kuhl, N.; Hirschl, S. Machine Learning Operations (MLOps): overview, definition, and architecture'. IEEE Access 2023, 11, 31866 to 31879. [Google Scholar] [CrossRef]
- Lewis, J.M. The limits of policy labs: characteristics, opportunities and constraints. Policy Des. Pract. 2021, 4(2), 242 to 256. [Google Scholar] [CrossRef]
- Madaio, M.A.; Stark, L.; Wortman Vaughan, J.; Wallach, H. 'Co-designing checklists to understand organizational challenges and opportunities around fairness in AI'. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems; ACM: New York, 2020; p. 1 to 14. [Google Scholar] [CrossRef]
- Madan, R.; Ashok, M. AI adoption and diffusion in public administration: a systematic literature review and future research agenda'. Gov. Inf. Q. 2023, 40(1), 101774. [Google Scholar] [CrossRef]
- March, S.T.; Smith, G.F. 'Design and natural science research on information technology'. Decis. Support Syst. 1995, 15(4), 251 to 266. [Google Scholar] [CrossRef]
- Margetts, H.; Dorobantu, C. 'Rethink government with AI'. Nature 2019, 568(7751), 163 to 165. [Google Scholar] [CrossRef]
- McGregor, S. 'Preventing repeated real world AI failures by cataloging incidents: the AI Incident Database'. Proc. AAAI Conf. Artif. Intell. 2021, 35(17), 15458 to 15463. [Google Scholar] [CrossRef]
- Misuraca, G.; van Noordt, C. AI Watch - Artificial Intelligence in public services: overview of the use and impact of AI in public services in the EU. In EUR 30255 EN; Publications Office of the European Union: Luxembourg, 2020. [Google Scholar] [CrossRef]
- Mitchell, M.; Wu, S.; Zaldivar, A.; Barnes, P.; Vasserman, L.; Hutchinson, B.; Spitzer, E.; Raji, I.D.; Gebru, T. 'Model cards for model reporting'. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19); ACM: New York, 2019; p. 220 to 229. [Google Scholar] [CrossRef]
- Mittelstadt, B. 'Principles alone cannot guarantee ethical AI'. Nat. Mach. Intell. 2019, 1(11), 501 to 507. [Google Scholar] [CrossRef]
- Mokander, J.; Morley, J.; Taddeo, M.; Floridi, L. Ethics-based auditing of automated decision-making systems: nature, scope, and limitations'. Sci. Eng. Ethics 2021, 27(4), 44. [Google Scholar] [CrossRef]
- Mokander, J.; Sheth, M.; Watson, D.S.; Floridi, L. 'The switch, the ladder, and the matrix: models for classifying AI systems'. In Minds and Machines; 2023; Volume 33, p. 221 to 248. [Google Scholar] [CrossRef]
- Morley, J.; Floridi, L.; Kinsey, L.; Elhalal, A. From what to how: an initial review of publicly available AI ethics tools, methods and research to translate principles into practices. Sci. Eng. Ethics 2020, 26(4), 2141 to 2168. [Google Scholar] [CrossRef]
- NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1; National Institute of Standards and Technology: Gaithersburg, MD, 2023. [Google Scholar] [CrossRef]
- OECD. OECD/LEGAL/0449; Recommendation of the Council on Artificial Intelligence. Originally adopted 22 May 2019, amended 3 May 2024. OECD: Paris, 22 May 2019. Available online: https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449 (accessed on 5 May 2026).
- Paleyes, A.; Urma, R.-G.; Lawrence, N.D. Challenges in deploying machine learning: a survey of case studies'. ACM Comput. Surv. 2022, 55(6)(Article 114), 1 to 29. [Google Scholar] [CrossRef]
- Peffers, K.; Tuunanen, T.; Rothenberger, M.A.; Chatterjee, S. 'A design science research methodology for information systems research'. J. Manag. Inf. Syst. 2007, 24(3), 45 to 77. [Google Scholar] [CrossRef]
- Raji, I.D.; Buolamwini, J. 'Actionable auditing: investigating the impact of publicly naming biased performance results of commercial AI products'. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society; ACM: New York, 2019; p. 429 to 435. [Google Scholar] [CrossRef]
- Raji, I.D.; Smart, A.; White, R.N.; Mitchell, M.; Gebru, T.; Hutchinson, B.; Smith-Loud, J.; Theron, D.; Barnes, P. 'Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing'. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* '20); ACM: New York, 2020; p. 33 to 44. [Google Scholar] [CrossRef]
- Rakova, B.; Yang, J.; Cramer, H.; Chowdhury, R. 'Where responsible AI meets reality: practitioner perspectives on enablers for shifting organizational practices'. Proc. ACM Hum.-Comput. Interact. 2021, 5(CSCW1), 1 to 23. [Google Scholar] [CrossRef] [PubMed]
- Sambasivan, N.; Kapania, S.; Highfill, H.; Akrong, D.; Paritosh, P.; Aroyo, L.M. 'Everyone wants to do the model work, not the data work: data cascades in high-stakes AI'. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems; ACM: New York, 2021; p. paper 39. [Google Scholar] [CrossRef]
- Sandvig, C.; Hamilton, K.; Karahalios, K.; Langbort, C. 'Auditing algorithms: research methods for detecting discrimination on internet platforms'. paper presented at Data and Discrimination: Converting Critical Concerns into Productive Inquiry, 64th Annual Meeting of the International Communication Association, Seattle, WA, 2014. [Google Scholar]
- Schuett, J. Risk management in the Artificial Intelligence Act'. Eur. J. Risk Regul. 2023, 15(2), 367 to 385. [Google Scholar] [CrossRef]
- Schuett, J. Three lines of defence against risks from AI. In AI and Society; 2024. [Google Scholar] [CrossRef]
- Sculley, D.; Holt, G.; Golovin, D.; Davydov, E.; Phillips, T.; Ebner, D.; Chaudhary, V.; Young, M.; Crespo, J.-F.; Dennison, D. 'Hidden technical debt in machine learning systems'. In Advances in Neural Information Processing Systems 28 (NIPS 2015); Curran Associates: Red Hook, NY, 2015; p. 2503 to 2511. [Google Scholar]
- Selbst, A.D.; Boyd, D.; Friedler, S.A.; Venkatasubramanian, S.; Vertesi, J. 'Fairness and abstraction in sociotechnical systems'. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19); ACM: New York, 2019; p. 59 to 68. [Google Scholar] [CrossRef]
- Simon, H.A. The Sciences of the Artificial, 3rd edn; MIT Press: Cambridge, MA, 1996. [Google Scholar]
- Sonnenberg, C.; vom Brocke, J. 'Evaluations in the science of the artificial: reconsidering the build-evaluate pattern in design science research'. In Design Science Research in Information Systems. Advances in Theory and Practice. DESRIST 2012. Lecture Notes in Computer Science; Peffers, K., Rothenberger, M., Kuechler, B., Eds.; Springer: Berlin, 2012; vol. 7286, p. 381 to 397. [Google Scholar] [CrossRef]
- Umbrello, S.; van de Poel, I. 'Mapping value sensitive design onto AI for social good principles'. AI Ethics 2021, 1(3), 283 to 296. [Google Scholar] [CrossRef]
- Veale, M.; Brass, I. 'Administration by algorithm? Public management meets public sector machine learning'. In Algorithmic Regulation; Yeung, K., Lodge, M., Eds.; Oxford University Press: Oxford, 2019; p. 121 to 149. [Google Scholar] [CrossRef]
- Vogl, T.M.; Seidelin, C.; Ganesh, B.; Bright, J. Smart technology and the emergence of algorithmic bureaucracy: artificial intelligence in UK local authorities'. Public Adm. Rev. 2020, 80(6), 946 to 961. [Google Scholar] [CrossRef]
- Whittlestone, J.; Nyrup, R.; Alexandrova, A.; Cave, S. 'The role and limits of principles in AI ethics: towards a focus on tensions'. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society; ACM: New York, 2019; p. 195 to 200. [Google Scholar] [CrossRef]
- Yeung, K. Algorithmic regulation: a critical interrogation. Regul. Gov. 2018, 12(4), 505 to 523. [Google Scholar] [CrossRef]
| Code | Definition |
|---|---|
| Dense | The framework specifies activity at this phase explicitly and prescriptively, with identifiable articles, principles, or functions; operational form is articulated; stakeholders or responsibilities are indicated. |
| Partial | The framework addresses the phase but with limited prescriptiveness; activity is named but operational form is not specified, or only a subset of the relevant activity is addressed. |
| Implicit | The framework does not address the phase as a distinct activity but coverage can be inferred from broader provisions or lifecycle framing. |
| Absent | The framework does not address the phase, even implicitly; no provision is reasonably available at the lifecycle phase concerned. |
| Pillar | Purpose | Primary Evidence | Primary Stakeholders | Value Added |
|---|---|---|---|---|
| 1. Pre-Deployment | Fitness for service | Questionnaires, impact assessments, red-team reports | Project teams, ethics, DPO, SRO | Structured anticipation; documented specification for live service |
| 2. Model Activation | Baseline for live service | Baselines, thresholds, runbooks | Project and service teams, data scientists | Calibrated reference for recognising normal behaviour in service |
| 3. Operational Response | Detection and response in service | Telemetry, alerts, incidents | Service teams, AI specialists, SRO | Continuous AI-specific visibility alongside general service management |
| 4. Closed-Loop Learning | Cross-government institutional learning | Reviews, updated registers, cross-departmental publications | Model owners, risk, procurement, DSIT RAI Unit | Conversion of operational experience into cross-government knowledge |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).