Preprint
Article

This version is not peer-reviewed.

Beyond Algorithmic Oversight: Internal Morality of Medicine and Meaningful Human Control in AI-Assisted Care

A peer-reviewed version of this preprint was published in:
Healthcare 2026, 14(12), 1638. https://doi.org/10.3390/healthcare14121638

Submitted:

08 May 2026

Posted:

09 May 2026

You are already at the latest version

Abstract
Background/Objectives: Artificial intelligence reshapes clinical practice and its effect on physician-patient relationship requires reconsideration of frameworks that have shaped modern medical ethics. When physician delegate expertise to algorithms they cannot verify, it becomes unclear who bears clinical responsibility. Methods: This article applies theoretically grounded normative approach to explore ethical conditions under which artificial intelligence can be integrated into clinical practice without compromising the moral foundations of medicine. The analysis is primarily based on Pellegrino and Thomasma’s concept of internal morality of medicine and the physician’s act of profession. It further draws on Kantian ethics of human dignity, Levinasian relational ethics, virtue ethics, and Vallor’s concept of technomoral wisdom. Results: AI systems do not satisfy the conditions under which moral responsibility can be ascribed to them. Clinical moral agency lies in the capacity to bear three distinct responsibilities – epistemic, relational, and phronetic – none of which can be fulfilled by AI. The implementation of AI in healthcare, therefore, must occur strictly under the condition of Meaningful Human Control, rather technical function of human oversight over algorithmic outputs. To ensure that MHC can function as an effective and ethically grounded safeguard, we propose five normative requirements: primacy of clinical judgement, prohibition of forced automation, traceability and explainability, transparency towards patients, and clinical authority over diagnostic tools. A dialog between the physician and the patients should remain the foundation of clinical decision-making. Proposed normative requirements aim to preserve internal morality of medicine in a form that harmoniously combines both technological progress and established medical ethics.
Keywords: 
;  ;  ;  ;  

1. Introduction

Among the many ways Artificial Intelligence (AI) is reshaping clinical practice, its effect on the doctor–patient relationship calls for a reconsideration of the frameworks that have shaped modern medical ethics. T. Parsons [1] has noticed that doctor–patient relationship is asymmetric where patient becomes dependent by assuming the role of the sick. Patients rely on someone who has better knowledge and skills to tackle health issues. As an alternative to the paternalistic approach, new models started emerging together with the theoretical framework elaborated by Szasz and Hollender [2]. These models tackle epistemic and power asymmetry with recognition of respect for the patient’s autonomy, dignity, and rights.
Subsequently, those principles of trust, autonomy, and shared decision–making were included into international guidelines and became an integral part of the professional standard. However, with a rapidly growing integration of AI into clinical practice, new forms of epistemic asymmetry emerge, for both the physician’s and the patient’s knowledge is becoming increasingly dependent on systems with a nearly limitless knowledge base [3,4]. On the positive side, AI integration may improve patient care and reduce healthcare professionals’ workload [5,6]. However, those healthcare professionals then become accountable for the results of the implementation of algorithms with limited transparency, which causes anxiety [7,8,9]. Such situation requires a deeper examination of what it means to practice medicine amidst the transformation of roles and responsibilities, particularly when (semi-)automated systems recommend high-stakes medical decisions while leaving the ultimate burden of liability to humans.
Long before the AI era, Pellegrino and Thomasma [10] defined medicine fundamentally as a moral enterprise that is aimed at the good of the ill person, and this is precisely what determines the internal morality of medicine. According to E. Pellegrino, “medicine is a moral community because its practitioners are bound together by a common goal – the good of the patient” [11]. Medical practice serves as a response to the patient’s call for help, and physicians take a moral commitment to treat the patient and reduce suffering [12]. In this context, there is a doubt if the integration of AI in healthcare systems will be able and, if yes, under what conditions will be able to sustain the moral goal of medicine. AI systems can calculate the probability of a diagnosis or the potential success of treatment options, but they cannot determine what the right clinical judgment would be. Clinical rightness requires synthesizing empirical data with the patient’s values and life-world and inherently relies on value-based judgments. Therefore, delegating such decisions to AI raises the risk of displacing physicians from the role that gives medicine its moral authority, namely, the ethical face-to-face encounter. Overreliance on assistive algorithms may limit – either voluntarily or involuntarily – the professional freedom of making deliberated decisions.
We argue that AI systems cannot participate in what Pellegrino calls ‘the act of profession’, defined as the public promise to serve the sick competently and virtuously. AI-assisted healthcare creates disparities in accountability that arise from the opacity of algorithms and their uncritical implementation in healthcare institutions. Neither the system nor the physician can fully account for decisions the system has shaped. When physicians delegate expertise to opaque algorithms they cannot independently verify, it becomes unclear who bears clinical responsibility: the physician using the system or the developers who built it. This ambiguity is the responsibility gap that creates accountability disparities [13].
Because the moral responsibility and deliberative essence of clinical practice are fundamentally non-delegable to machines, our central argument is that the integration of AI in healthcare, therefore, must occur strictly under the condition of ‘Meaningful Human Control’ (MHC) [14] as a positive condition for the physician’s exercise of phronesis as ‘technomoral wisdom’ [15]. In technological environments, physicians are expected not only to manipulate but also to assume the role of a moral interpreter in order to assist patients in seeking out decisions that best meet their values and life circumstances. This deliberative communication between the physician and the patient has long served as a normative ideal in medicine. However, for this purpose, healthcare professionals must be given full professional autonomy to practice decisional authority over algorithmic outputs in order to prevent the treatment process from turning into a purely technological one [16]. In this context, meaningful human control must be assured in deliberative communication if we want to maintain autonomy and dignity of both doctors and patients. Although the framework we present here is normative, it provides a necessary theoretical justification of the need to sustain human factor at all levels of AI-mediated clinical encounters.

2. Methods

This article applies theory-driven normative approach to examine the ethical conditions under which artificial intelligence may be integrated into clinical practice while preserving the moral foundations of medicine. The analysis is grounded primarily in Pellegrino and Thomasma’s [10] interpretation of the internal morality of medicine and the physician’s act of profession. This conceptual framework was selected because it views medicine as a moral practice oriented toward the good of the patient and therefore provides a strong basis for evaluating AI-assisted care.
The analysis is further relies on Kantian ethics of human dignity, Levinasian relational ethics, virtue ethics, and Vallor’s [15] concept of technomoral wisdom. These frameworks are used to clarify why clinical judgment cannot be reduced to technical prediction and why moral responsibility in medicine remains tied to human agency, relational presence, and practical wisdom.
The argument proceeds in four stages. First, we reconstruct the ethical significance of the physician–patient relationship in terms of vulnerability, dignity, trust, and clinical judgment. Second, we examine whether AI systems can satisfy the conditions required for moral agency in clinical practice, focusing on epistemic, relational, and phronetic responsibility. Third, we analyze how AI-mediated decision-making may affect trust, responsibility attribution, professional autonomy, and patient dignity. Fourth, we derive normative requirements for Meaningful Human Control in AI-assisted healthcare.
Empirical literature on AI implementation in healthcare, including studies on algorithmic opacity, automation bias, algorithmic bias, clinician accountability, and patient trust, was used to contextualize the normative analysis. Regulatory and governance documents, particularly the EU AI Act and WHO guidance on ethics and governance of AI for health, were examined as current institutional reference points. The proposed requirements for Meaningful Human Control were developed by applying the above ethical frameworks to the distinctive risks introduced by AI-assisted clinical decision-making.

3. The Internal Morality of Medicine and the Dimensions of Dignity

Medicine cannot be reduced to applied biology or a simple transactional service. It should rely on phronesis – practical wisdom required to apply general rules to an authentic, individualized clinical situation [17]. Illness, according to Pellegrino [11], is not just a breakdown of body, but an ‘ontological assault’ that compromises the patient’s freedom, rationality, identity, and self-determination. Therefore, the main goal (telos) of medicine is not purely epistemic but entails the ‘internal morality’ that is derived from the face-to-face clinical encounter between the doctor and the patient [11]. In this encounter, the physician faces not a clinical case but a vulnerable person, and it is here that the Kantian moral imperative “to treat human being never merely as a means but always as an end itself” [18] is met. The obligation to consider the patient’s best interests stems from the moral obligation to respect and protect the ‘essential and inviolable core of humanity’ in each person [19]. Even though dignity can be compromised by adverse individual and organizational conditions [20,21,22], healthcare professionals are expected to honor their patients’ esteem and take measures to mitigate their severe dependency [23].
Illness compromises physical and psychological independence. This condition calls for what Pellegrino terms the physician’s ‘act of profession’. In making such a declaration, the physician publicly commits to using specialized clinical knowledge exclusively for the patient’s healing and well-being. The physician–patient relationship is therefore founded upon a bond of trust and moral obligation rather than the mere application of technical solutions. Within this moral framework, Pellegrino argues that comprehensive clinical judgment must answer three consecutive questions. The physician must first determine ‘what can be wrong’ and ‘what can be done’ and then decide ‘what should be done for the patient’ [12]. Classical diagnostic tools historically helped doctors discover what could be wrong. Today, AI systems actively assist in addressing ‘what can be done’ by outlining sophisticated treatment alternatives. Answering this final question of ‘what should be done’ requires phronesis – the practical wisdom to integrate empirical knowledge with the patient’s values and context [17].
This need to address vulnerability demonstrates that safeguarding human dignity cannot be reduced solely to technical parameters or results of calculations [24]. Genuine care and respect for the patient’s dignity require a relationship in which an experienced specialist is able to truly hear, see, and value the patient [25]. Since medicine is a moral practice oriented toward the good of particular persons, honoring this dignity demands an active, empathetic engagement of a moral agent who understands the need to preserve patient-centered care as a whole [26]. It is within the physician’s moral responsibility to synthesize empirical data with human values and the unique perception of the good, and on that basis to take what Pellegrino calls the ‘right action’ [17].

4. AI, Moral Agency, and Moral Responsibility

The claim that only the physician can determine what should be done presupposes that clinical responsibility belongs to a moral agent, and here the integration of AI into clinical judgment becomes conceptually problematic. AI systems increasingly shape the evidential basis of medical decisions, yet they do not satisfy the conditions under which moral responsibility can be ascribed. The responsibility of the physician cannot be transferred to a system that lacks moral agency. Specifically, clinical moral agency lies in the capacity to bear three distinct responsibilities - epistemic, relational, and phronetic – none of which can be fundamentally sustained by an AI.
According to Pellegrino [17], a moral agent must understand moral norms, act consciously, and assume responsibility. Artificial intelligence systems do not meet any of these requirements. Even though artificial intelligence can optimize clinical data, it inherently lacks empathy and a ‘caring touch’ [27,28]. Although large language models (LLMs) can process ethical knowledge and generate the most likely solutions based on training data, such operations merely mimic rational thinking. Artificial intelligence lacks the consciousness required for moral agency and conscious engagement with what matters [29]. Furthermore, artificial intelligence lacks moral sensitivity, i.e., the uncodified, practical human ability to recognize morally significant aspects of a situation [30]. This deficit is not what larger datasets or more sophisticated architectures can remedy, since the knowledge in clinical judgment is more contextual than statistical. The missing point in this expectation is that moral responsibility which requires full self-conscious awareness and ability to comprehend what is right and wrong is what makes an entity a moral agent [31]. Therefore, even ethically preprogrammed AI tools cannot be regarded as moral agents because they lack the three types of responsibility.
Epistemic responsibility. The physician is responsible not only for what he or she knows, but for how that knowledge is applied on a specific individual. With AI, the question is no longer whether a prediction corresponds to reality, but what probability attaches to the predicted event. AI tools can produce an accurate prediction but not a moral justification of why a specific alternative should or should not be applied to specific patients. Only physicians will be able to distinguish what is statistically probable from what is clinically right [32,33,34]. Since AI operates within a scientific framework that does not seek to establish causal links, it cannot be considered epistemically superior to human clinical reasoning [35].
Relational responsibility. Even if an artificial intelligence tool were to meet the epistemological requirements of clinical reasoning, it would still lack the ability to respond to the patient as a suffering human being. Relational responsibility, in a Levinasian sense, requires a direct response to the call of another person’s suffering [36,37,38]. An algorithm processes biological and behavioral signals but cannot receive or perceive suffering as a call that demands an empathetic response. Delegating this responsibility to artificial intelligence could be viewed as negligence that violates human dignity, since the patient has no one to entrust with his or her vulnerability. Therefore, discussions about AI ‘autonomy’ often miss the fact that although these systems process data adaptively, they lack experiential or relational grounding [39]. Gordon and Gunkel [40] reject the exclusion of AI from moral agency on the basis of intrinsic properties such as free will and propose a relational account of moral status. But the relation that grounds moral agency in clinical practice is the face-to-face encounter, into which AI cannot enter.
Phronetic responsibility. Clinical responsibility requires practical wisdom, i.e., the capacity to act rightly in a particular case. AI cannot be regarded as a moral agent because it lacks tacit knowledge [41] that the Aristotelian tradition calls phronesis: the application of general principles to a unique situation. Even where robotic AI tools exceed human performance in matters of techne, they do not act contextually. AI cannot recognize when a standard protocol should be suspended to safeguard a patient’s dignity. Such recognition requires that the physician read clinical findings against what the patient reports of his or her existential condition. This is the work of phronesis, or ‘technomoral wisdom’ [15] that cannot be delegated to algorithms.

5. Anthropomorphism, Algorithmic Authority, and Erosion of Trust

Despite the fact that no AI system of any kind has any characteristics of a moral agent, the design of such systems can lead users into treating them as moral subjects. Clinical chatbots, for instance, simulate attentiveness and responsiveness in ways that their users or, in case of clinical settings, patients might regard as more empathic than interactions with human clinicians (e.g., Hippocratic AI). This simulation of communication by artificial intelligence systems leads to anthropomorphism: users begin to attribute moral characteristics and even moral responsibility to systems that have none of these distinctly human traits [42,43,44]. Such attribution can create the illusion of a trusting relationship with an AI system, which poses significant risks in a clinical setting. Trust means placing something valuable in another person’s care in a vulnerable situation [45]. A physician in traditional clinical settings is expected to ‘to honor fidelity to the trust’ [46]. However, in case of AI, it is paradoxical if a patient entrusts his or her vulnerability to technology that has neither free will nor obligations.
The belief that we can expect help even in worst situations is a critical element of trust. We put our trust in professionals who have the knowledge and skills to help. However, when a patient humanizes AI and puts his or her trust in it, the real ‘saving hand’ does not exist. At the same time, the majority of clinicians and patients believe that physicians should remain accountable for the errors of AI [8,47]. In clinical settings, this means that responsibility will depend on the physician’s ability to recognize those errors and further ‘recklessness in acting’ [48]. A physician’s ability to recognize errors, in turn, struggles with the ‘responsibility gap’, when the physician is no longer able to predict the behavior of the AI systems and therefore can no longer bear moral responsibility and liability for it [49].
Some experts, like Asadollahi [50], call this ‘responsibility choreography’. We agree with such a term for a situation where physicians still keep all the accountability and culpability but have lost the actual power (agency) to make responsible and ethical medical decisions. They are expected to be responsible for the outcome but are not the ones allowed to decide it. We believe this is not just a technical issue – this is a crisis for the medical profession.
This ‘responsibility gap’ is also a thinking issue that causes real difficulties. The problem is that AI can make mistakes. For example, some AI tools are trained only on certain groups of people, which means that such tools will not work well when the patient population is highly diverse [51,52]. In addition, an algorithm might work in one hospital but fail in another because every place uses the technology differently or simply because of the difference in the level of competences between AI users [53]. Some experts say we should keep checking the AI throughout its life to fix this [54,55]. However, this does not help the doctor at that particular moment. A doctor who is standing there with a patient cannot look inside the ‘black box’ to see why the AI gave a certain answer. Instead, the system just expects the doctor to stay alert and double-check the machine’s math against his or her own logic.
Obviously, vigilance is the quality physicians are expected to have. They are meant to notice all the small bits of relevant information and their mismatch. In real-life clinical settings, however, physicians often operate under time and bureaucratic pressure. Some studies suggest that under time pressure, physicians tend to succumb to ‘automation bias’ and accept the results generated by artificial intelligence without properly analyzing them [56,57]. ‘Confirmation bias’ lies on top of automation, and physicians often accept only the pieces of evidence that confirm the AI-solution and ignore the contradicting ones [58].
The proposal that clinicians should first form their own opinion on the case and only after that consult AI systems is reasonable [59]. However, it increases the cognitive load and leads to faster burnout. In the long-term perspective, there is a risk of ‘skill decay’, when physicians lose the sole skills they need to recognize AI’s errors.
As a result, the quality of patient-physician relations suffers when an ‘algorithmic authority’ takes over human judgement [60]. Therefore, the physicians’ perceived competence and commitment to patient welfare are to close the ‘reliability gap’ in which neither the doctor nor the patient can truly verify the work of the algorithm [61].
Otherwise, when the doctor loses autonomy, the patient loses it as well. We are seeing a new kind of ‘the computer knows best’ attitude, which has been called transitive paternalism [62,63]. AI prioritizes data points over the human side of medicine, pushing the patient’s personal values to the side [64,65].
The current situation in the healthcare sector raises a complex ethical question: if statistical data show that artificial intelligence is more accurate than a physician, are we morally obligated to use it?
Pellegrino’s [17] answer to this question would be a categorical ‘‘no.’’ True treatment requires ‘the right action,’ one that reconciles the patient’s values with the specific clinical context. Such a reconciliation can only be performed by a human being as a moral agent [66]. Clinical accuracy is important, yet this does not mean that it can replace the entire decision-making process. Ultimately, the impact of AI on healthcare in ethical matters depends on whether the autonomy of clinical practice is ensured [67].

6. From Regulatory Frameworks to Clinical Virtue: Meaningful Human Control

EU Regulation 2024/1689 [68], better known as the AI Act, may well be seen as the most complex and ambitious attempt to regulate the use of AI. The Act addresses the issue by establishing risk categories and oversight requirements. In healthcare, where AI is evaluated as high-risk systems, the rule is clear: it should help the physician to make a decision, but should not overtake this process.
The evaluation of the current situation in regulation differs among scholars. Some state that the basic normative framework has already been established. The AI Act sets umbrella-like set of rules, and other EU- and national-level legal instruments cover healthcare sector-specific issues [69]. Others note that significant gaps remain in questions of responsibility and oversight [70]. Despite the obvious progress, guidelines tailored to the requirements of the clinical setting still need to be developed in order to ensure a safe and trustworthy use of AI [71,72].
Compliance with regulatory requirements alone cannot mitigate the complicated issues arising in healthcare, especially the ethical ones. The AI Act tries to solve these issues technically, yet it does not touch the core ethical problem of the structural erosion of the physician’s moral agency. Based on the analysis above, we consider questions of responsibility and oversight worth understanding beyond the limits of formal requirements.
Our analysis requires paying deeper attention to two points of the AI Act’s framework. The Act sets the principle of shared responsibility: AI developers are responsible for safe design, and the physicians are responsible for safe use. Human oversight is mandatory, and the operators (physicians) should understand the limitations of models, prevent bias, and be able to interfere or shut down the system if needed. These provisions also cover management, high-quality training data, transparency, explainability, and accountability of AI models, as well as other requirements.
The WHO’s guidance [73] on ‘Ethics and Governance of Artificial Intelligence for Health’ also sets the aforementioned requirements as key governance elements, but also expands this framework by noting ethical requirements for protecting patient autonomy, building trust, preventing harm, and ensuring fairness.
Both documents regulate procedural compliance and mitigate technical harm. However, they do not address the question how clinical judgement and human dignity should be protected under AI pressure. Neither the EU Act nor the WHO guidance prevent the erosion of trust or the ‘statistical optimization’ of moral duty. ‘Human oversight’, unfortunately, mainly reflects only technical control functions.
We argue that another concept needs to be introduced. ‘Meaningful Human Control’ (MHS), we believe, is a better option, because it is more oriented towards the aspects of enabling, enacting, and evaluating control [74].
AI systems, as of today, are not yet ‘ethical rational agents’ and are not able to recognize moral rules and operate them [75]. Therefore, in decision making, humans should be in the driver’s seat, and an AI system should be traceable for these human agents [76]. In this regard, MHC should represent the idea that humans must keep real and effective oversight over AI systems in order to keep decisions, responsibilities, and societal values being shaped by humans [77]. Understanding such a purpose of MHC is essential because it shapes the institutional design of healthcare settings [78].
MHC, so understood, functions as a moral precondition for virtuous clinical practice. It exists to preserve the internal morality of medicine in the sense Pellegrino [11] defined. However, the two ideas need to be kept distinct here. The virtue is what physicians cultivate themselves. Pellegrino [79] called it ‘a habitual moral disposition to make right choices in complex clinical circumstances’, alongside with virtues of fidelity to trust and intellectual honesty. MHC, by contrast, is the institutional condition that allows this virtue to be exercised. Even the most virtuous clinician cannot practice medicine according to its internal morality if the surrounding structures do not permit it. Our concern in this section is primarily with securing those structures.
A dignity-oriented reading of MHC yields three interdependent obligations. Physicians must retain epistemic, moral, and relational authorities. These together form the baseline for dignity-preserving healthcare, and they correspond directly to the three responsibilities we examined earlier in connection with moral agency. Under epistemic control, it is necessary to ensure that clinicians understand the limitations, uncertainties, and biases of AI systems in order to control automation bias and protect the relation autonomy of clinicians. Moral control means that clinicians keep the freedom to engage morally sensitive issues without algorithmic pressure, which in turn requires acknowledging that AI lacks moral sensitivity and cannot perceive what is ethically at stake in a given clinical situation; this parallels phronetic responsibility. Relational control means that clinical interactions preserve the interpersonal conditions constitutive of social dignity and allow patients to participate meaningfully in decisions about their care; this parallels relational responsibility.
To ensure that MHC can function as an effective and ethically grounded safeguard in AI-supported healthcare, we propose five normative requirements. These requirements are meant jointly to satisfy the three control dimensions just outlined. We do not claim that they exhaust every requirement that might emerge as deployment patterns change, but we regard them as core conditions in whose absence meaningful human control cannot be sustained. It is worth noting what sets these requirements apart from ordinary professional autonomy. AI adds scale, opacity, and institutional-statistical authority to clinical recommendations in ways that can displace deliberation rather than inform it. The requirements provided below take into account these characteristics of the use of artificial intelligence in medical practice.
Primacy of Clinical Judgment. If meaningful human control is to exist, clinical judgment cannot be made subordinate to algorithmic output. Physicians need the authority to override or set aside AI recommendations whenever those recommendations conflict with the patient’s values, the clinical context, or the physician’s own professional assessment. A clinician who cannot decline an AI recommendation cannot sensibly be held accountable for its consequences. A patient whose care is dictated by an unchallengeable algorithm has been cut off from the relational and interpretive dimensions that uphold personal dignity in medicine. Protecting the primacy of clinical judgment therefore protects both the physician’s moral agency and the patient’s dignity.
Prohibition of Forced Automation. When clinicians are compelled to follow algorithmic pathways, they become operators of a technical system rather than independent moral agents. System designs that enforce such compulsion also heighten the danger that patients will be reduced from human beings, which undermines the interpersonal respect that clinical practice should embody. This ethical requirement transcends the procedural safeguards currently provided by the WHO and the EU. Hospital governance, professional accreditation bodies, and departmental protocols each have a part to play in ensuring that forced automation does not become entrenched in clinical workflows.
Traceability and Explainability. Decisions supported by AI ought to be traceable and, as far as possible, explainable. The aforementioned WHO and EU documents already require transparency, traceability, and explainability, but they frame these mainly as governance mechanisms. We argue for a stronger reading. Clinicians must understand enough about how an AI system generates its outputs to judge whether those outputs are reliable and where their limits lie. They must grasp enough about the system’s workings, biases, and sources of errors to know when its recommendations can be trusted in a particular case. Traceability should make it possible to reconstruct decisions by retracing key points in the AI’s reasoning process. Explainability should give physicians enough information to integrate AI outputs into their own deliberation. Without these elements, clinicians cannot critically assess algorithmic recommendations, and patients cannot give genuinely informed consent.
Transparency towards Patients. Patients should know when AI is being used in their diagnosis or treatment. They should be told the system’s purpose, likely benefits, risks, and limitations. Such transparency lets patients ask questions, voice preferences, and understand how their treatment is proceeding. Dignity-preserving communication of this kind reinforces trust in both the individual physician and the healthcare system as a whole. WHO and EU frameworks treat transparency as a procedural requirement; they do not ground it in dignity or recognize it as a moral practice foundational to trust.
Retaining Clinical Authority. The patient’s right to give or withhold informed consent for a particular treatment should not be stretched to include limiting the clinician’s choice of diagnostic or decision-support tools needed to deliver safe and responsible care. Selecting such tools, including AI systems, falls within the physician’s professional responsibility. MHC, therefore, requires that clinicians keep authority over the methods and technologies through which they discharge their moral and professional duties. A clarification is needed here. This requirement concerns clinical judgment authority over tools used in a specific case, not budgetary authority over institutional resources. Institutions legitimately decide which tools are available, while clinicians decide which available tools to use in a given situation. This distinction separates the professional autonomy we are defending from any broader guild claim over healthcare resources.
Where do these five requirements take effect? They span several levels of healthcare governance. Hospital governance bodies must ensure that institutional policies neither mandate forced automation nor undermine clinical authority. Professional societies and accreditation bodies should weave MHC principles into standards of practice and continuing education. Medical licensing authorities need to ensure that competence frameworks reflect the skills responsible AI-mediated practice demands. Departmental protocols must translate these principles into concrete workflow arrangements that preserve deliberative space for clinical judgment.

7. Conclusions

Ethical implementation of Artificial Intelligence in clinical settings requires more than human oversight over algorithmic outputs. A dialog between the physician and the patients as the sole moment that integrates scientific knowledge, patients’ values, and situational context should remain the cornerstone of clinical decision making. Artificial Intelligence-based systems lack moral sensitivity and relational presence, and therefore they cannot safeguard the dignity, which this dialogue is meant to protect. For that reason, they cannot gain full membership in it. However, Artificial Intelligence progressively affects the processes of diagnostics and treatment. Under such pressure, defending the borders of clinical responsibility and trust becomes essential.
The analysis of dignity in philosophical and clinical context, conducted in this article, shows that patients depend on physicians not only because of the technical skills the clinicians possess. Patients put trust in physicians due to their interpretive and relational engagement, which grants ethical validity to clinical decisions. If the boundaries are blurred, physicians risk losing the authority they need to perform their professional duties. However, they will remain legally and morally responsible for the results beyond their control.
The ground principles on which we believe AI-assisted healthcare should be built include primacy of clinical judgement, prohibition of forced automation, traceability and explainability, transparency towards patients, and clinical authority over diagnostic tools. Understanding of these principles should go beyond seeing them as administrative protocols and technical requirements. First of all, they are meant to protect the conditions under which virtuous clinical practice can continue in an AI-assisted era. Each principle safeguards a specific capacity physicians should be able to exercise: to align evidence with values, to resist algorithmic pressure and reconstruct its outputs, and to choose the proper tools and to reveal them to patient. Taken together, they aim to preserve what Pellegrino called the internal morality of medicine, in a form that sits comfortably with both technological progress and established medical ethics. Whether these five requirements will prove adequate to that task is a question that ongoing evaluation must answer as AI deployment evolves and new difficulties surface.
Artificial Intelligence should not overcome clinicians’ capacities but rather support and enhance them. Patients’ dignity requires protection in digital era. Responsibility for it lies on the shoulders of clinicians and manifests their moral duty towards their patients. We argue that Meaningful Human Control can be reached primarily through a meaningful communication between the physician and the patient. How this communication will be constructed, how physicians and patients will react to it, which points of it will be most vulnerable to algorithmic pressure – these are empirical questions that remain open. Qualitative and observational studies of AI-mediated clinical encounters could test and refine the normative framework we have developed here.

Author Contributions

Conceptualization, A.O. and G.U.; Methodology, E.P. and A.U.; Writing – Original Draft Preparation, A.O., E.P., A.U., G.U.; Writing – Review & Editing, A.O., G.U. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

During the preparation of this manuscript, the authors used Gemini 3.1 Pro, Claude Opus 4.7 for the purposes of editing for structure and clarity. The authors have reviewed and edited the output and take full responsibility for the content of this publication.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Parsons, T. Suggestions for a sociological approach to the theory of organizations. Adm. Sci. Q. 1956, 1, 63–85. [Google Scholar] [CrossRef]
  2. Szasz, T.S.; Hollender, M.H. A contribution to the philosophy of medicine: The basic models of the doctor–patient relationship. A.M.A. Arch. Intern. Med. 1956, 97, 585–592. [Google Scholar] [CrossRef] [PubMed]
  3. Topol, E.J. High-performance medicine: The convergence of human and artificial intelligence. Nat. Med. 2019, 25, 44–56. [Google Scholar] [CrossRef]
  4. Sauerbrei, A.; Kerasidou, A.; Lucivero, F.; Hallowell, N. The Impact of Artificial Intelligence on the Person-Centred Doctor-Patient Relationship: Some Problems and Solutions. BMC Med. Inform. Decis. Mak. 2023, 23, 73. [Google Scholar] [CrossRef]
  5. Hassanein, S.; El Arab, R.A.; Abdrbo, A.; Abu-Mahfouz, M.S.; Gaballah, M.K.F.; Seweid, M.M.; Almari, M.; Alzghoul, H. Artificial intelligence in nursing: an integrative review of clinical and operational impacts. Front. Digit. Health 2025, 7, 1552372. [Google Scholar] [CrossRef]
  6. Würtenberger, A.; Groneberg, D.A.; Mache, S. Digital stress perception and associations with work- and health-related outcomes among general practitioners in Germany: a quantitative study. BMC Health Serv. Res. 2025, 25, 535. [Google Scholar] [CrossRef]
  7. Bodur, G.; Cakir, H.; Turan, S.; Seren, A.K.H.; Goktas, P. Artificial intelligence in nursing practice: a qualitative study of nurses’ perspectives on opportunities, challenges, and ethical implications. BMC Nurs. 2025, 24, 1263. [Google Scholar] [CrossRef]
  8. Nouis, S.C.; Uren, V.; Jariwala, S. Evaluating accountability, transparency, and bias in AI-assisted healthcare decision-making: a qualitative study of healthcare professionals’ perspectives in the UK. BMC Med. Ethics 2025, 26, 89. [Google Scholar] [CrossRef] [PubMed]
  9. Tun, H.M.; Rahman, H.A.; Naing, L.; Malik, O.A. Trust in artificial intelligence-based clinical decision support systems among health care workers: systematic review. J. Med. Internet Res. 2025, 27, e69678. [Google Scholar] [CrossRef] [PubMed]
  10. Pellegrino, E.D.; Thomasma, D.C. A philosophical basis of medical practice: Toward a philosophy and ethic of the healing professions; Oxford University Press: New York, NY, USA, 1981. [Google Scholar]
  11. Pellegrino, E.D. The Internal Morality of Clinical Medicine: A Paradigm for the Ethics of the Helping and Healing Professions. J. Med. Philos. 2001, 26, 559–579. [Google Scholar] [CrossRef] [PubMed]
  12. Pellegrino, E.D.; Engelhardt, H.T., Jr. The Anatomy of Clinical Judgments. In Clinical Judgment: A Critical Appraisal; Spicker, S.F., ., Towers, B., Eds.; D. Reidel Publishing Company: Dordrecht, Netherlands, 1979; Volume 6, pp. 169–194. [Google Scholar] [CrossRef]
  13. Novelli, C.; Taddeo, M.; Floridi, L. Accountability in artificial intelligence: what it is and how it works. AI Soc. 2024, 39, 1871–1882. [Google Scholar] [CrossRef]
  14. Santoni de Sio, F.; van den Hoven, J. Meaningful human control over autonomous systems: a philosophical account. Front. Robot. AI 2018, 5, 15. [Google Scholar] [CrossRef]
  15. Vallor, S. Technology and the virtues: A philosophical guide to a future worth wanting; Oxford University Press: New York, NY, USA, 2016. [Google Scholar] [CrossRef]
  16. Charon, R. Narrative medicine: Honoring the stories of illness; Oxford University Press: Oxford and New York, NY, USA, 2006. [Google Scholar]
  17. Pellegrino, E.D. Toward a Virtue-Based Normative Ethics for the Health Professions. Kennedy Inst. Ethics J. 1995, 5, 253–277. [Google Scholar] [CrossRef]
  18. Kant, I. Groundwork of the Metaphysics of Morals; Cambridge University Press: Cambridge, UK, 1998. [Google Scholar]
  19. Groselj, U. Ethics a Model Is Important: Interview with Professor Edmund D. Pellegrino. Theor. Med. Bioeth. 2023, 44, 533–538. [Google Scholar] [CrossRef] [PubMed]
  20. Peičius, E.; Urbonas, G.; Harrison, W.D.; Urbonienė, A.; Kuznecovienė, J.; Butkevičienė, R.; Astromskė, K.; Kalėdienė, R. Dignity violations and barriers to dignity assurance for terminally ill patients at the end of life: a cross-sectional analysis. Medicina 2022, 58, 294. [Google Scholar] [CrossRef]
  21. Jacobson, N. A Taxonomy of Dignity: A Grounded Theory Study. BMC Int. Health Hum. Rights 2009, 9, 3. [Google Scholar] [CrossRef]
  22. Jacobson, N. Dignity violation in health care. Qual. Health Res. 2009, 19, 1536–1547. [Google Scholar] [CrossRef] [PubMed]
  23. Edlund, M.; Lindwall, L.; von Post, I.; Lindström, U.Å. Concept determination of human dignity. Nurs. Ethics 2013, 20, 851–860. [Google Scholar] [CrossRef]
  24. Różyńska, J. Taking the principle of the primacy of the human being seriously. Med. Health Care Philos. 2021, 24, 547–562. [Google Scholar] [CrossRef]
  25. Ullrich, L.; Buchholz, M.; Dietmayer, K.; Graichen, K. Toward fully autonomous driving: AI, challenges, opportunities, and needs. IEEE Access 2026, 14, 1–26. [Google Scholar] [CrossRef]
  26. Kumah, E. Artificial intelligence in healthcare and its implications for patient-centred care. Discov. Public Health 2025, 22, 524. [Google Scholar] [CrossRef]
  27. Stokes, F.; Palmer, A. Artificial intelligence and robotics in nursing: Ethics of caring as a guide to dividing tasks between AI and humans. Nurs. Philos. 2020, 21, e12306. [Google Scholar] [CrossRef] [PubMed]
  28. Alakwe, K.O. Human dignity in the era of artificial intelligence and robotics: Issues and prospects. J. Humanit. Soc. Sci. Stud. 2023, 5, 87–97. [Google Scholar] [CrossRef]
  29. Manna, R. The Problem of Moral Agency in Artificial Intelligence. In Proceedings of the 2021 IEEE Conference on Norbert Wiener in the 21st Century (21CW), Chennai, India, 22-25 July, 2021; pp. 1–4. [Google Scholar] [CrossRef]
  30. Graff, J. Moral Sensitivity and the Limits of Artificial Moral Agents. Ethics Inf. Technol. 2024, 26, 13. [Google Scholar] [CrossRef]
  31. Parthemore, J.; Whitby, B. Moral agency, moral responsibility, and artifacts: What existing artifacts fail to achieve (and why), and why they, nevertheless, can (and do!) make moral claims upon us. Int. J. Mach. Conscious. 2014, 6, 141–161. [Google Scholar] [CrossRef]
  32. Feldman, R.C.; Aldana, E.; Stein, K. Artificial Intelligence in the Health Care Space: How We Can Trust What We Cannot Know. Stanf. Law. Policy Rev. 2019, 30, 421–447. Available online: https://repository.uclawsf.edu/faculty_scholarship/1753 (accessed on 2026-05-03).
  33. Wadden, J.J. Defining the undefinable: the black box problem in healthcare artificial intelligence. J. Med. Ethics 2022, 48, 764–768. [Google Scholar] [CrossRef]
  34. Lang, B.H.; Nyholm, S.; Blumenthal-Barby, J. Responsibility Gaps and Black Box Healthcare AI: Shared Responsibilization as a Solution. DISO 2023, 2, 52. [Google Scholar] [CrossRef]
  35. Kerasidou, A.; Kerasidou, C. Epistemic authority and medical AI: epistemological differences and challenges in medical practice. Med. Health Care Philos. 2026, 29, 89–95. [Google Scholar] [CrossRef]
  36. Benaroyo, L. The significance of Emmanuel Levinas’ ethics of responsibility for medical judgment. Med. Health Care Philos. 2022, 25, 327–332. [Google Scholar] [CrossRef]
  37. Levinas, E. Totality and Infinity: An Essay on Exteriority; Duquesne University Press: Pittsburgh, PA, USA, 1969. [Google Scholar]
  38. Levinas, E. Ethics and Infinity: Conversations with Philippe Nemo; Duquesne University Press: Pittsburgh, PA, USA, 1985. [Google Scholar]
  39. Boyd, R.L.; Markowitz, D.M. Artificial intelligence and the psychology of human connection. Perspect. Psychol. Sci. 2026, 21, 1–29. [Google Scholar] [CrossRef]
  40. Gordon, J.-S.; Gunkel, D.J. Moral Status and Intelligent Robots. South. J. Philos. 2021, 60, 88–117. [Google Scholar] [CrossRef]
  41. Polanyi, M. The Tacit Dimension; Routledge & Kegan Paul: London, UK, 1966. [Google Scholar]
  42. Placani, A. Anthropomorphism in AI: Hype and Fallacy. AI Ethics 2024, 4, 1–9. [Google Scholar] [CrossRef]
  43. Ladak, A.; Loughnan, S.; Wilks, M. The moral psychology of artificial intelligence. Curr. Dir. Psychol. Sci. 2024, 33, 27–34. [Google Scholar] [CrossRef]
  44. Bonnefon, J.-F.; Rahwan, I.; Shariff, A. The moral psychology of artificial intelligence. Annu. Rev. Psychol. 2024, 75, 653–675. [Google Scholar] [CrossRef]
  45. DeCamp, M.; Tilburt, J.C. Why We Cannot Trust Artificial Intelligence in Medicine. Lancet Digit. Health 2019, 1, e390. [Google Scholar] [CrossRef]
  46. Pellegrino, E.D.; Thomasma, D.C. The Virtues in Medical Practice; Oxford University Press: New York, NY, USA, 1993. [Google Scholar] [CrossRef]
  47. Khullar, D.; Casalino, L.P.; Qian, Y.; Lu, Y.; Chang, E.; Aneja, S. Public vs Physician Views of Liability for Artificial Intelligence in Health Care. J. Am. Med. Inform. Assoc. 2021, 28, 1574–1577. [Google Scholar] [CrossRef] [PubMed]
  48. Verdicchio, M.; Perin, A. When Doctors and AI Interact: On Human Responsibility for Artificial Risks. Philos. Technol. 2022, 35. [Google Scholar] [CrossRef] [PubMed]
  49. Matthias, A. The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics Inf. Technol. 2004, 6, 175–183. [Google Scholar] [CrossRef]
  50. Asadollahi, A. Governing by design: algorithmic normativity, clinical standards, and health policy implications of AI in healthcare. AI Ethics 2026, 6, 119. [Google Scholar] [CrossRef]
  51. Drukker, K.; Chen, W.; Gichoya, J.W.; Gruszauskas, N.P.; Kalpathy-Cramer, J.; Koyejo, S.; Myers, K.J.; Sá, R.C.; Sahiner, B.; Whitney, H.M.; Zhang, Z.; Giger, M.L. Toward Fairness in Artificial Intelligence for Medical Image Analysis: Identification and Mitigation of Potential Biases in the Roadmap from Data Collection to Model Deployment. J. Med. Imaging 2023, 10, 061104. [Google Scholar] [CrossRef]
  52. Celi, L.A.; Cellini, J.; Charpignon, M.L.; Dee, E.C.; Dernoncourt, F.; Eber, R.; Mitchell, W.G.; Moukheiber, L.; Schirmer, J.; Situ, J.; Paguio, J.; Park, J.; Gichoya-Wawira, J.; Yao, S. MIT Critical Data. Sources of Bias in Artificial Intelligence That Perpetuate Healthcare Disparities – A Global Review. PLoS Digit. Health 2022, 1, e0000022. [Google Scholar] [CrossRef]
  53. Cross, J.L.; Choma, M.A.; Onofrey, J.A. Bias in Medical AI: Implications for Clinical Decision-Making. PLoS Digit. Health 2024, 3, e0000651. [Google Scholar] [CrossRef]
  54. Abramoff, M.D.; Tarver, M.E.; Loyo-Berrios, N.; Trujillo, S.; Char, D.; Obermeyer, Z.; Eydelman, M.B.; Maisel, W.H. Considerations for Addressing Bias in Artificial Intelligence for Health Equity. npj Digit. Med. 2023, 6, 170. [Google Scholar] [CrossRef] [PubMed]
  55. Vrudhula, A.; Kwan, A.C.; Ouyang, D.; Cheng, S. Machine Learning and Bias in Medical Imaging: Opportunities and Challenges. Circ. Cardiovasc. Imaging 2024, 17, e015495. [Google Scholar] [CrossRef]
  56. Bashkirova, A.; Krpan, D. Confirmation Bias in AI-Assisted Decision-Making: AI Triage Recommendations Congruent with Expert Judgments Increase Psychologist Trust and Recommendation Acceptance. Comput. Hum. Behav. Artif. Hum. 2024, 2, 100066. [Google Scholar] [CrossRef]
  57. Khera, R.; Simon, M.A.; Ross, J.S. Automation Bias and Assistive AI: Risk of Harm From AI-Driven Clinical Decision Support. JAMA 2023, 330, 2255–2257. [Google Scholar] [CrossRef] [PubMed]
  58. Rosbach, E.; Ammeling, J.; Krügel, S.; Kießig, A.; Fritz, A.; Ganz, J.; Puget, C.; Donovan, T.; Klang, A.; Köller, M.C.; Bolfa, P.; Tecilla, M.; Denk, D.; Kiupel, M.; Paraschou, G.; Kok, M.K.; Haake, A.F.H.; de Krijger, R.R.; Sonnen, A.F.-P.; Kasantikul, T.; Dorrestein, G.M.; Smedley, R.C.; Stathonikos, N.; Uhl, M.; Bertram, C.A.; Riener, A.; Aubreville, M. When Two Wrongs Don’t Make a Right: Examining Confirmation Bias and the Role of Time Pressure During Human-AI Collaboration in Computational Pathology. In Proceedings of the CHI ‘25: 2025 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 2025; Association for Computing Machinery: New York, NY, USA; pp. 1–18. [Google Scholar] [CrossRef]
  59. Rosenbacke, R.; Melhus, Å.; McKee, M.; Stuckler, D. AI and XAI Second Opinion: The Danger of False Confirmation in Human-AI Collaboration. J. Med. Ethics 2025, 51, 396–399. [Google Scholar] [CrossRef]
  60. Montanari Vergallo, G.; Campanozzi, L.L.; Gulino, M.; Bassis, L.; Ricci, P.; Zaami, S.; Marinelli, S.; Tambone, V.; Frati, P. How Could Artificial Intelligence Change the Doctor-Patient Relationship? A Medical Ethics Perspective. Healthcare 2025, 13, 2340. [Google Scholar] [CrossRef]
  61. Önder, O.; Buhr, E.; Rudra, P.; Ursin, F. Trust and Artificial Intelligence in the Doctor-Patient-Relationship: Epistemological Preconditions and Reliability Gaps. Ethics Inf. Technol. 2025, 27, 60. [Google Scholar] [CrossRef]
  62. Berber, A.; Mijić, J. Balancing Autonomies: The Impact of AI on Patient and Physician Decision-Making in Healthcare. In Reconfiguring Human Autonomy; Anzalone, M., Achella, S., Battaglia, F., Donise, A., Eds.; Springer: Cham, Switzerland, 2026; pp. 139–153. [Google Scholar] [CrossRef]
  63. McDougall, R.J. Computer knows best? The need for value-flexibility in medical AI. J. Med. Ethics 2019, 45, 156–160. [Google Scholar] [CrossRef] [PubMed]
  64. Zhang, J.; Zhang, Z. Ethics and Governance of Trustworthy Medical Artificial Intelligence. BMC Med. Inform. Decis. Mak. 2023, 23, 7. [Google Scholar] [CrossRef]
  65. Kingsford, P.A.; Ambrose, J.A. Artificial Intelligence and the Doctor-Patient Relationship. Am. J. Med. 2024, 137, 381–382. [Google Scholar] [CrossRef]
  66. Benzinger, L.; Ursin, F.; Balke, W.-T.; Kacprowski, T.; Salloch, S. Should Artificial Intelligence Be Used to Support Clinical Ethical Decision-Making? A Systematic Review of Reasons. BMC Med. Ethics 2023, 24, 48. [Google Scholar] [CrossRef]
  67. Lorenzini, G.; Arbelaez Ossa, L.; Shaw, D.M.; Elger, B.S. Artificial Intelligence and the Doctor-Patient Relationship: Expanding the Paradigm of Shared Decision Making. Bioethics 2023, 37, 424–429. [Google Scholar] [CrossRef]
  68. European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). OJ L 2024, 2024/1689. Available online: http://data.europa.eu/eli/reg/2024/1689/oj (accessed on 2026-05-03).
  69. Schmidt, J.; Schutte, N.M.; Buttigieg, S.; Novillo Ortiz, D.; Sutherland, E.; Anderson, M.; de Witte, B.; Peolsson, M.; Unim, B.; Pavlova, M.; Stern, A.D.; Mossialos, E.; van Kessel, R. Mapping the Regulatory Landscape for Artificial Intelligence in Health within the European Union. npj Digit. Med. 2024, 7, 229. [Google Scholar] [CrossRef] [PubMed]
  70. Vardas, E.P.; Marketou, M.; Vardas, P.E. Medicine, Healthcare and the AI Act: Gaps, Challenges and Future Implications. Eur. Heart J.-Digit. Health 2025, 6, 833–839. [Google Scholar] [CrossRef] [PubMed]
  71. Mateus, M.; Alho, I.; Neves, A.L.; Lopes, H.; Correia, M. ‘First, Do No Harm’ in the Digital Era: Examining the Practicality of the European Health Data Space Proposal and Ethical Implications of Artificial Intelligence: A Systematic Literature Review. BMC Med. Ethics 2026, 27, 35. [Google Scholar] [CrossRef]
  72. van Kolfschooten, H.; van Oirschot, J. The EU Artificial Intelligence Act (2024): Implications for Healthcare. Health Policy 2024, 149, 105152. [Google Scholar] [CrossRef]
  73. World Health Organization. Ethics and governance of artificial intelligence for health; WHO: Geneva, Switzerland, 2021; Available online: https://iris.who.int/handle/10665/345502 (accessed on 2026-05-03).
  74. Hille, E.M.; Hummel, P.; Braun, M. Meaningful Human Control over AI for Health? A Review. J. Med. Ethics 2023, 0, 1–9. [Google Scholar] [CrossRef]
  75. Manna, R.; Nath, R. Kantian Moral Agency and the Ethics of Artificial Intelligence. Problemos 2021, 100, 139–151. [Google Scholar] [CrossRef]
  76. Santoni de Sio, F.; Mecacci, G. Four Responsibility Gaps with Artificial Intelligence: Why They Matter and How to Address Them. Philos. Technol. 2021, 34, 1057–1084. [Google Scholar] [CrossRef]
  77. Robbins, S. The many meanings of meaningful human control. AI Ethics 2024, 4, 1377–1388. [Google Scholar] [CrossRef]
  78. Davidovic, J. On the purpose of meaningful human control of AI. Front. Big Data 2023, 5, 1017677. [Google Scholar] [CrossRef] [PubMed]
  79. Pellegrino, E.D. Professionalism, profession and the virtues of the good physician. Mt. Sinai J. Med. 2002, 69, 378–384. [Google Scholar] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.