Educational Strategies for Clinical Supervision of Artificial Intelligence Use

To be contacted at rabdulnour@bwh.harvard.edu or at the Department of Medicine, Brigham and Women’s Hospital, 75 Francis St., Boston, MA 02115.

N Engl J Med 2025;393:786-97.

DOI: 10.1056/NEJMra2503232

The new england journal of medicine

Review Article

Medical Education

Educational Strategies for Clinical Supervision of Artificial Intelligence Use

Raja‑Elie E. Abdulnour, M.D.,¹ Brian Gin, M.D., Ph.D.,²

and Christy K. Boscardin, Ph.D.^3,4

H^{uman–computer interactions have been occurring for de cades, but recent technological developments in medical artificial intelli}gence (AI) have resulted in more effective and potentially more dangerous interactions. Although the hype around AI resonates with previous technological revolutions, such as the development of the Internet and the electronic health re cord,¹ the appearance of large language models (LLMs) seems different. LLMs can simulate knowledge generation and clinical reasoning with humanlike fluency, which gives them the appearance of agency and independent information process ing.² Therefore, AI has the capacity to fundamentally alter medical learning and practice.^3,4 As in other professions,⁵ the use of AI in medical training could result in professionals who are highly efficient yet less capable of independent problem solving and critical evaluation than their pre-AI counterparts.

Such a challenge presents educational opportunities and risks. AI can enhance simulation-based learning,⁶ knowledge recall, and just-in-time feedback⁷ and can be used for cognitive off-loading of rote tasks. With cognitive off-loading, learners rely on AI to reduce the load on their working memory, a strategy that facilitates mental engagement with more-demanding tasks.⁸ However, off-loading of complex tasks, such as clinical reasoning and decision making, can potentially lead to automation bias (overreliance on automated systems and risk of error), “deskill ing” (loss of previously acquired skills), “never-skilling” (failure to develop essential competencies), and “mis-skilling” (reinforcement of incorrect behavior due to AI errors or bias).⁹ These risks are especially troubling because LLMs operate as unpre dictable black boxes¹⁰; they generate probabilistic responses with low reasoning transparency, which limits assessment of their reliability. For example, in one study, more than a third of advanced medical students missed erroneous LLM answers to clinical scenarios.¹¹

The inherent variability and potential inaccuracies of AI-generated output can leave even experienced clinicians uncertain about AI recommendations. This di lemma is not novel; it mirrors the broader challenge of confronting unfamiliar clinical problems. Such moments require adaptive practice — the capacity to shift fluidly between efficient, familiar, routinized behavior and innovative, flexible problem solving.¹² Critical thinking is the structured cognitive tool set that under lies adaptive practice in the face of uncertainty. It enables clinicians to bring as sumptions to the surface and engage in self-reflection that helps them recognize knowledge gaps and biases, mitigate errors, adapt to new problems, and generate or adopt new knowledge (i.e., learn).^13,14 Thus, critical thinking is foundational to adaptive practice in the age of AI.

Clinicians supervising medical learners, henceforth referred to as educators, must explicitly teach, assess, and model critical thinking to promote lifelong adaptive …

The New England Journal of Medicine | Research & Review Articles on Disease & Clinical Practice