For the past several years, physicians have been reassured by many, I included, that artificial intelligence isn’t going to replace them.
Advocates sharing my position hold that AI will not replace physicians, it will assist and augment them. The physician will do more than simply remain in the loop, exercising professional judgment and relating to patients human to human. AI as a tool will increase physician freedom and free up physician time. And, in using the tool, physicians will correct it, improving AI qua tool.
But, to play devil’s advocate, what if we’re wrong and keeping the physician in the loop makes the AI worse?
That’s the considerably more disturbing possibility raised by a JAMA Perspective piece, Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care? Its authors, among them Dr. Ezekiel Emanuel, ask whether autonomous AI might eventually provide better medical care than physicians assisted by AI.
It’s tempting to argue about whether they’re right, but to play devil’s advocate, let’s assume that they are. Let’s assume that AI becomes demonstrably better than physicians at certain medical decisions. And, let’s assume that, at least in some circumstances, adding a physician to review the AI’s recommendation actually makes the result less accurate.
Now, we have a much more interesting problem.
Who’s practicing medicine?
For purposes of our thought experiment, let’s assume an AI system reviews a patient’s history, lab results, imaging studies, medications, genetic information, and thousands of other data points that no human physician could realistically process simultaneously. As a result, the AI system concludes that Drug A should be discontinued and Drug B prescribed instead.
In our devil’s advocate world, where do we go from here with that conclusion?
Traditionally, we know a physician would make the actual decision. In fact, our legal system in regard to medical practice is built around that fact.
One solution, of course, is to require a physician to review the AI’s decision and approve it. That’s the “human in the loop” model.
However, to play devil’s advocate, we’ve had to assume the uncomfortable premise that started this discussion: in some circumstances, inserting the physician into the decision actually makes the result worse. So, what exactly is the physician in the loop supposed to do?
If the physician simply clicks “approve,” physician oversight is theater. That’s because it’s maintaining the appearance of physician decision-making while the actual medical judgment is being exercised elsewhere, by AI.
Or, if the physician is required to independently analyze every recommendation, we’ve reinserted the supposedly inferior human decision-maker into a process specifically designed to take advantage of AI’s superior capabilities.
But, third choice, if no physician reviews the decision at all, the problem becomes even more interesting, because then we’ve effectively separated the practice of medicine from the physician.
Doing so is not merely an issue in regard to potential malpractice liability. States license people to practice medicine. Hospitals credential people and medical staffs grant privileges to people. Professional liability policies insure people and medical practices. Corporate practice of medicine laws in many states restrict who may control medical decisions precisely because those decisions are supposed to be made by people, that is, by licensed professionals.
None of that legal architecture was designed for an autonomous machine making medical judgments.
Interestingly, at almost exactly the same time that the JAMA piece raised the prospect of autonomous AI, the American Medical Association released a new framework named The Physician’s Role in the Digital and AI Era of Medicine. In it, the AMA emphasizes enduring physician responsibilities including clinical judgment, accountability, human connection, and stewardship of technology.
The AMA’s AI framework makes perfect sense in today’s world. The only problem is that AI has an annoying habit of refusing to remain in today’s world.
If autonomous AI actually becomes safer and more accurate than physician-supervised AI, insisting on human oversight could eventually produce a bizarre result: we’d be requiring the less accurate decision-maker to supervise the more accurate one.
If that were true, then questions such as the following become unavoidable: Who owes the duty of care? Who establishes the standard of care? Who obtains informed consent? Who is responsible when the AI is wrong? Is it the physician who didn’t make the decision? Is it the hospital that deployed the system? Is it the company that developed it and is that company functionally practicing medicine without a license?
I don’t pretend to know how those questions will ultimately be answered, but if Dr. Emanuel and his coauthors are right, we won’t be able to claim that AI is merely a “tool” and thereby avoid the analysis.
A stethoscope is a tool. An MRI machine is a tool. Neither decides what disease you have and tells you which drug to take.
But if AI eventually does the decision-making better than a physician can, we’re going to need an entirely new legal framework for determining who, or what, is actually practicing medicine.
To be complete in playing the role of devil’s advocate, this leads to perhaps an even more uncomfortable question:
If the machine makes the medical decision and the physician is there primarily to absorb the liability, who exactly is the tool?
