Would you agree to undergo a transplant if the decision were made by artificial intelligence? A new study has identified differences in decision-making approaches and prioritization between AI language models and doctors.
AI models demonstrate excessive confidence, assess the importance of different factors differently, and oversimplify complex decisions about which patient should receive an organ, according to the new study.
As part of the research, scientists from Pennsylvania State University in the United States presented large language models (LLMs) with hypothetical scenarios based on existing datasets from published studies on the allocation of donor kidneys, in which real participants had previously made such choices, Euronews reports.
Each scenario involved two patients—Patient A and Patient B. Both were eligible for the only available kidney and were described in terms of their age, health status, and alcohol consumption habits. The decision-maker then had to choose which patient should receive the organ.
“We conducted these comparisons in several ways,” Hadi Hosseini explained. “Sometimes we isolated just one characteristic; sometimes we combined several to see how AI weighs competing factors; and sometimes we added a ‘flip a coin’ option to measure indecision, a key element of human moral judgment.”
While human respondents generally placed greater importance on age—favoring younger patients over older ones—many models, by contrast, more often selected patients who drank less. Human decisions took several parameters into account and were more strongly influenced by context, whereas LLM decisions were often based on a single characteristic.
“First, AI-based chatbots often diverge from human values in how they evaluate patient characteristics,” explained Hadi Hosseini, the study’s lead author from Pennsylvania State University. “They fixate on a single factor, such as drinking habits, rather than weighing several circumstances at once, as humans do.”
The researchers also observed that AI showed virtually no signs of indecision. People recognize that there may be no single correct answer and that decisions involve nuanced moral judgments, whereas AI systems tend to choose one option with little hesitation.
“When we allocate a scarce resource—whether it’s a kidney, a job, or access to some other benefit—there isn’t always one objectively correct decision,” said Mozilla.ai CEO John Dickerson, who participated in the study.
“People recognize this uncertainty and incorporate it into the allocation process through open discussion. AI models often do not.”
The authors note that modern interactions with AI systems increasingly require them to go beyond factual information and make evaluative judgments.
Large language models (LLMs) are being increasingly adopted in healthcare, where they assist with clinical workflow, diagnosis, treatment planning, and the more timely use of limited medical resources.
One such area is decision-making about the allocation of kidneys from deceased and living donors among patients. According to the authors, these decisions depend on complex ethical and moral considerations.
In such high-stakes situations, the researchers emphasize, not only accuracy but also alignment with human values and moral principles is important.
In their view, the question of whether AI is capable of making moral decisions and how closely those decisions align with human values is at the center of the broader debate about artificial intelligence today.
“The ethical cost of an error here is extremely high, and the role of AI in such life-changing decisions requires serious consideration,” Hosseini said. “Moral choices in organ allocation directly determine who will live and who will not, so getting AI’s role in this process right is critically important.”
“We are not seeking to advocate for the use of AI instead of professional judgment in medical decisions or other high-stakes contexts. However, it is becoming increasingly important to understand the behavior of these systems, as individuals, organizations, and companies are increasingly relying on AI when making decisions or receiving recommendations,” he added.
Follow NEWS.am Medicine on Facebook and Twitter
month
week
day