Scientists have developed the first platform that allows medical artificial intelligence algorithms to be tested in conditions that closely mirror real clinical practice.

The platform makes it possible to assess how accurate, reliable, and fair such systems are for different patient groups. As a first demonstration, the researchers tested eight algorithms designed to detect diabetic eye disease — one of the most common complications of diabetes.

Normally, when hospitals choose a new AI tool, they primarily look at cost and compare its performance to that of physicians. But important questions often remain unanswered: Does the algorithm work equally well across different age and ethnic groups? Does it avoid hidden bias? Can it deliver equally accurate results for all patients? The new project directly addresses these issues.

The study was published in The Lancet Digital Health. To conduct the evaluation, scientists used a massive database of about 1.2 million eye images from patients who underwent screening in London.

This is one of the most diverse populations in terms of age, health status, and background, making the findings particularly robust.

The algorithms demonstrated high accuracy. Depending on the type of analysis, their performance ranged from about 83% to nearly 99%. In other words, the systems confidently detected both early signs of disease and more serious changes that can lead to vision loss. Importantly, the algorithms performed equally well across people of different skin tones and ethnic backgrounds — a finding confirmed for the first time on such a large dataset.

According to the researchers, this platform could eventually form the basis of a national system: clinics could upload images, receive automated assessments, and physicians could make decisions more quickly. This would benefit patients, clinicians, and AI developers alike, ensuring fair evaluation of technologies and faster diagnosis.