AI Assistant Reduces Misdiagnoses by 16% in Penda Health Trial

From January 30 to April 18, 2025, the Penda Health clinic network in Nairobi, Kenya, conducted a trial of the AI Consult system—an AI assistant for physicians. The study involved 106 primary care providers, akin to general practitioners, who encounter a wide range of diseases, increasing the risk of errors. The physicians were divided into two groups: one utilized AI Consult, while the other served as a control group.

What sets AI Consult apart is its collaborative approach; it does not make decisions for the doctor but works alongside them. The system analyzes medical histories, examination findings, and lab results, then categorizes the information into one of three colors: green (no issues), yellow (minor inconsistencies), and red (critical problems). Examples of issues identified include conflicting medication prescriptions, missing essential tests, and overlooked diagnoses.

In total, the doctors managed around 40,000 patient visits, from which over 5,000 appointments were randomly selected for review by a panel of 108 independent specialists. The results showed that the group using the AI assistant had a 16% reduction in diagnostic errors and a 13% decrease in treatment errors. When focusing solely on «red» errors, the findings were even more striking, with a 31% decrease in diagnostic mistakes and an 18% reduction in treatment errors. Over time, the number of «red» alerts decreased as physicians learned from their previous errors.

In this study, AI Consult operated on ChatGPT 4o, which became outdated by today’s standards, lagging twice behind ChatGPT 3 in the Healthbench benchmark—indicating that future versions could yield even better results.

P.S. You can support me by subscribing to the channel «escaped neural network«, where I explore the creative side of AI.