Docs vastly outperform computer algorithms in diagnostic accuracy

Register now

The maxim “doctors know best” is getting support from a new study led by Harvard Medical School researchers that shows physicians made a correct diagnosis more than twice as often as 23 commonly used symptom checkers—websites and apps that help patients with self-diagnosis.

The study’s results, published October 10 in the journal JAMA Internal Medicine, are being touted as the first direct comparison between human-made and computer-based diagnoses.

In the study, doctors vastly outperformed symptom checkers, arriving at a correct diagnosis 72 percent of the time versus 34 percent of the time for the digital platforms. In addition, 84 percent of physicians listed the correct diagnosis in the top three possibilities, compared with 51 percent for the symptom checkers.

“People are out there using them,” says Ateev Mehrotra, MD, associate professor in the Department of Health Care Policy at Harvard Medical School and senior investigator for the study, who estimates that “there are more than 100 million uses per year of these symptom checkers” among consumers.

“While in the future computers might outperform physicians, these computer algorithms that are being used on a regular basis clearly do not,” adds Mehrotra.

Also See: Lawmakers question efficacy, security of health apps

Researchers evaluated the diagnostic accuracy of 23 symptom checkers using 45 clinical vignettes, which included the patient’s medical history but had no physical examination or test findings. The study also compared the diagnostic performance of physicians with symptom checkers for those same clinical vignettes using Human Dx, a web/app-based online platform on which doctors generate differential diagnoses for clinical vignettes.

When it comes to diagnostic accuracy, the difference between physicians and computer algorithms was most pronounced in more severe and less common conditions. However, there was less difference between doctors and the digital platforms in diagnosing acute and more common illnesses.

And, despite outperforming the symptom checkers, the physicians in the study still made diagnostic errors in about 15 percent of cases—a similar error rate highlighted by a recent Institute of Medicine report, according to Mehrotra.

“The combination of computer and physician might be better than physician alone,” concludes Mehrotra, who says developing computer-based algorithms to be used in conjunction with human decision-making may help further reduce diagnostic errors.

For reprint and licensing requests for this article, click here.