Algorithms show potential in measuring diagnostic errors using big data
While the problem of diagnostic errors is widespread in medicine, with an estimated 12 million Americans affected annually, a new approach to quantifying and monitoring these errors has the potential to prevent serious patient injuries, including disability or death.
“The single biggest impediment to making progress is the lack of operational measures of diagnostic errors,” says David Newman-Toker, MD, director of the Johns Hopkins Armstrong Institute Center for Diagnostic Excellence. “It’s very difficult to measure because we haven’t had the tools to look for it in a systematic way. And most of the methods that look for diagnostics errors involve training people to do labor-intensive chart reviews.”
However, a new method—called the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE)—uncovers misdiagnosis-related harms using specific algorithms and big data. The automated approach could replace labor-intensive reviews of medical records by hospital staff, which researchers contend are limited by poor clinical documentation, low reliability and inherent bias.
According to Newman-Toker, SPADE utilizes statistical analyses to identify critical patterns that measure the rate of diagnostic error by analyzing large, existing clinical and claims datasets containing hundreds of thousands of patient visits. Specifically, algorithms are leveraged to look for common symptoms prompting a physician visit and then pairing them with one or more diseases that could be misdiagnosed in those clinical contexts.
“We did some work in this area as it relates to stroke,” says Newman-Toker, who is also a professor of neurology at the Johns Hopkins University School of Medicine. “Using SPADE, we can measure how often a patient comes to the hospital with dizziness, is mistakenly told it’s a benign ear condition, is sent home and comes back with a big stroke.”
In addition, he contends the metrics “can also measure how often a patient comes to a clinic with a fever, is told it’s a viral infection but is later admitted to the hospital with bacterial sepsis.”
Newman-Toker says that SPADE is most effective when applied to acute and subacute diseases for which a misdiagnosis that leads to hospitalization, disability or death is likely to occur within six months to a year. “You can do this basically for any acute, harmful disease where the patient is at increased risk of suffering consequences because of the misdiagnosis.”
Ultimately, he believes the approach to quantifying and monitoring these errors could be leveraged for operational diagnostic performance dashboards and national benchmarking, resulting in improvements in quality and safety across a broad range of clinical problems and settings.
The framework for establishing valid symptom-disease pairs using data analytics is described in an article published on Monday in the journal BMJ Quality & Safety.
“This is the first real description of a method that could be used broadly across a range of conditions to operationally measure diagnostic errors and associated bad outcomes so that we can track our performance and see whether our interventions are making a difference,” notes Newman-Toker.
In the case of stroke, the Armstrong Institute leverages a “family” of analytic algorithms that “look back” and “look forward” measuring diagnostic error and misdiagnosis-related harms, according to Newman-Toker. “We’re in the process of expanding to other diseases beyond stroke.”
While further research is needed to validate SPADE across a wider range of symptoms and diseases, he sees the method as being applicable to causes of disability and death from diagnostic error such as cancers, infections and vascular events. “All of the data that’s been published on this subject suggests that this approach works,” concludes Newman-Toker. “Even for cancer, which is a slightly tougher nut to crack, we believe that it can be done with a slightly more sophisticated data analysis.”