Hadoop Overcomes Shortfalls of EHRs, Data Warehouses

As a flexible, open source storage technology that enables healthcare organizations to store data in its native form, Hadoop is head and shoulders above electronic health records and enterprise data warehouses when it comes to the ability to run algorithms and query the sheer volume of data required for population health management to drive better outcomes.


As a flexible, open source storage technology that enables healthcare organizations to store data in its native form, Hadoop is head and shoulders above electronic health records and enterprise data warehouses when it comes to the ability to run algorithms and query the sheer volume of data required for population health management to drive better outcomes.  

That’s the view of Charles Boicey, enterprise analytics architect for Stony Brook Medicine, an academic research institution and medical center located on Long Island, N.Y. Boicey, who is responsible for Stony Brook Medicine’s population management ecosystem, spoke on Tuesday at Health Data Management’s Healthcare Analytics Symposium in Chicago.

“I was really hooked on the ability to store this data and then retrieve it in sub-second time,” he told the conference. In addition, Boicey said that a clinician with a user interface has the ability to ask a simple question such as “I want to see all my diabetic patients that haven’t had an A1C in the last 6 months,” which he argued providers can’t do with an EHR system.   

According to Boicey, EHRs were not designed to process high volume/velocity data, nor were they intended for complex algorithm-based operations and pattern recognition. He also makes the case that an EHR is primarily transactional in nature, taking feeds from source systems. Further, Boicey points out that enterprise data warehouses suffer from latency problems that limit its real-time capabilities.

“The trouble with both [EHR and EDW] is that we don’t get the data in its entirety and in its native form,” he lamented. However, when it comes to leveraging big data in healthcare, Boicey says Hadoop is capable of ingesting data “in its complete form, in its entirety, and its natural state” regardless of data format or speed of ingest, providing a big data analytics platform that can’t be performed within the EHR environment.

He added that the Cleveland Clinic, Mayo Clinic and U.S. Department of Veterans Affairs are early adopters of the technology.

Also See: Why Hadoop (Truly) Matters

Jerrod Gladden, regional vice president of healthcare for analytics vendor Actian, agreed with Boicey that Hadoop provides capabilities desperately needed in healthcare. “Systems are under pressure and the way we’ve done things in the old days needs to change,” Gladden argued. “These systems were built for operational workloads but we’re using them for analytics—and that’s not what they were built for.” 

Actian’s end-to-end analytics platform, which runs 100 percent natively in Hadoop, is helping organizations to harness big data to drive better outcomes by gaining insights from the data. For example, a state agency was able to uncover and recover duplicate or erroneous Medicaid claims payouts using the Actian analytics platform. The agency’s ability to detect data redundancies and correlate records was achieved with greater speed and precision than was previously possible.

More for you

Loading data for hdm_tax_topic #better-outcomes...