How health IT real-world testing will affect providers, patients
The burden of the process lies on IT product developers, but the program is likely to have broader implications for the healthcare industry.
The real-world testing of certified health information technology products is finally underway. And while health IT developers have the obligation to assess how their products perform in the real world, the testing will have both direct and indirect effects on providers, patients and others.
Real-world testing (RWT) is one of the Conditions of Health IT Certification established by the 21st Century Cures Act Final Rule issued in 2020 to test health IT products’ interoperability and functionality.
Developers need to submit their RWT plans, conduct the testing and report the results each year. The plans and results will be publicly available on ONC’s Certified Health IT Product List.
“It’s another way vendors can prove that their software really works,” says Jeff Robbins, president and founder of New Orleans-based Dynamic Health IT, which is helping several health IT developers engage in RWT.
The basics of testing
The Department of Health and Human Services expects RWT to occur in the clinical settings in which a health IT product is marketed and, when possible, using real patient data and production environments.
“The intent of RWT is to demonstrate that a product as certified in the lab continues to perform as expected and provide a feedback loop to learn from,” explains Hans Buitendijk, chair of the Electronic Health Records Association (EHRA).
The RWT program has only just launched. Developers had to submit their first RWT plans by Dec. 15, 2021, and are now in the year-long process of collecting and measuring how their products operate in the field, according to Rob Anthony, director of the certification and testing division in ONC’s Office of Technology. The first results will be published by March 15, 2023.
Not surprisingly, the most immediate impact on providers is the burden on those participating in RWT to provide and exchange data for care coordination and other uses.
“RWT is an opportunity to improve usability.”
The final rule gives health IT developers the flexibility to design the testing to minimize the burden on these providers. Some developers are attempting to do so by pulling data from logs and other metrics rather than end use deployment, says Michelle Knighton, vice chair of EHRA’s certification workgroup and director of regulatory programs for NextGen Healthcare.
Providers may also be receiving compensation as a way to offset the cost or burden for participating in RWT. The final rule does not prohibit this form of compensation, however, there is a potential risk that providing compensation or other concessions could incentivize a provider to tilt the testing results, warns Robbins.
Enabling product comparisons
One of the biggest benefits of RWT will be the ability of providers to compare how different products perform in the real world and to use that information when purchasing health IT solutions, says Michelle Bond, sales director for Dynamic Health IT.
In turn, RWT will likely influence health IT developers to improve their health IT products in response to the testing results, to be more competitive in the market. “RWT is an opportunity to improve usability,” says Knighton.
And while RWT does not reflect on patient outcomes per se, a better-functioning health IT product can have an impact on the point of care, contends Anthony.
RWT also may enhance patients’ access to their electronic health records. One of the criteria being measured in RWT is a patient’s ability to view, download and transmit records to a third party, Robbins says, adding that the RWT requirements to measure how products transmit patient data to public health authorities will additionally have a population-level positive impact for disease monitoring and epidemiology.
Potential downstream effects
As RWT matures, it may inform ONC about what measures should be used in RWT and in what ways the program should be updated. “ONC will see if the information we can glean will help developers determine what makes for a better measure. We’ll know more once we get results in and see which are most robust. It’s still a learning process the first year,” says Anthony.
Buitendijk also sees the long view. “This is round one. We’re all going to learn more, and over time, ONC will start to react to that. It takes two to three rounds to understand how much it’s helping and where there’s burden,” he says.
The ultimate goal is increased interoperability and functionality, which will improve the healthcare ecosystem, which is expected to provide benefits for all stakeholders. “Overall, the intent is that everyone will benefit from continuous improvement and provide insight of how adoption and progression of interoperability are taking hold,” Buitendijk concludes.