6 key components in evaluating an HIE

Published
  • April 25 2017, 4:00am EDT

HIE development depends on solid assessments of the key components

A comprehensive report from the Agency for Healthcare Research and Quality outlines a six-step process for healthcare provider organizations to evaluate the creation of a heath information exchange and to develop a realistic plan. A key first step in developing an HIT project is selecting and tasking an evaluation team, which has members with expertise to tackle some of the core components of the program. The scope of these team members, and their areas of expertise, is outlined in the following slides.

AHRQ emphasizes that the evaluation team is only the first step in the process. Other steps crucial to making an HIE decision include characterizing the project; assessing the value; developing the evaluation plan; creating the dissemination plan; and utilizing evaluation measures.

1. Technical implementation team

HIT implementation experts are needed, ideally those who have experience in HIE projects and standards, as well as clinical data warehousing. They can help you determine what is technically possible to measure; understand the technical infrastructure of HIE; and the types and sources of data being exchanged, in order to decide what data are available for the evaluation and estimate the effort required to collect and process the data. This team also requires technical expertise and authority to demonstrate the feasibility of the evaluation plan to project stakeholders.

Content Continues Below

2. Healthcare operations team

These individuals have a detailed understanding of the administrative and business side of healthcare operations. Their expertise is needed to document healthcare workflows pre- and post-HIE to determine efficacy and measure impact. While technical expertise enables data exchange and captures the raw measurement data, healthcare operations expertise is needed to understand how the data can be used to create meaningful measures.

This group needs a healthcare operations expert who can communicate with staff and administrators represent their needs to the evaluation team and help determine what stakeholders need to measure. Also necessary is an expert on determining the validity of measures for financial assessments of HIE costs and savings, such as reduced utilization of services.

3. Clinical care team

This team should include well-respected clinicians in the community serving as clinical champions. HIE projects nearly always have a stated goal of improving clinical outcomes and clinician acceptance of HIE and the evaluation goals will be major determinants of the success and sustainability of the HIE. It would be ideal to find an evaluation member with both clinical and healthcare operations expertise to help balance the two perspectives, such as a senior clinician with administrative experience.

4. Research methodology team

These are people with experience designing and conducting healthcare studies, preferably from a public health or population-level perspective, with training in biostatistics. Research methodologists help determine which measures are feasible and relevant, and the corresponding data and sample size requirements. They also can help determine appropriate study design and data collection tools, analyze quantitative data and interpret study results. They further can manage protection of human subjects and HIPAA compliance.

Content Continues Below

5. Project management team

These individuals have demonstrated project management expertise, preferably with experience in multi-organizational projects and ideally have health IT experience. They can help develop and manage the evaluation work plan. This team will manage data collection from multiple organizations and coordinate activities across organizations and their stakeholders. An experienced project manager can serve as the bridge across parties and the evaluation team, and help ensure timely completion of tasks.

6. Healthcare consumer representation

These are patients or their caregivers who can represent the patient perspective to the evaluation team. They should be influential and well-respected, familiar with the consumer experience in the HIE community and able to share first-hand experiences. Even though almost everyone is a patient at some point, it is preferable to have a dedicated patient representative rather than asking team members to represent multiple perspectives.

A caveat: Beware of shortcuts

Although an organization might be able to conduct a successful evaluation with a team that lacks some of these areas of expertise, the absence of these professional skills and knowledge has limited many HIE project evaluation efforts. AHRQ strongly recommends that organizations assemble a team with all of the above components of expertise before developing an evaluation plan. The complete AHRQ report on HIE evaluation is available here.