The Hospital Compare website, operated by the Centers for Medicare and Medicaid Services, is meant to help consumers learn about the quality of hospitals, but a recent study contends that the statistical methodology used by Hospital Compare underestimates heart attack or Acute Myocardial Infarction (AMI) mortality rates for small hospitals.

“The underestimation of AMI mortality rates at small hospitals, as seen in Hospital Compare, contradicts previously established research and consistent findings that mortality rates are typically higher at low-volume hospitals,” said Jeffrey Silber, MD, co-author of the study, professor of pediatrics at the Children's Hospital of Philadelphia and professor of health care management at The Wharton School.

Jeffrey Silber, MD

The website is intended to help patients and their families make decisions about providers by providing a side-by-side comparison between facilities in their area.

The study, published in the Journal of the American Statistical Association, makes the case that Hospital Compare’s statistical methodology—the random effects logit model—actually shrinks mortality rates from small hospitals to resemble the national average.

“Hospital Compare’s finding of average risk at small hospitals is a mistake because the current model is not properly calibrated,” said Edward George, professor of statistics at the University of Pennsylvania’s Wharton School and co-author of the study. “It’s a mistake that has implications for patients.”

Also See: Hospitals come up short in new CMS rating system

CMS officials were not immediately available for comment about the findings of the study.

Nonetheless, last year, the agency issued a statement saying it “designed the methodology to be inclusive of as many hospitals and as many measures as possible,” which “prevents the methodology from limiting star rating calculations to certain types of hospitals based on characteristic or size.” However, at the same time, CMS noted that it will “continue to re-evaluate and make any needed modifications to the methodology over time.”

“As a model for AMI hospital mortality rates, we have found the hierarchical random effect logit model used by Hospital Compare to be inadequate, compared to alternatives that model hospital effects as a functions of hospital attributes,” conclude the authors, who add that “patients deserve to have the most accurate information available so they can make well-informed healthcare decisions.”