“It’s really critical that the CIO communicate the risks to the organization of agreeing to be a beta site,” says Terry Wilk, vice president and CIO at 215-bed Henry Medical Center, Stockbridge, Ga. Wilk was involved in a challenging beta test in his previous job at another hospital.
“It’s going to take more time than you think; whatever the estimate is, you probably need to plan on doubling that amount of time,” the CIO says. “You must communicate that to everyone upfront and not just let the vendor set expectations. We have found the vendors to be so excited to have a real, live hospital serve as a guinea pig that they gloss over the details.”
Nevertheless, Wilk says he’d be willing to be a beta tester again–for the right opportunity to influence the design of a system that meets the organization’s specific needs.
Expect Some Pain
“It’s going to be painful; it’s not an easy thing to do,” he stresses. “The whole idea is to break the system so it can be fixed, and that’s why you get some stuff for free.
“You have to go into it expecting problems to arise.”
Being a beta tester can help a hospital solve a specific problem, says Judy Corzine, administrative director and CIO at Stormont-Vail Regional Healthcare, Topeka, Kan. The integrated delivery system is testing on behalf of its 175 employed physicians the latest releases of practice management and outpatient electronic health records software from NextGen Healthcare Information Systems Inc., Horsham, Pa.
“We have some specific issues on the billing side, so the intent is to reduce days in accounts receivable dramatically,” Corzine says. “If we wait for someone else to do the test, the software may not be designed to meet our needs.”
The organization is testing new workflows for billing plus enhanced electronic prescribing, among other functions. So far, the provider has spent about six months analyzing various refinements of the software. “Part of the reality of beta testing is that you learn things along the way,” Corzine says.
Because NextGen automatically gives its clients free upgrades to new releases, Stormont-Vail is not reaping any special discounts for being a beta tester, Corzine points out. But for many hospitals and delivery systems, obtaining discounted or free software can be a major benefit of serving as a test site, consultants say.
Some vendors will waive fees for upgrades or maintenance for beta testers, notes Barbara Cox, senior principal at Noblis Inc., a Falls Church, Va.-based, not-for-profit research and consulting firm. In some cases, vendors will make donations to a hospital’s foundation, she adds. “I like to see in contracts some way of reimbursing the hospital for the cost of testing the new product and establishing its credibility in the market,” she says.
For example, a hospital could demand that once staff members have devoted a specific amount of time to the project, the vendor will pay for additional time at an hourly rate, the consultant advises. “But vendors usually will resist that mightily.”
One significant non-financial benefit of being a test site is that “you get a lot of attention from the vendor,” says Corzine of Stormont-Vail. “So in the future, if you run into issues, they get quickly resolved.”
Words of Wisdom
The key to a successful beta test is managing expectations, consultants and CIOs say.
“Make sure at the outset that there’s a clear picture of what you want to accomplish and both parties buy into it,” says Laura Jantos, principal at ECG Management Consultants Inc., Seattle. “You have to understand your organization and have a good sense of whether it is sophisticated in managing the level of risk inherent in being a beta site,” she adds.
Jantos points out that many beta tests now involve upgrades of existing systems or additions of new modules, rather than development of new applications.
“In some cases, hospitals may not know that they are getting into a beta relationship,” she cautions. “It may be couched in language that doesn’t sound like beta.”
Cox, the Noblis consultant, says CIOs must stress to other senior executives that any test might fail. “You have to understand that it’s a beta project and may not be permanent,” she says.
“If people in the organization buy into the idea that this is a higher risk investment and involves pushing the envelope, then when bugs are encountered they will deal with them,” says Alan Dowling, executive director of the center for health innovation at Noblis. “If they are not aware of the risks upfront, it can create a very difficult political situation in the organization.”
Hospitals must carefully determine if a beta test has the potential to yield an application that’s critical to achieving a specific goal, Dowling adds. “If the application is not critical to success, then it may not be worth considering,” he says. “But if you’re looking for a specific capability, and the test could yield a potential product that would satisfy a need, that’s a different story.”
Cox stresses that hospitals testing clinical systems must set up a separate test environment “so that the testing doesn’t affect the production environment or touch the patient in any way.”
At Stormont-Vail, for example, I.T. staff members created a duplicate production environment to test that all data would flow well among various systems. “So it’s pretty complex,” Corzine says.
The CIO stresses that estimating the amount of time involved in beta tests is extremely difficult. “The main time commitments are systems engineers putting it in the test environment; applications staff testing the software; and interface staff making sure all interfaces work. Plus there are meetings with the vendor to validate everything,” she says.
“Virtually every beta contract I’ve seen has materially underestimated the cost of people and processes to support the new adoption,” says Dowling of Noblis.
Be the first to comment on this post using the section below.