Unquestioned acceptance of AI runs into some rational roadblocks
While providers rush to achieve potential benefits, some healthcare constituents recognize the need for caution and thoughtful implementation.
While healthcare organizations are still agog over the potential benefits of artificial intelligence, outside constituencies aren’t ready to give them carte blanche for just any use in all instances.
Consumers say they have a vote in what forms of AI they’re willing to accept, and more professional organizations are insisting that key guardrails need to be in place to ensure appropriate use of the technology by providers and other healthcare entities.
And increasingly, healthcare organizations are heeding these concerns. More examples are emerging of how they’re working with clinicians and vendors to engage diverse perspectives on optimal ways to use AI.
Consumer caution
Consumers are growing in acceptance of the AI technology, but specific use cases can be challenging, according to results of a recent study by Druid AI, an enterprise platform for AI agents.
The study draws on 15 months of user experience with agents in education, healthcare, retail banking, and human resource and information technology environments, analyzing actual experiences of organizations that are implementing agentic AI, says Joseph Kim, the company’s CEO.
In healthcare, patient Identity and verification accounts for 26 percent of consumer experience workflow volume. Other top patient-facing applications include patient access and appointment management (19 percent) and patient FAQs and knowledge sharing, mentioned by 13 percent.
“Together, those three front-door workflow types represent 57 percent of the published mix,” researchers noted. “The signal is straightforward: AI adoption in healthcare demand starts with access, verification and common patient questions.”
In terms of consumer engagement, Druid AI’s data shows that voice accounts for 54 percent of consumer interactions, while chat represents 46 percent of interactions. While voice uses still lead, “chat is already close enough that healthcare leaders should treat both as first-class service channels.” Voice reliance in healthcare is stronger than the other three industry segments in the study, “which makes sense in a service environment.”
Other AI use cases in healthcare lag behind – specifically, applications in clinical and case operations, contact center assistance, patient intake and data capture, and billing and insurance together comprise 24 percent of AI uses. Still, patient communication – especially in after-hour and weekend periods – may make the most sense for impactful investments in agentic applications.
The overall findings speak “directly to how patients prefer to engage with healthcare services, including populations often underserved by digital-first approaches,” researchers conclude. “The implications for health systems … are real, both for patient comfort and access.”
Oversight sought
Amid burgeoning interest in using AI within healthcare, there are also growing calls for oversight of appropriate uses for the technology. Several initiatives are underway, seeking to ensure the ethical use of AI within healthcare.
This week, the Federation of State Medical Boards (FSMB) announced that it was forming a workgroup on the regulation of artificial intelligence in the practice of medicine. The Washington-based organization said it’s forming the group with the intent of supporting state medical boards “as they navigate the rapid integration of (AI) into patient care and the growing range of regulatory sandbox frameworks being proposed and implemented across the country.”
FSMB says the workgroup is especially focused on AI tools “that perform clinical functions with little or no direct physician supervision,” building on its 2024 guidance on how best to use AI in clinical practice.
“As AI technologies continue to evolve and expand within healthcare, it is important that innovation and regulation move in tandem so that patients feel safe,” says Humayun J. Chaudhry, DO, FSMB’s president and CEO. “State medical boards remain the front line of patient protection, and this workgroup will help ensure that boards have the tools they need to oversee AI-enabled care, address new risks to safety and quality, and maintain accountability.”
The workgroup is expected to assess current regulations and evidence regarding AI, evaluate medical practice and business models, identify regulatory gaps and potential risks to patient safety, and then identify regulatory best practices for state medical boards.
Getting engagement
Healthcare organizations are increasingly realizing the depth of the transformation that AI will require, and they are seeking partnerships with outside companies or clinician champions to improve rollouts.
For example, Hartford (Conn.) HealthCare is partnering with Vitea, a Johns Creek, Ga.-based technology provider, to strengthen the governance and oversight of AI technologies across the integrated healthcare system.
The partnership is intended to do more than just throw AI into the stream of care. Rather, Hartford HealthCare is focusing on ensuring that technologies are implemented responsibly and with safeguards that bring reassurance to clinicians.
In this partnership, Hartford HealthCare will use Vitea’s platform “to support oversight of AI tools and applications across the organization,” a news release indicated.
"As we continue to innovate, we also recognize the importance of strong governance and thoughtful oversight,” says Joel Vengco, senior vice president and CIO at Hartford HealthCare. “This partnership supports our commitment to deploying AI in ways that are responsible, transparent and aligned with the high standards our patients and communities expect."
"Partnerships like this help create the structure, oversight and accountability necessary to advance innovation while maintaining trust, safety and clinical integrity," adds Barry Stein, vice president and chief clinical innovation officer at Hartford HealthCare and founder of the Center for AI Innovation in Healthcare.
In another example, Presbyterian Healthcare Services has been highlighted in an article by Healthcare Innovations where the New Mexico-based organization is exploring the use of AI with a leadership role being played by a nurse practitioner.
The organization plans to implement a precision-care AI platform to be used by its 200 primary care clinicians. The nurse practitioner’s role is to anticipate potential points of failure and to fill a niche as a trustworthy, experienced champion for AI. Executives at the system say her superpower is being able to understand how the technology will interface with front-line, patient-facing caregivers.
Successful implementation of new technology, such as AI, depends on whether it solves a commonly recognized problem, fits into existing workflows, and is championed by someone who’s been in the trenches with users and can readily demonstrate a range of benefits for using it, the author of the article notes.
Fred Bazzoli is the Editor in Chief of Health Data Management.