AI, racial bias and the imperative for equitable technology
Nurses can play a critical role in ensuring that artificial intelligence tools don’t embed bias that influences treatment decisions.
Artificial intelligence has moved from a futuristic concept into a working part of everyday clinical practice, quietly shaping decisions in triage, deterioration alerts and diagnostic support.
However, the same tools that promise faster, more consistent care can just as easily inherit and magnify the racial inequities that have long shaped the healthcare system.
Because nurses spend more time at the bedside than any other member of the care team, they are uniquely positioned to notice when a “smart” system produces an outcome that does not match what they are observing in the patient in front of them. Understanding how bias enters these systems, and what nurses can do about it, has become an essential piece of professional competence rather than an optional area of interest.
How bias becomes embedded in AI
Artificial intelligence systems do not evaluate patients the way a nurse does; they identify statistical patterns buried in historical records and treat those patterns as reliable guides for future decisions.
When the data used to train a model reflects years of unequal testing, unequal spending or unequal access to specialists, the algorithm absorbs that inequality as though it were simply how healthcare works. A tool built this way can look objective on paper while still steering resources away from the same patients who were historically underserved.
This is the central danger nurses must recognize – a biased algorithm rarely announces itself, it simply produces recommendations that feel neutral because they are wrapped in numbers rather than words.
Real consequences at the bedside
One of the clearest examples of embedded bias involves a device far more familiar to nurses than any algorithm – the pulse oximeter. A 2020 study by Sjoding and others found that pulse oximeters were much more likely to miss dangerously low blood oxygen levels in Black patients than in White patients because the devices tend to overestimate oxygen saturation in people with darker skin tones.
This gap matters enormously in fast-moving situations such as respiratory decline, where a falsely reassuring reading can delay supplemental oxygen or escalation of care.
Because oximetry readings feed directly into many hospital early-warning systems, an inaccuracy at the bedside does not stay contained to a single monitor; it travels forward into every downstream tool that relies on that number, including the deterioration algorithms on which nurses on every shift depend.
A widely cited 2019 investigation by Obermeyer and others illustrates how bias also can hide inside algorithms designed specifically to identify patients who need extra support.
The researchers examined a commercial care management tool used across the country and found that it relied on prior healthcare spending as a stand-in for the severity of a patient’s illness. Because less money had historically been spent treating Black patients with comparable levels of illness, the algorithm consistently scored those patients as healthier than equally sick. White patients, meaning fewer of them, were flagged for additional case management resources.
When the researchers corrected the model to use a more direct measure of illness rather than cost, the share of Black patients identified for extra care nearly tripled, demonstrating that the bias was not an unavoidable byproduct of the data but a fixable design choice.
The nurse's role in equitable technology
Nurses are not passive bystanders to these problems. Professional guidance already asks them to take an active role in addressing bias wherever it appears. The American Nurses Association in 2019 called on nurses to recognize how bias and discrimination shape care delivery and to work toward inclusive, equitable strategies at both the bedside and the policy level.
Building on that expectation, Cary and others in 2025 introduced BE FAIR, or the Bias Elimination for Fair and Responsible Artificial Intelligence in Healthcare, framework. It asks nurses to participate in AI governance committees, question outputs that create unexplained disparities in care and advocate for training data that genuinely represents the communities being served.
Taken together, these sources make clear that evaluating an algorithm's fairness is no longer solely the responsibility of a data science department. It falls squarely within a nurse's ethical scope of practice.
The path forward
Nurses don’t need to become programmers or statisticians to protect their patients. However, nurses need to ask the same questions of an algorithm that they would ask of any new medication or device – who was it tested on, how well does it perform across different groups of patients, and what happens when it is wrong.
The studies by Sjoding and Obermeyer both demonstrate that after bias is identified, it often can be corrected without sacrificing a tool's overall usefulness, which means staying silent is never the safer option.
As artificial intelligence becomes further embedded in triage, monitoring and care-coordination systems, nurses who insist on transparency and equity at every stage of a tool's use will remain one of the most reliable safeguards their patients have.
Sonya Curtis BSN, RN, RN-BC, SEP, is senior director of clinical informatics for Aledade.