System enables telementoring on battlefield with AR technology
An augmented reality tool aims to help inexperienced physicians in war zones perform complicated medical procedures guided remotely by mentors.
Developed by Purdue University researchers, the approach involves a transparent headset screen display that enables mentees to view the patient in front of them, as well as receive real-time on-screen feedback from mentors located in separate locations using video monitors.
In addition, the system leverages computer vision algorithms to track and align the virtual notes and marks from the mentor with the surgical region in front of the mentee.
“The most critical challenge is to provide surgical expertise into the battlefield when it is most required,” says Juan Wachs, James A. and Sharon M. Tompkins Rising Star Associate Professor of Industrial Engineering at Purdue, who led the research team. “Even without having highly experienced medical leaders physically co-located in the field, with this technology, we can help minimize the number of casualties while maximizing treatment at the point of injury.”
An augmented reality headset, worn by the mentee in the field, is designed to replace current telestrator technology that uses a separate video screen and freehand sketches to provide feedback.
“There is an unmet need for technology that connects healthcare mentees in rural areas with experienced mentors,” says Edgar Rojas Muñoz, a doctoral student in industrial engineering, who worked on the project. “The current use of a telestrator in these situations is inefficient because they require the mentee to focus on a separate screen, fail to show upcoming steps and give the mentor an incomplete picture of the ongoing procedure.”
Supported by the Department of Defense, the technology has completed a round of clinical evaluation, and in the next few months it will be tested at a Navy base in Virginia, where mentees and mentors will experiment with a simulated battlefield.