How augmented reality will make surgery safer – the VOSTARS project

The VOSTARS (‘Video Optical See-Through Augmented Reality surgical System’) medical visor is a head-mounted display (HMD) system that is capable of superimposing the patient’s imaging diagnostics in perfect 3D unison with their anatomy.

The visor also presents a patient’s medical data such as heart rate, body temperature, blood pressure, and breathing rates, conveniently into the surgeon’s field of vision, in a drive to increase accuracy by focusing on the operation and reduce time by never having to look away.

Although Augmented Reality for surgical procedures has been talked about in academic and industrial research since the 1990s, a tool joining a surgeon’s natural perception with patient data has not yet been widely implemented.

The latest developments of photonics components, like the small, high-luminous micro display and the LED optical waveguide, have been crucial to making what was science fiction into reality.

Project coordinator Dr Vincenzo Ferrari, biomedical engineering researcher at the Department of Information Engineering, at the University of Pisa, explains:

“With this state-of-the-art, highly ergonomic visor, we intend to provide all the information required to improve surgery. The primary goal is to reduce not just surgery times, but also the time spent under anaesthetic and the cost involved in any operation.”

“For the patient, this means a saving of 20 minutes every 3 hours of surgery and the guarantee of an extremely accurate intervention,” said Dr Ferrari.

The VOSTARS system works by capturing what the surgeon sees from a head-mounted camera. The system then ‘merges’ this real-time footage of reality with the patient’s medical images, from CT, MRI, or 3DUS scans.

The central processor, using the most advanced registration techniques available for surgical navigation, then presents a real-time hybrid image on the visor ‘dashboard’ for the surgeon.

“Rather than having information on separate screens, all important patient data, like the surgical target in the anatomy, anaesthetic info, breathing and heart rates for example, are integrated instantly into the visor.

 VOSTARS is an Innovation Action project, receiving funding under H2020 in the framework of the contractual Public Private Partnership on Photonics.

Eleven European partners from Italy, France, Germany and United Kingdom are collaborating to achieve its goals.