Using augmented reality in the operating theatre could help surgeons to improve the outcome of reconstructive surgery for patients. The study was published by the research team at SpringerOpen and two Greek researchers participated, Liana Spyropoulou and Dimitri Amiras.

In a series of procedures carried out by a team at Imperial College London at St Mary’s Hospital, researchers have shown for the first time how surgeons can use Microsoft HoloLens headsets while operating on patients undergoing reconstructive lower limb surgery. The HoloLens is a self-contained computer headset that immerses the wearer in ‘mixed reality’, enabling them to interact with ‘holograms’ – computer-generated objects made visible through the visor. In the UK, headsets are currently only available for developers.

The Imperial team used the technology to overlay images of CT scans – including the position of bones and key blood vessels – onto each patient’s leg, in effect enabling the surgeon to ‘see through’ the limb during surgery.According to the team trialling the technology, the approach can help surgeons locate and reconnect key blood vessels during reconstructive surgery, which could improve outcomes for patients.

In the procedures used to trial the technology, five patients requiring reconstructive surgery on their legs underwent CT scans to map the structure of the limb, including the position of bones and the location and course of blood vessels.

Images from the scans were then segmented into bone, muscle, fatty tissue and blood vessels by Dr Dimitri Amiras, a consultant radiologist at Imperial College Healthcare NHS Trust (ICHNT), and loaded into intermediary software to create 3D models of the leg. These models were then fed into specially designed software that renders the images for the HoloLens headset, which in turn overlays the model onto what the surgeon can see in the operating theatre.

Dr Amiras said: “After seeing some videos of the HoloLens I contacted Dr Pratt to see if we could use this technology to bring the information from the CT directly to the operating theatre. St Mary’s Hospital is a major trauma centre, giving us the opportunity to try and improve the pre-operative planning for reconstructive flaps. Over time, the scanning protocol has been optimised to give excellent images of the anatomy, however, at first we had to rely on rough measurements of anatomical landmarks taken from 3D CT reconstructions to guide surgery. “Now, using the HoloLens, we can identify where the blood vessels are in 3D space and use virtual 3D arrows to guide the surgeon. Currently, data preparation is a time consuming process, but in the future much of this could be automated, with the consultant radiologist checking the accuracy of the model against the original scan. “I think this is a great example of what can be achieved in an Academic Health Science Centre.”

However, the researchers are confident that, once refined, the approach could be applied to other areas of reconstructive surgery requiring tissue flaps, such as breast reconstruction following mastectomy. The next steps include trialling the technology in a larger set of patients, with procedures carried out by teams at multiple centres.