Imagine being told by a neurosurgeon that the area of interest in your brain is “around about here” as he or she points to a model of a skull. You may trust the surgeon but if he or she could show you the entry route, and indeed the cause of the problem, it may reassure you more.
If he or she could explain why the operation will begin at this part of the skull and why the incision will be at a particular angle, you might feel more at ease. Using a virtual reality (VR) headset allows this exchange and has been found to reassure patients, and mean they enter the operating theatre more relaxed.
Using 3D medical imaging, patients can ‘tour’ a virtual model of their own brain to understand the area at the centre of the procedure. The surgeon can point out some of the obstacles that the surgical team has to be aware of. A VR ‘run through’ before the operation has been found to help patients, particularly younger ones who may not understand the medical terms but are at ease with VR through gaming.
Take a seat
In other areas of surgery, VR is being used by surgeons to plan procedures. Stanford Medical Center, the school of medicine at Stanford University, opened its Neurological Simulation Lab in 2017, using a VR system by Surgical Theater. The software creates a 3D model of the brain, compiled using MRIs (magnetic resonance images), CT (computed tomography) scans and angiograms (where dye is injected to show the paths of blood vessels).
Students sit in comfortable, cinema-style reclining chairs, facing three large screens. Wearing VR headsets, an avatar leads them through the areas of the brain, pointing out any deformities in the bone which might make entry difficult or any changes in tissue which might indicate a tumour.
The 3D model shows the precise location of a tumour or aneurysm. The 360° view means the surgeon can plan the most effective route for the surgery and improves accuracy in surgery. For example, it can make the surgeon aware of an artery close to a tumour, which cannot be seen from an x-ray. VR ensures there are no unplanned surprises during the procedure.
“We can plan out how we can approach a tumour and avoid critical areas like the motor cortex or the sensory areas,” said Gary Steinberg, MD, PhD, a neurosurgeon and Chair of Neurosurgery at Stanford University.
During the operation, surgeons can refer to a 3D rendering which provides more detail than is possible with video feeds, said Steinberg.
Educational tools
Continuing research means that even experienced surgeons have to learn new procedures. For each ‘new’ technique, it is estimated that at least 25 operations need to be completed for the surgeon to perform at a basic level of safety - and 75 to 80 for optimal proficiency. To overcome the problem of finding this number of ‘practice patients’, many companies offer immersive technology for surgeon training.
One is Osso VR, a VR surgical training platform, based in California. Co-founder Justin Barrad trained as an orthopaedic surgeon but originally wanted to be a gamer. The immersive training tool is used by medical students, experienced surgeons learning a new technique, as well as sales reps, demonstrating medical devices. Proprietary simulation software allows users to ‘run through’ a procedure.
Studies show that using VR improved surgical performance in terms of knowledge of instruments, the flow of operation, forward planning and knowledge of a specific procedure.
London-based Fundamental Surgery, offers a multi-user platform to allow students to collaborate in a virtual operating theatre. Users can access FundamentalVR remotely from around the world, allowing specialist consultants to host lectures and train multiple students.
It has recently introduced HomeVR, with educational simulations accredited by medical bodies and using standalone VR headsets, such as Oculus Quest and HTC Vive Focus Plus. Both simulation platforms use HapticVR, to bring the sense of touch to the immersive operating experience. The software is hardware-agnostic and can mimic the physical cues, the medical tools and tissue variation that would be experienced in the operating theatre, with haptic technologies providing tactile vibration and force feedback. Data tracking provides analysis from hundreds of data points. It measures and records surgical gaze, respect for tissue and efficiency of movement, displayed as real-time feedback on a single dashboard.
In the operating theatre, internal views of a patient via video feeds provide valuable insight throughout a procedure. A common complaint, however, is that the surgeon has to turn his or her head to look at a separate screen which is often placed at a 90° angle to the operating table.
To avoid neck craning, a company in Seattle, Proprio Vision, says its light field imaging system simplifies complex information to create interactive 3D visualisations for surgeons to use in real time.
The system uses a light field camera array to capture all the light radiating through the space between the surgeon and the patient on the operating table and Nvidia Quadro RTX 6000 GPUs for imaging processing. The company found that using GPUs accelerates the 3D point cloud triangulation for image between six to 10 times faster than using a CPU. This speeds up the surface reconstruction to develop low latency 3D light fielding rendering.
The system captures the live view and fuses it with pre-operative image, using multi-modal image rendering and registration, image processing and correction, 3D geometry reconstruction and CT segmentation. Using a VR headset, the surgeon can examine at microscopic levels and navigate freely, says the company. The high-definition volumetric representation of the surgical field can be magnified and re-focused and frees the surgeon from surgical microscopes or loupes.
Advances in image processing, namely high bandwidth processing with reduced latencies brings VR opportunities for remote guidance, as someone wearing a headset can experience a medical procedure remotely to receive tuition and guidance. It can introduce remote training for routine operations and encourage collaboration as specialists can examine the same medical scenario to contribute their specialist knowledge without having to take extended periods away from their main duties.
