The Operating Theatre Journal - Flipbook - Page 26
Putting the power of immersive XR into practice
for enhanced surgical outcomes
John Dabill, Director, EMEA, HTC VIVE
One of the biggest leaps forward to support the Operating Theatre
has been the rapid evolution of technology over the last decade.
From ultra-precise imaging and visualisation, through to robotics,
and more. In the last few years, immersive technology in the form of
Virtual Reality (VR) and Mixed Reality (MR) has emerged as another
powerful tool, helping to improve outcomes in surgery. This looks set
to continue in the future, growing to be more versatile and powerful
as a support tool for surgical procedures.
VR headsets like the VIVE Focus 3 are now capable of displaying 3D
imagery in high-resolution 4K, with millimetre tracking accuracy.
This means fine movements can be tracked in incredibly detailed 3D
models, and it’s also easy to develop software which means different
surgical instruments can be represented by the controller, delivering a
highly accurate and immersive experience. The headsets are powerful
standalone devices which do not require any external hardware, and
can be worn comfortably for long periods, reducing the set-up costs
and resources required.
But what does this mean in practice?
There are programs all over the world leveraging the power of
immersive technology, including several VIVE partners.
In the UK, University College London’s (UCL) work with ARuVR
is already setting an impressive standard to follow. We all know
the challenges of training, fraught with the inherent risk of live
procedures, and there are also difficulties around inclusivity because
of distance and timing. UCL and ARuVR developed a program which
allowed for students to dive into accurate 3D immersive training
scenarios and learn complex surgical techniques, anywhere, any
time. Through virtual training, UCL was able to triple the number of
students able to view surgical procedures, while minimising patient
risk and saving time and resources. Development of skills was found
to be faster, with increased knowledge retention as well.
In the US, NYU Langone used immersive technology to help them
achieve the highest survival rates in the country for neurology and
neurosurgery procedures in 2022. They built highly accurate 3D
models of patients’ brains, and then rehearsed the surgery in VR –
enabling them to try different approaches before deciding on their
preferred routine. They would then practise repeatedly, being able to
replicate all of the physical movements required with high accuracy,
from moving their own body through to fine surgical instrument
movements. Arguably the proof is in the outcomes, and NYU Langone
used immersive technology to make a significant positive difference.
Elsewhere back in the UK, FundamentalVR has taken immersion
to new levels. Not content with creating ultra-realistic visuals and
physics, the team developed highly-accurate haptic feedback tools.
This enables a complete sensory experience, delivering realism
without any patients being put at risk. In fact, Fundamental Surgery
is so sophisticated that it is Continuing Professional Development
accredited with the Royal College of Surgeons. Not only is this
validation of the meticulous approach from FundamentalVR, but it’s
also a huge vote of confidence for immersive technology as a tool.
There are other organisations who are taking advantage of our eyetracking module, adding it to a headset so that it’s easy to see where
the trainee’s focus is. The trainer can see in real-time where the
trainee is looking, and provide corrections or feedback as needed, and
this process has even been automated by using AI to track a person’s
gaze.
In the not-too-distant future, remote surgery will be entirely possible.
There are already sophisticated robotics being used today with
precision, and combined with fast, high-bandwidth internet and
immersive VR headsets, fully remote surgery is within realistic reach.
This is especially relevant for emergency scenarios as well as highspecialised procedures where experts may be physically distant from
patients in need.
The next evolution of VR headsets has been to incorporate MR,
which allows for virtual overlays to be seen over real-world objects.
Devices like VIVE XR Elite have full colour RGB passthrough, giving
people the ability to wear a headset while still seeing their real-world
surroundings. There are already training scenarios in other fields who
use the virtual overlays to display additional information to the user,
whether that’s diagnostic reference images, or animations showing the
next steps. It’s an incredibly powerful and versatile new way to use
immersive technology. Related to that, it’s becoming easier and easier
to have real-world instruments represented in XR, with hardware like
VIVE Wrist Tracker, which can be physically attached to a real-world
object and then tracked accurately in XR.
We’re in an era where medical
professionals are under
constant strain, with everincreasing pressure against
limited resources and money.
Immersive technologies like
VR and MR can provide key
support and help to significantly
improve patient outcomes,
while also improving efficiency
and saving time and money.
John Dabill.
Director HTC EMEA, and part of the EMEA Leadership team.
Since joining HTC, John has been working with customers and partners
across the EMEA region to develop XR solutions and immersive
experiences that will define the next generation of computing, across
hardware, software, the metaverse and more.
Among his responsibilities, John leads the development partner
program in EMEA, to develop software and services to grow the
portfolio of end-to-end solutions for business customers and help
accelerate the adoption of XR solutions.
A long-time technology veteran, John has held senior leadership
positions across the tech sector over several years, most recently
working at BlackBerry and Bullitt Group before joining HTC VIVE.
26
THE OPERATING THEATRE JOURNAL
www.otjonline.com