Enhancing Surgical Collaboration with AR

Developed by Rarelight Immersive, a leading European agency specializing in immersive technologies, the app utilizes real-time communication and shared experiences in the operating room.

Challenges

Rarelight Immersive was tasked with creating an AR application that would allow multiple surgeons to simultaneously view and interact with 3D surgical models in a shared augmented reality environment. Key challenges included:

• Seamless real-time synchronization of model manipulation, position, orientation, and scaling across multiple HoloLens 2 devices.
• Efficient management of user roles and control within the shared AR environment.
• Reliable voice communication between users.

Solution

Rarelight Immersive assembled a skilled team and leveraged strategic partnerships to address these challenges. Key elements of the solution included:

• Core Application Development: Rarelight Immersive led the overall application design, user interface development, and integration of various AR functionalities.
• Real-time Communication and Synchronization (Pixel Legend): Rarelight Immersive partnered with Pixel Legend, specialists in real-time communication and AR/VR development, to handle the crucial integration of the Photon SDK within the Unity engine for HoloLens 2.
• Pixel Legend’s responsibilities included:
• Implementing the Photon SDK to enable multi-user connectivity and data synchronization.
• Developing robust networking protocols to ensure seamless real-time interaction.
• Prototyping and optimizing the HoloLens 2 experience for optimal performance and user experience.

Results

The AR application successfully achieved the following:

• Enhanced collaboration among surgeons through real-time shared visualization and interaction.
• Improved surgical planning and precision thanks to shared 3D model manipulation.
• A streamlined and efficient workflow in the operating room.

This project, led by Rarelight Immersive with key technical contributions from Pixel Legend, showcases the transformative potential of AR in surgical settings. By providing a shared, interactive environment, the application empowers surgeons to collaborate more effectively, ultimately leading to improved patient outcomes.