Video Calls in XR
Since there’s chat, why not video call too?
Lesson
WebRTC is still hard, especially cross-region and network complexity.
The lesson in full: While WebRTC has been around for a while, building a robust, scalable video call system is still hard. The complexity of network conditions, NAT traversal, multi-region latency, etc. all add up to an unpredictable user experience. While there are TURN/STUN servers to help with some of these problems, going there requires major investment, which is not feasible for an innovation project, so we had to land somewhere in the middle.
What it is
A WebRTC-based video call system integrated with the same ‘Chat’ backbone of the Kognitwin platform. As we also deliver XR solutions on the web via the same application, this function also works in VR headsets like Quest 3. In other words, a desktop user can see their colleague moving around in 3D space (as they are in VR) and then call them while collaborating on a piece of equipment together, and vice versa.
Tools and technologies
WebRTC · WebSocket · Quest 3 · TURN/STUN
Lineage
Built on lessons from Chat and Messaging Backbone, Real-Time 2D/3D Collaboration, VR Video Player.
Lessons carried into Agent-Operable Engines, Omniverse Remote Rendering.