Spatial Intelligence for Agents
AI agents that see, simulate and reason over an industrial facility; the work was featured in NVIDIA’s keynote.
Lesson
What if massive tech stacks and libraries are restructured for agents from the ground up? More space to focus on bringing value and collaboration.
The lesson in full: Libraries and SDKs are made to build things, but not everyone can or likes to consume them in the same way. What if working with agents is a fundamental prerequisite? When those ‘what ifs’ turn into reality, the problems shrink drastically, leaving space to focus on leveraging tools and tech to solve real problems, bring real value and justify the work.
What it is
A milestone of the Omniverse and agent work. At GTC Berlin on 21 October 2026, Falkor showcased AI agents that see, simulate and reason over industrial facilities, in videos shown in NVIDIA’s keynote and in sessions. The scene is SimReady OpenUSD derived from the facility’s engineering CAD, running in Falkor Surface on Azure, with every light from a fixture in the engineering data, found by semantic mining and physically simulated. An agent requests a view, identifies the equipment behind a pixel, changes the scene and looks again. Where no current CAD exists, agents powered by models such as GPT-6 Astra build an OpenUSD model from engineering drawings and anchor it to laser scans. Underneath all of this is a larger shift. NVIDIA now ships Omniverse’s core as standalone, embeddable libraries alongside the Kit application framework: ovstage for shared OpenUSD scene data, ovrtx for rendering, ovphysx for physics and ovstream for streaming. They are part of NVIDIA’s Agent Toolkit, for developers and AI agents alike, and most ship agent skills, with MCP servers alongside. The keynote framed CUDA-X the same way, as libraries that give agents “the tools of the trade”. Our Omniverse integration moved onto those libraries in 2026; on top I built the browser streamer and an MCP server over the render engine, so agents can set views, overlays and camera paths.
Results
- Featured in NVIDIA’s GTC Berlin 2026 keynote.
Tools and technologies
NVIDIA Omniverse · OpenUSD · Azure · Semantic Mining · GPT-6 Astra · Point Cloud
Links
Lineage
Built on lessons from Omniverse Remote Rendering, Agent-Operable Engines, Drawings to 3D Scene, Multi-Agent Reasoning Engine, UX Reboot and Design System.