Agent Workflows – NVIDIA Livestream
Generic harness and flow are good, but it takes more to get to production.
Lesson
Agents can only do as much as their context; visualizing and selective maintenance are key.
The lesson in full: Agents get better all the time, and so do the harnesses (and their UIs/apps), but this just raises the bar for context, especially spatial and vision data. The same applies to the workflow and the agent team: too wide and it dilutes the work (too many chefs in the kitchen); too few and distraction and context rot occur right away. And without a form of dreaming/crystallization of context/memory, one can’t go far.
What it is
How the agents work ‘under the hood’, shown on the NVIDIA Omniverse livestream ‘Building AI Agent Workflows to Automate Industrial Operations’ together with 5 video examples from my own experiments (eg. lighting a plant with only the fixtures documented in its CAD, building a 3D model from drawings alone, etc.). This was co-presented with our VP Research and our spatial intelligence lead (who built the CAD MCP tool that the lighting runs use). The naive 1-or-few agents setup got distracted easily or bloated the context, especially with visual tasks. So now there are 4 agents with clear roles. Coordinator: manages the work between the others. Task runner: uses the tools, traverses the data graph via the Falkor SDK (the one created together with the CLI tool in 2022) and constructs the scene. Reviewer: runs on a different model, approves the runner’s findings or not, then kicks the process off again. Basically pair programming. Memory manager: consolidates the findings at the end into a clear context for later runs. The human is still in the loop (intent, answering the agent’s questions, validating the result). This was also shown together with my Agent Workbench, which replays real runs and visualizes the agents’ traces in 3 views (for transparency and validation), as the reporting on high-visual tasks is lackluster on current harnesses.
Tools and technologies
NVIDIA Omniverse · Multi-Agent Systems · MCP · Falkor SDK · Agent Workbench
Links
Lineage
Built on lessons from Multi-Agent Reasoning Engine, Reasoning Engine, Standalone AI Agents, Agent-First Ways of Working, Chat and Messaging Backbone, Deployment CLI and SDK, Agent-Operable Engines, Omniverse Remote Rendering, Drawings to 3D Scene.