Spatial Inspection Video
Video with a seek bar in 3D, powered by OCR. TBs of such.
Lesson
You can’t connect the dots looking forward, trust they will somehow connect in your future (after Steve Jobs, Stanford 2005).
The lesson in full: I would not have guessed that the dots of handling big video data, working with YOLO models and building 3D experiences would connect in this way, but they did. Downside: while it worked, the road to production and scale was tough as the resources needed to process and serve such videos at scale were non-trivial.
What it is
While being evaluated for the EFA (Enterprise Framework Agreement) for digital twin, the POC team received an open challenge: ‘We have TBs of inspection footage, underwater, taken by ROVs. What can you do with it?’. Using the intersection of my experience in video processing, 3D visualization and AI/ML, I built a pipeline to extract geospatial metadata that was burned into video frames via OCR, reconstruct the 3D path of the ROV, visualize the video in the 3D coordinates of the facility and synchronize the video playback with the 3D position of an ‘ROV sphere’ in the 3D model. After the POC phase, this feature was further developed to support practical features like raw telemetry data, points of interest, and multiple cameras per ROV.
Results
- Deployed to offshore facilities in Trinidad & Tobago, Norway.
- 40+ TBs of video processed.
- Contributed to winning Enterprise Framework Agreement.
Tools and technologies
OCR · YOLO · FFmpeg · Video Processing
Lineage
Built on lessons from Digital Twin Platform, 360 Video Production, Auto-Navigated 360 Video, VR Video Player.
Lessons carried into Point Cloud Streaming.