Talk to My Document
Embedding-based document search in Kognitwin, co-engineered with Microsoft.
Lesson
Knowing about AI models is not the same as using them to solve real-world problems.
The lesson in full: Beyond the obvious insight on embeddings and vector search, the biggest lesson: knowledge of AI/ML and applied application in real-world usage are 2 very different things. Take the 3D world for example, the ones that create the 3D engines (Unity, Unreal) and the ones that use them (game developers) are 2 distinct groups. I still remember the GPT-3 document included a line ‘turn-based text structure can exhibit assistant behaviour’, and that was 8 months before ChatGPT was released. Knowing that GPT-3 can complete text is not the same as building the application layer for ChatGPT.
What it is
Documents of all types are one of the biggest data sources in Kognitwin with 100K to 3 million documents in each facility, depending on size. Before this project, document search was keyword-based, focusing on metadata (title, linked equipment, tags) not the content itself. This project enabled search on document content and used that as grounding for AI copilots/agents. Key technical challenges: JSON-izing unstructured documents (PDF, Word, Excel, Images), creating chunks on structured JSON document trees for embedding, retrieving and ranking relevant documents for grounding, and indexing millions of documents within reasonable time and cost. This project was also co-engineered with a team of 15+ developers and data scientists from Microsoft.
Results
- Indexed 600K documents for 1 facility in 1 week.
- 8 million+ vectors during POC.
- Issue tracking and code via GitHub across KDI and the Microsoft team.
Tools and technologies
Vector Embeddings · Qdrant · Chunking · Multimodal · OCR · Azure Cognitive Services
Lineage
Built on lessons from Catbot AI Test Bed, Reasoning Engine.
Lessons carried into Drawings to 3D Scene, Multi-Agent Reasoning Engine.