World's Fair 2026

Semantic Blindness: 500,000 Sensors Confused an LLM - Raahul Singh & Vanč Levstik, Phaidra

Raahul Singh , Vanč Levstik

Overview

This talk addresses the challenge of "semantic blindness" in Large Language Models (LLMs) when dealing with vast, complex, and inconsistently named datasets, such as sensor names in large-scale industrial environments. The core thesis is that LLMs struggle with sheer scale and naming ambiguity, leading to errors and hallucinations. The proposed solution involves a hybrid approach that leverages the LLM for planning and decision-making while offloading structured data processing, retrieval, and set operations to deterministic code.

Who should watch

• AI Engineers working with large-scale data.
• Product Managers building AI-native systems.
• Builders facing challenges with LLM recall and accuracy at scale.
• Those dealing with inconsistent or ambiguous naming conventions in datasets.
• Engineers looking to improve the reliability and efficiency of LLM applications.

Key takeaways

• LLMs can suffer from semantic blindness when presented with hundreds of thousands of similar or inconsistently named items, leading to poor recall and hallucinations.
• Traditional approaches like RAG (Retrieval-Augmented Generation) and simple sharding of data for LLMs fail at scale due to context window limitations, semantic similarity issues, and LLM frequency penalties.
• A hierarchical structure inherent in many physical systems (e.g., data centers with halls, aisles, racks, GPUs) can be exploited. The depth of this hierarchy grows slowly, while the width grows rapidly, allowing for more manageable context.
• The solution involves using LLMs for planning and generating search patterns rather than direct data sifting. A deterministic resolver then executes these plans using set operations on pre-indexed data structures.
• This hybrid approach significantly improves accuracy, maintaining 100% correctness across tests, and drastically reduces token usage compared to naive LLM methods.
• The cost of querying remains flat regardless of system size, unlike exponential cost growth with naive LLM approaches.
• The talk advocates for a shift in AI-native system development: start with LLM-driven exploration (Software 3.0) and mature towards incorporating deterministic code (Software 1.0) for structured, rule-based tasks.
• *The real skill is knowing which work is the LM not best for.*

Notable quotes

Watch on YouTube →

Unofficial community note. Prefer the recording for nuance.