This session presents an AI driven approach for validating facility management ticket data using an enterprise LLM integrated into the existing data platform. We compare a rule based baseline with a semantic LLM model that identifies inconsistencies, provides explanations, and improves data quality. The talk covers architecture insights, prompt design, evaluation results, and the roadmap toward automated ticket recategorization.

Target Audience: Data Scientist, Data Engineer, Product Owner, AI Strategist, Facility management
Prerequisites: no
Level: Basic

Extended Abstract:
In many organizations, facility management ticket data suffers from inconsistent or incorrect categorization, causing inefficiencies and reducing the reliability of operational reporting and analytics. This session presents a real world project with Heraeus GmbH and TOL GmbH that applies Large Language Models (LLMs) to improve the quality assurance of ticket data through semantic validation. Instead of depending solely on manual reviews or rigid rule sets, the solution introduces an AI driven workflow capable of interpreting free text descriptions and evaluating their consistency with selected categories.

The session guides participants through the complete project journey: analyzing user behavior, identifying recurring categorization issues, and designing a structured and governable data foundation suitable for AI applications. Initially, a rule based model was created to capture known error patterns, establishing a baseline for performance comparison. Building on this, an enterprise LLM was integrated into the data landscape and equipped with relevant metadata, category definitions, and domain-specific context to support accurate semantic assessment.

A major focus is the iterative development of prompt strategies. These iterations were essential to ensure that the LLM not only detects inconsistencies but also generates clear explanations and reliable confidence scores. Key insights include how structured prompts, context enrichment, and targeted metadata influence model output, as well as how to balance predictive accuracy with interpretability for business stakeholders.

Performance comparisons across project phases reveal substantial improvements over the rule based approach, particularly in identifying subtle or ambiguous inconsistencies in ticket descriptions. Beyond improved metrics, the LLM’s ability to produce understandable reasoning increased transparency and user trust.

The session concludes with a forward-looking perspective: enabling the system not only to detect misclassifications but also to recommend suitable categories, paving the way for automated ticket routing and further operational efficiency. Attendees will gain practical insights into designing enterprise-ready LLM solutions, preparing data for AI, and building scalable, explainable workflows that deliver real business value.