
The solution cut Tier 1 ticket volume and reduced reliance on vendors, lowering support costs.
Meet our client
Client:
Industry:
Market:
Technology:
In a Nutshell
Client’s Challenge
A leading U.S. telecom provider was spending heavily on third-party vendors to handle Tier 1 support tickets — the majority of which stemmed from simple, repetitive issues like connectivity resets. With rising volumes, scattered documentation, and no automation in place, the process was slow, costly, and frustrating for customers.
Our Solution
We built a structured LLM-based support framework that automates Tier 1 troubleshooting using decision trees, intelligent classification, and guided user flows. By orchestrating open-source LLMs (Mistral) with internal knowledge bases and observability layers, we created a scalable agentic system that resolves issues in real time — no call center needed.
Client’s Benefits
The solution cut Tier 1 ticket volume and reduced reliance on vendors, lowering support costs. Customers resolved issues faster through self-service, improving satisfaction. The system created a scalable base for automating more complex tasks.
A Deep Dive
1. Overview
The client initiated a strategic AI program to streamline its customer support processes, starting with automating Tier 1 troubleshooting. They engaged deepsense.ai to fast-track this initiative using a structured, playbook-driven support system powered by open-source building blocks (ragbits), and internally deployed LLMs (Mistral). Our multi-phase approach delivered measurable cost savings, accelerated ticket resolution, and built a roadmap toward full agentic AI integration across service tiers.
2. Client
A large U.S.-based telecommunications provider serving residential and business customers with internet and digital services.
Key Stats:
- Operates across multiple states
- Handles thousands of support tickets monthly
- Historically outsourced Tier 1 support, creating high dependency and cost
Achievements:
- Transitioned from manual and outsourced ticket handling to automated triage
- Pioneered use of open-source LLMs in support ops
3. Challenge
Business Challenge
The client was facing high costs for handling Tier 1 support, primarily due to their reliance on third-party service providers. Over time, they discovered that around 80% of all support tickets could be traced back to issues on the customer’s end or with third-party systems, making much of the outsourced effort inefficient. Additionally, the support process was slow, leading to delays in issue resolution and a negative impact on customer satisfaction.
Technology Challenge
The client had no existing automation in place for ticket triage or troubleshooting, which meant every issue had to be handled manually. Their support knowledge was scattered across various sources, including playbooks, call audits, and user manuals, making it difficult to access relevant information quickly. Although they had internal LLMs available, these were underutilized due to the absence of a structured integration. Additionally, there was no system capable of managing multi-step support workflows, limiting their ability to scale or streamline the resolution process.
4. Solution
Our Approach
We designed and delivered a structured, playbook-driven support system powered by LLM orchestration. Our focus was on building a robust and extensible framework that could guide users through multi-step troubleshooting flows with high accuracy and low latency.
What We Delivered
We developed a structured support agent framework with the following key components:
- playbook-driven guided conversations, using decision trees to handle complex troubleshooting paths
- LLM-powered classification and clarification, turning vague user input into actionable steps
- automated resolution verification, with built-in logic to escalate unresolved cases
- conversational rephrasing of technical steps, making instructions more user-friendly
- guardrails and fallback handling, ensuring edge cases are escalated safely
Deliverables by Milestone
Phase 1: POC for Tier 1 Issues (4 weeks)
- Basic chatbot integrated with internal LLM (Mistral NeMo)
- Retrieval from indexed troubleshooting playbook
- Outcomes: user self-resolution or escalation to call center
Phase 2: Expanded Tier 1 Coverage (4 weeks)
- Playbook-based decision trees enabled systematic handling of advanced troubleshooting paths
- Additional data sources integrated
- Ability to create Tier 2 tickets when escalation is needed, based on resolution checks
Phase 3: Agentic Tier 2 Automation (6–10 weeks)
- Integration with external systems (client database, network management tools)
- Actionable agentic flows (e.g., remote resets)
- Safe deployment with human review before autonomy
- Implementation of an observability and analytics layer to monitor key support metrics
Technologies Used
- LLMs: Mistral NeMo (on-prem)
- Infra: ragbits modules (Flow Control, LLM orchestration, Prompts Management)
- Observability: Query + Event Tracking, Logging Dashboards
- UI: JavaScript-based Support Chat Widget
5. Process
- Requirements Gathering
Collaborated with the Customer Support team to assess historical tickets, tooling gaps, and infrastructure. - Design & Planning
Developed phased roadmap to maximize early value while planning long-term automation. - Implementation
Deployed a structured, playbook-driven support system with Mistra, integrated with existing documentation and ticketing tools.
- Validation & Evaluation
Used observability and logging for response evaluation and feedback gathering. - Knowledge Transfer
Provided clear documentation and integration support for internal teams.
6. Outcome
Quantitative Results
- Reduced Tier 1 ticket volume with self-service chat
- Cost savings through reduced third-party support dependence
- Cut response latency via token-by-token streaming and caching
Qualitative Results
- Higher customer satisfaction due to instant support
- Empowered internal AI team with structured, extensible tooling
- Built long-term foundation for Tier 2+ automation
Lessons Learned
- A phased, structured approach to LLM-driven support accelerates adoption while reducing operational and technical risk
- Observability, and continuous feedback loops are essential for reliable LLM deployment
- Self-service UI (even a simple floating button) greatly improves customer access
7. Summary
Final Thoughts
This case demonstrates how targeted, LLM-powered support systems can transform traditional customer support operations. By automating Tier 1 troubleshooting and laying the groundwork for deeper automation, we helped the client reduce operational costs, improve resolution times, and enhance the customer experience — all while building a scalable foundation for future AI-driven initiatives.