Internal-Facing GenAI Chatbot for Customer Care

TL;DR

Built an internal GenAI chatbot for customer care agents at Volvo Cars, transforming an 8-12 month vehicle delivery wait into a personalized learning experience. As Associate Product Manager, led ideation workshops, defined success metrics, and delivered an MVP using LangChain, Azure OpenAI, and RAG architecture. The solution achieved 94% agent satisfaction, 35% increase in product ARR, and saved 2,500+ hours annually across 30 agents in 6 markets. By Phong Trang - Product Manager - thanhphong.trang@gmail.com

Overview

As a Web Platform Product Manager working with the customer care team, I led the product development of an AI-powered virtual agent designed to empower Volvo Cars customer care agents with instant access to accurate, multi-market knowledge.

The Challenge: Customer care agents across 12+ markets struggled with fragmented information sources, outdated content, and time-consuming manual searches, leading to inconsistent customer experiences and extended handling times.

The Impact: Delivered a production-ready chatbot achieving 35% automated resolution rate (ARR), 94% agent satisfaction, and measurable improvements in First Contact Resolution (FCR) and Average Handle Time (AHT).

Key Metrics:

• 35% ARR (Automated Resolution Rate) within 90 days
• 94% Agent Satisfaction at alpha release
• 10pp FCR improvement on bot-eligible intents
• 20-30% AHT reduction target achieved
• 4.5/5 Bot CSAT maintained

Tech Stack

Core:

AI Models:

Problem Statement

Current Challenges

Existing Agent Workflow Pain Points:

• Memory Dependency: Agents rely heavily on personal experience and memory
• Fragmented Sources: Must navigate multiple systems (Salesforce, public websites, internal tools)
• Information Quality Issues: Content is often outdated, lacking, or hard to locate
• Time-Consuming: Significant time spent searching across systems
• Manual Collaboration: Dependent on colleague availability via Teams/face-to-face
• Personal Workarounds: Creating and maintaining individual notes/cheat sheets

Business Impact:

• Inconsistent customer experiences across agents and markets
• Extended handling times
• Lower first contact resolution rates
• Increased agent frustration and training burden
• Knowledge gaps during cross-border chat transfers

💡 DIBB Framework Alignment

• Data: Top 20 intents account for ~65% of volume; 40% of contacts after-hours; high average handle time on order-status and account questions
• Insight: Most top intents follow structured flows with clear outcomes; knowledge is scattered; escalations lack context
• Belief: Automating top intents and grounding responses in existing knowledge will lift ARR and CSAT while reducing costs
• Bet: Build a guardrailed, grounded chatbot with seamless handoff in 3–4 weeks to realize immediate savings and learn quickly

Ideation Phase

User Stories Mapping Workshop

<p><strong>🎯 Goal:</strong> To identify and align on the key user personas, their motivations, pain points, and the outcomes they expect from the chatbot — converting these insights into structured user stories that guide design and feature prioritization.</p>
<p><strong>👥 Participants:</strong> Product Manager, UX Designer/Researcher, Technical Lead/Solution Architect, Selected frontline agents or end users</p>
<p><strong>👨🏻‍🦱 My Role:</strong> I facilitated the session by framing the objectives, ensuring every story was actionable and traceable to user needs. I also captured dependencies, clarified business impact, and synthesized outputs into the initial backlog.</p>

<p><strong>📤 Workshop Outcome:</strong></p>
<img src="/assets/portfolio-img/chatbot/workshop-user-stories.webp" alt="User Stories Mapping" style="width: 100%;">

Core Feature Mapping Workshop

<p><strong>🎯 Goal:</strong> To translate user stories into tangible features, grouped and prioritized (High, Medium, Low) based on impact, feasibility, and alignment with MVP scope — creating a clear product definition for the first release.</p>
<p><strong>👥 Participants:</strong> Product Manager, Engineering Lead/Developers, UX Designer, Data Analyst, Customer Support or Operations Representative</p>
<p><strong>👨🏻‍🦱 My Role:</strong> I led the feature prioritization discussions, balancing user value with technical complexity and business constraints, maintaining focus on MVP deliverables, resolved scope debates, and documented final feature groupings used for roadmap planning.</p>

<p><strong>📤 Workshop Outcome:</strong></p>
<img src="/assets/portfolio-img/chatbot/workshop-core-feature.webp" alt="Core Feature Mapping" style="width: 100%;">

User Experience Mapping Workshop

🎯 Goal: To visualize the end-to-end chatbot experience — defining conversation flows, handoff points, and emotional journey touchpoints to ensure the interaction feels intuitive, human, and consistent across channels.

👥 Participants: Product Manager (facilitator), UX/UI Designer, UX Writer, Engineering Lead (for flow feasibility), Support/Service Team Representative

👨🏻‍🦱 My Role: Ensured alignment between design intent and product goals by helping translate insights from user stories and features into journey maps, validated feasibility with engineering, and documented key UX principles and success metrics.

📤 Workshop Outcome:

<img src="/assets/portfolio-img/chatbot/agent-hand-off.webp" alt="Flow Details 1" style="width: 100%;">

<img src="/assets/portfolio-img/chatbot/edge-case.webp" alt="Flow Details 2" style="width: 100%;">

Success Metrics Definition

💼 Business Value (Viability)

Operational and economic outcomes that justify scaling.

💬 User Value (Usefulness)

Signals that the tool is actually helping agents do their job (not just “liked”).

⚙️ Tech (Usability + Feasibility)

Quality, reliability, and safety gates required before scaling.

Proceed / Pivot / Pause Verdict

In the first validation phase (few user tests + early alpha telemetry), I used a gated decision mechanism to avoid “glowing feedback” masking reliability or safety issues. The rule was: we only proceed when user value is real and technical risk is bounded.

Decision gates

Gate 1 — Tech safety and reliability (must pass)

• PII Leakage Incidents: must remain 0 critical
• Hallucination Rate: must be at or below the audit threshold (target ≤ 2% of audited conversations)
• Escalation Accuracy: must meet the safety bar (target ≥ 90%)
• Uptime/Latency: must be stable enough for daily use (targets: uptime ≥ 99.9% business hours; response time median ≤ 1s, P95 ≤ 2s)

If Gate 1 fails, we Pause (data/model hardening and guardrails), regardless of positive sentiment.

Gate 2 — User value

• Bot CSAT stays at target (≥ 4.5/5 or ≥ 75% positive)
• Feedback participation is sufficient to learn (≥ 25% of conversations)
• Early qualitative evidence shows the bot reduces search friction and improves agent confidence without increasing escalations due to low trust

If Gate 2 fails but users still show a clear pain point, we Pivot (adjust scope, flows, or content strategy) rather than scale.

Gate 3 — Business viability

• ARR tracks toward the target (≥ 35% within 90 days)
• Ops outcomes trend in the right direction (FCR +10pp, AHT ↓ 20–30%, cost/contact ↓ 20–30% vs baseline)

If Gate 3 is not directionally positive but Gates 1–2 pass, we Proceed narrowly (tighten scope to the intents where viability is strongest), rather than broad rollout.

Verdict definitions

• Proceed: Gate 1 passes + Gate 2 passes, and Gate 3 is at least directionally positive (or can be made positive by narrowing to top intents).
• Pivot: Gate 1 passes, but Gate 2 or Gate 3 fails due to concept/UX/content mismatch → adjust scope, flows, or knowledge strategy; re-test in a new 1–2 week loop.
• Pause: Gate 1 fails (safety/reliability) or the data/model cannot support trustworthy answers → invest in grounding, evaluation, and guardrails before expanding capability.

Rapid Prototyping with AI

Accelerating Time-to-Feedback with Lovable.dev

To validate concepts before full development, I used Lovable - an AI-powered full-stack development platform that enables rapid prototyping of web applications.

<blockquote>
  <p><strong>💡 Benefits of this Approach:</strong></p>
  <ul>
    <li><strong>Speed to Feedback:</strong> Prototype in 1 day vs. 2-4 weeks traditional development</li>
    <li><strong>Cost Efficiency:</strong> Validate concepts before committing full engineering resources</li>
    <li><strong>Real Interactions:</strong> Stakeholders can click, type, and experience features rather than reviewing static mockups</li>
    <li><strong>Iterative Refinement:</strong> Quickly incorporate feedback and re-test within same sessions</li>
    <li><strong>Technical Validation:</strong> Engineering team can assess architecture and feasibility with real code</li>
  </ul>
</blockquote>

<p><strong>Quick Interactive Prototype (3 Intents):</strong></p>
<img src="/assets/portfolio-img/chatbot/lovable-demo.webp" alt="Lovable Prototype" style="width: 100%;">

My Validation Approach:

1. Product Definition → Created comprehensive product requirements document
2. AI-Powered Prototyping → Used Lovable to translate requirements into working prototype within hours
3. Stakeholder Validation → Brought functional prototype to key stakeholders:
   • User Testing: Customer Care agents tested chatbot interface for usability
   • Business Leadership: Demonstrated working dashboard for strategic alignment
   • Design Team: Validated look & feel and Volvo brand consistency
   • Development Team: Showcased feature functionality and technical feasibility

Technical Architecture

Data Ingestion Pipeline

Key Capabilities:

• Recursive Scraping Solution: Asyncio-based for performance
• Article De-duplication: Handles complex URL variants and parameters
• CDN Bot Detection Bypass: Custom headers and request throttling
• Media Extraction: Images and video scraping with JS dynamic DOM rendering (scroll-based)

Example Challenge: Support articles with complex URL structures requiring de-duplication logic:

- /support/car/xc60/article/8d7b79b0ad7b6a78c0a80151311fedcd
- /support/car/xc60/article/f1df148bc7f01a62c0a81f6f0b6b960e_c917de1ac7f46b81_8d7b79b0ad7b6a78c0a80151311fedcd

LLM Logic Architecture

<h4>Core Components</h4>
<ul>
  <li><strong>Basic RAG Flow:</strong> Retrieval-Augmented Generation with reranking</li>
  <li><strong>Conversational History:</strong> Context-aware rephrasing</li>
  <li><strong>Routing Logic:</strong>
    <ul>
      <li>Semantic routing for intent classification</li>
      <li>LLM routing with structured output</li>
    </ul>
  </li>
  <li><strong>Comparison Handling:</strong> Specialized flow for vehicle comparisons</li>
  <li><strong>Agent Handoff:</strong> Optional Langgraph integration for complex workflows</li>
</ul>

<img src="/assets/portfolio-img/chatbot/techneed.webp" alt="Semantic Routing Architecture" style="width: 100%;">

Integration Points

<ul>
  <li><strong>CRM/Ticketing:</strong> Zendesk/Freshdesk/Salesforce Service (create/update tickets, retrieve status)</li>
  <li><strong>Knowledge Base/CMS:</strong> Help Center articles, FAQs, and policy docs via API or sitemap</li>
  <li><strong>Order/Account Systems:</strong> Read-only endpoints for order and ticket status; secure links for self-service</li>
</ul>

<img src="/assets/portfolio-img/chatbot/integration-point.webp" alt="Integration Architecture" style="width: 100%;">

Scalability & Performance:

• Load Expectations: MVP 50–100 concurrent chats; architected to scale 10x without changes
• Latency Targets: ≤1s median, ≤2s P95 per bot response
• Resilience: Timeouts, graceful fallbacks, circuit breakers, exponential backoff

Solution: The MVP

Final Product Look & Feel

<img src="/assets/portfolio-img/chatbot/upskill.webp" alt="Chat Interface 1" style="width: 100%;">

<img src="/assets/portfolio-img/chatbot/realtime-assistance.webp" alt="Chat Interface 2" style="width: 100%;">

Project Scope

Timeline: 5–10 weeks from kickoff to pilot

Web Platform Team Composition:

• Product Manager - Product strategy, UX writing, DIBB cadence
• Full-Stack Engineer - Widget, admin console, integrations
• Conversational/ML Engineer - NLU, grounding, guardrails
• Designer (part-time) - Chat UI, admin dashboard
• QA/DevOps (part-time) - Test plans, observability

Rollout Strategy

Phased Approach:

1. Internal Testing: Multiple structured rounds to ensure system functionality, integration quality, and agent readiness
2. Alpha Release: Early access for targeted agent segment to assess UX, gather feedback, and surface blind spots
3. Market Expansion:
   • Phase 1: Northern Europe deployment
   • Phase 2: Additional European markets
   • Planned: U.S. and APAC following global system migrations

Results & Impact

Early Performance Metrics

Adoption Exceeded Expectations:

• 94% Agent Satisfaction at end of alpha release
• 35% ARR achieved within initial markets
• Significant FCR Improvement: First contact resolution rates increased measurably
• AHT Reduction: Average handling times decreased across all tested markets
• Response Quality: Agent feedback validated knowledge sources and search capabilities

Qualitative Impact:

• Agents reported quicker access to accurate support materials
• Reduced frustration from searching across fragmented systems
• Increased confidence in handling complex multi-market queries
• Positive sentiment toward AI-assisted workflows

Next Steps

Building on these results:

• Continued Market Expansion: Aligned with technical readiness
• Capability Enhancements:
  • Proactive suggestions
  • Onboarding support
  • Sales enablement features
• Data-Driven Iteration: Leveraging growing dataset and agent feedback

Key Learnings

1. Success is Driven by Agent Trust, Not Just Technology

Even with sophisticated AI, initial agent skepticism was real—trust dropped sharply when the bot’s first few answers weren’t accurate.

Takeaway: Invest up front in content quality, ground-truthing, and onboarding experiences. Early wins are powerful levers to drive engagement.

2. MVP Scope Discipline Was Essential, But Harder Than Expected

Many stakeholders pushed for more markets, more vehicle coverage, or more “automation.” Without constant alignment to the MVP’s narrow goal, rollout risked delays, confusion, and diluted results.

Takeaway: Clear and early-defined scope guardrails kept the team learning fast.

3. Analytics That Link Usage to Operational KPIs Made the Difference

Tracking bot adoption, correlating usage with average handle time (AHT) and first contact resolution (FCR), and segmenting by agent and market provided proof (not just narrative) for business leaders.

4. Integration Was the Main Source of Risk and Bottlenecks

The technical lift was less about front-end LLM/AI work, and more about integration pains (e.g., Salesforce, disparate knowledge sources, language packs, access control, and legacy systems).

Takeaway: Project momentum depended on quick, pragmatic resolutions when data or integration blockers surfaced.

5. Operational Realities Constrained the “Ideal” Experience

Not every market was ready as back-end knowledge synchronization and VPN access issues meant some rollouts paused.

Takeaway: Pushing back on universal promises in favor of pragmatic, phased approaches was necessary.

6. Privacy and Compliance Required Constant Attention

Even in an internal tool, questions about PII surfaced early and often.

Takeaway: Having clear guardrails and fast incident resolution built trust with corporate stakeholders.