AI Product Blueprint · Case Study

ResolveIQ

An intelligent decision system that classifies support tickets, retrieves context, drafts replies, and escalates to humans when it matters.

Support teams lose 40–50% of time on repetitive questions. Historical data has the answers — it's just never structured enough to use. ResolveIQ turns messy support history into a governed AI workflow with measurable speed and accuracy.

Download Full PRD Try the Decision Engine

Historical Tickets

AI Components

0-Path

Decision Engine

$0M ARR

Opportunity

Live Decision Flow

ACTIVE

Ticket arrives

Inbound

Classify + Retrieve

Draft response

Route or escalate

Decision

Human review

Loop

The Real Pain

Support teams face a data & decision problem

Repetitive tickets + inconsistent answers + gut-feel escalation = wasted time and bad CX.

Support teams receive 40–50% repetitive tickets — billing, password resets, status checks. Each consumes agent time and delays responses. The same issue gets five different answers. Escalation happens on gut feel — "this customer sounds angry" — not data.

500–1000 historical tickets already sit in Zendesk, Intercom, WhatsApp. They contain answers, patterns, and escalation signals. But they're unsearchable, uncategorized, and unused as training material.

The AI opportunity: turn this history into a decision system that classifies, retrieves similar cases, drafts replies, and escalates intelligently.

Without ResolveIQ

Response Time8 hours
First-contact resolution45%
Escalation accuracy70%
Cost per ticket$2.50

The Manual Process

1
Ticket arrives
→ ?
2
Agent searches for similar cases
→ Maybe finds one
3
Agent writes response from scratch
→ Takes 30 min
4
Agent guesses if escalation needed
→ 70% accuracy

The Architecture

Four AI components. One decision system.

ResolveIQ doesn't just chat. It classifies, retrieves, reasons, and routes.

Classification Engine

Claude 3.5 Sonnet · Structured Output

Classifies ticket into category, priority, sentiment, confidence. Forces deterministic JSON — no hallucination.

88%+ accuracy

Retrieval System

OpenAI Embeddings · pgvector

Searches historical resolved cases + KB. 'This issue was solved 12 times. Here's how.'

80%+ relevance

Response Generation

Claude 3.5 Sonnet · Constrained

Drafts customer reply from KB only. Never makes false promises. Agents approve 65%+ unchanged.

0% hallucination

Escalation Engine

Rule-Based Decision Logic

Decides: auto-resolve? Human review? Escalate? Based on confidence, category, sentiment, policy flags.

95%+ precision

Interactive · Three-Path Decision Engine

AI recommends. Humans control.

Tap a path below to see how the engine reasons about a real ticket.

Incoming Ticket

"How do I reset my password?"

ClassificationProductQuestion · Low risk

Confidence92%

routes to

Auto-Resolve Path15–20%

Send drafted response immediately

Trigger Criteria

Confidence > 85 + Low risk + No flags

Every human action logged → improves next month's rules

The Hard Part

Data strategy comes before AI

Week 1

Collect 500–1000 Historical Tickets

Export from Zendesk, Intercom, WhatsApp, email.

Week 2–3

Clean & Anonymize

Dedupe, mask PII, normalize 6–10 categories, label outcomes.

Week 4

Create Golden Test Set

50–100 tickets manually labeled by manager as ground truth.

Week 5

Prepare Knowledge Base

Compile approved documents, policies, FAQs in searchable format.

Why Data Comes First

Garbage in = garbage out. Most support teams have 80% of the data they need but don't realize it. Data prep takes 2–3 weeks; the AI pipeline takes 1 week. The bottleneck is data, not models.

Quality Gates

Can I find answers to common questions in the KB?
Can I find similar cases using keyword search?
Are categories sensible and quick to pick?
Is data anonymized (no customer PII)?
Is the golden test set diverse (billing, tech, complaints)?

Rigorous Evaluation

How we know it actually works

Golden Test Set Metrics

Category Accuracy

Compare AI to human labels on 50–100 tickets

≥ 88%

Sentiment Detection

Validate angry/calm detection vs ground truth

≥ 85%

Confidence Calibration

Bin by confidence; measure actual resolution

80+ resolves 75–85%

Response Quality

Manual review of 30 drafted responses

0% hallucination · 85%+ actionable

Escalation Precision

Manager spot-checks 50 escalations

95%+ correct

Live Testing Phases

Shadow Mode (Week 3–4)

Test · No Impact

Run AI on real tickets, don't send responses. Track: classification match ≥ 80%, escalations sensible ≥ 90%, drafts useful ≥ 70%.

Limited Automation (Week 5–8)

Controlled Rollout

Auto-resolve ProductQuestion only, confidence > 85, no flags. Monitor: follow-up < 5%, satisfaction parity > 95%.

Expansion (Week 9+)

Scale Slowly

One category per week. Never automate Billing, Complaints, P1/P2. Weekly: does performance hold?

Technical Architecture

Build-ready system design

Ticket Intake

Classification

Retrieval

Escalation Rules

Response Gen

Routing

Human Review

Audit & Learning

Component	Tool	Why
LLM Inference	Claude 3.5 Sonnet	Excellent structure + intent understanding
Embeddings	OpenAI text-embedding-3-small	Cost-efficient, handles Arabic
Vector DB	Supabase pgvector	Low ops, scales, auditable
Data	PostgreSQL	Reliable, queryable
Frontend	React Dashboard	Retool or custom
Integrations	Zapier / n8n	Email, CRM, WhatsApp triggers
Monitoring	Supabase logs + custom	Track all AI decisions

Monthly Cost

Claude API (~1k tickets)$2
Embeddings (~1k searches)$0.02
Supabase (pgvector)$50–100
n8n / Zapier$30–50

TOTAL~$100 / mo

Scales to ~$500/mo at 10k tickets/mo.

Business Impact

What measurably changed

Metric	Baseline	Target	Timeline	Status
Response Time	8 hours	2 hours	3 months	On track
First Contact Resolution	45%	70%	6 months	On track
Tickets / Agent / Day	12	18	3 months	Testing
Escalation Accuracy	70%	95%	2 months	Testing
Cost per Ticket	$2.50	$1.75	6 months	Design
Customer Satisfaction	3.8 / 5	4.4 / 5	4 months	Design

Revenue Impact

+$0M

ARR opportunity

Example: 5-agent team, $50k avg deal, 50% resolution-rate lift. Actual impact depends on volume, deal size, and team.

Risk Management

Risks & how we manage them

HIGH

AI Generates False Promise

Constrain prompts to KB only. Zero external knowledge. Target 0% hallucination. Audit first 50 responses.

MED

Wrong Escalation Decision

Test escalation rules on 200 historical tickets pre-rollout. Target 95% precision. Daily monitoring.

MED

Historical Data is Biased

Include resolved + escalated cases. Label what worked AND didn't. Golden set must be diverse.

MED

Agents Don't Trust AI

Start review-only (no auto-resolution). Show accuracy openly. Build trust gradually.

LOW

Knowledge Base Stale

Monthly KB review. Version every document. Alert team on policy changes.

LOW

Compliance Risk

No customer data to external APIs. Mask PII before processing. Maintain audit log.

Implementation Timeline

From concept to GA in 16 weeks

Week 4

MVP Ready

Week 8

Shadow Mode

Week 12

Limited Auto

Week 16

GA Launch

Month 4+

Expansion

What this case study shows

Beyond a chatbot: system thinking

Problem Discovery

Started with observed pain. Validated root cause (unstructured historical data). Defined opportunity. Not 'AI is cool, let's use it somewhere.'

Product Methodology

End-to-end workflow. Human-in-the-loop, not autonomous. Evaluation framework: golden set, offline testing, shadow mode. Not 'here's a chatbot.'

AI PM Thinking

Model selection justified (Claude for structure; rules for transparency). Evaluation rigor. Risk management: escalation precision > automation speed.

Who should care

Built for builders and backers

For Hiring Managers · AI PM

A PM who ships, not pitches.

Understands support ops, not just LLMs. Designed a decision system, not a chatbot. Has an evaluation framework, not just enthusiasm. Built human-in-the-loop first. Knows when to use rules vs ML.

Ideal role fit