Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Support Ticket Triage & Routing

[YourCompany.com] · Customer Support Department · Prepared by FullSpec · [Today's Date]

This document outlines the complete test and QA strategy for the Support Ticket Triage & Routing automation. It covers unit testing of the Ticket Analyzer and Smart Router agents, integration testing with Zendesk, Slack, and HubSpot, edge case validation, and final end-to-end production acceptance. The FullSpec team will execute all build, integration, and test phases; your team will validate the final production run and approve go-live.

01Testing approach

The Support Ticket Triage automation is tested across three phases: unit validation of each AI agent in isolation, integration testing of all connected tools and APIs together in a staging environment, and end-to-end production testing on a live sample of real tickets with human review. Each phase has explicit pass criteria and rollback conditions.

Phase 1: Unit Testing

Estimated time: 3 business days

Scope

Test Ticket Analyzer and Smart Router agents individually with mock data, no live API calls.

Environment

Isolated test harness on FullSpec development server. No external tool connections.

Credential rule

No API credentials required; mock responses simulate Zendesk, HubSpot, and Gmail.

Pass condition

All test cases return expected output within defined tolerances. No crashes or timeout errors.

Who runs it

FullSpec QA engineer.

Phase 2: Integration & Edge Case Testing

Estimated time: 4 business days

Scope

Both agents connected to staging instances of Zendesk, Slack, HubSpot, and Gmail. Real API calls with safe test data.

Environment

FullSpec staging account with sandbox Zendesk organization, test Slack workspace, and HubSpot sandbox portal.

Credential rule

API tokens and OAuth credentials stored in FullSpec vault. Tokens scoped to staging resources only. No production data accessed.

Pass condition

Agents read and write correctly to all tools. Slack messages format properly. HubSpot records log without errors. All edge cases handled or flagged for manual review.

Who runs it

FullSpec integration engineer with support team observer.

Phase 3: End-to-End Production Test

Estimated time: 5 business days

Scope

Automation running on live Zendesk, Slack, and HubSpot with real incoming support tickets. First 50% sample, then 100% after approval.

Environment

Production Zendesk organization, live Slack workspace, production HubSpot instance.

Credential rule

Production API tokens issued by your team. Credentials stored securely in FullSpec vault with read-only audit logging enabled.

Pass condition

Routing accuracy is >= 94%. No customer-facing errors. Slack notifications arrive within 10 seconds. All tickets assigned and tagged correctly. Process owner approves final results.

Who runs it

FullSpec DevOps engineer with your support team lead observing live tickets.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1: Unit test cases

All Phase 1 tests use sandbox credentials and mock API responses. No real Zendesk, Slack, HubSpot, or Gmail data is accessed. Mock responses are stored in FullSpec test fixtures and regenerated if tool schemas change.

Ticket Analyzer Agent: Reads ticket body and extracts issue category, priority level, and customer sentiment. Tests validate category detection accuracy, priority assignment logic, and sentiment scoring across common support scenarios.

Test ID

Scenario

Input

Expected Output

Pass Criteria

T01

Billing issue with anger signal

Ticket body: 'I was charged twice for my monthly plan. This is unacceptable. I need a refund NOW.'

category: billing, priority: high, sentiment: -0.85 (angry)

Category matches billing exactly; priority is high; sentiment score below -0.7

T02

Technical issue, normal urgency

Ticket body: 'The export feature is not working. Can someone help troubleshoot? It keeps timing out after 30 seconds.'

category: technical, priority: normal, sentiment: 0.1 (neutral)

Category is technical; priority is normal; sentiment between -0.3 and +0.3

T03

Feature request with low priority

Ticket body: 'Great product so far! Would love to see dark mode in the next release.'

category: feature-request, priority: low, sentiment: +0.65 (positive)

Category is feature-request; priority is low; sentiment above +0.5

T04

Account access issue, high urgency

Ticket body: 'Cannot log in to my account. Getting error code 403. This is a production blocker for our team.'

category: account-access, priority: high, sentiment: -0.4 (frustrated)

Category is account-access; priority is high; sentiment below -0.2

T05

Complaint with service outage signal

Ticket body: 'Your service has been down for 2 hours. Our business is losing money. This is completely unacceptable.'

category: complaint, priority: high, sentiment: -0.9 (very angry)

Category is complaint; priority is high; sentiment below -0.8; contains outage keyword

T06

Multi-line ticket with mixed signals

Ticket body: 'Subject: Login broken after update. Body: Updated yesterday and now I cannot access my account. It keeps redirecting to a blank page. Can you please help?'

category: technical (account-access secondary), priority: high, sentiment: -0.3 (concerned)

Primary category is technical; secondary is account-access; priority is high

T07

Empty or minimal ticket

Ticket body: 'Help'

category: unknown (flag for manual review), priority: normal, sentiment: 0.0 (neutral)

Category is marked unknown; priority defaults to normal; agent flags for human review

T08

Non-English ticket

Ticket body: 'Je ne peux pas accéder à mon compte. S'il vous plaît aidez-moi.'

category: account-access, priority: normal, sentiment: -0.4 (frustrated)

Agent detects language; translates or flags; category still extracted; priority assigned

Smart Router Agent: Takes categorized ticket and matches it to the correct queue, team, or individual agent based on routing rules, skill assignment, and current capacity. Tests validate rule logic, agent lookup accuracy, and SLA target assignment.

Test ID

Scenario

Input

Expected Output

Pass Criteria

T09

Billing high-priority routes to billing-urgent queue

ticket_category: billing, priority: high, customer_tier: enterprise

queue: billing-urgent, sla_minutes: 30, target_agent: null (queue assignment)

Queue is billing-urgent; SLA is 30 minutes; no individual agent override

T10

Technical low-priority routes to tech-general queue

ticket_category: technical, priority: low, customer_tier: free

queue: tech-general, sla_minutes: 480 (8 hours), target_agent: null

Queue is tech-general; SLA is 480 minutes; assignment is to queue not individual

T11

Account-access high-priority escalates to lead engineer

ticket_category: account-access, priority: high, customer_tier: enterprise, assigned_qa: yes

queue: account-access, target_agent: 'Lead Engineer', sla_minutes: 15

Agent is assigned to Lead Engineer; SLA is 15 minutes; escalation is marked

T12

Complaint routes to support lead with escalation flag

ticket_category: complaint, priority: high, customer_tier: standard

queue: escalations, target_agent: 'Support Lead', escalation_flag: true, sla_minutes: 60

Queue is escalations; escalation flag is true; SLA is 60 minutes

T13

Feature request routed to product team

ticket_category: feature-request, priority: low, customer_tier: any

queue: product-feedback, sla_minutes: 1440 (24 hours), target_agent: null

Queue is product-feedback; SLA is 24 hours; no urgency flag

T14

Unknown category flags for manual review

ticket_category: unknown, priority: normal

queue: hold, target_agent: 'Support Lead', escalation_flag: true, sla_minutes: 60

Ticket held for manual classification; escalation flag set; SLA is 60 minutes

T15

All available agents at capacity, queue overflow

ticket_category: technical, priority: normal, available_agents: 0, queue_depth: 12

queue: tech-general, target_agent: null, overflow_flag: true, backlog_position: 12

Agent is not assigned (capacity full); overflow flag is true; backlog position tracked

T16

Premium customer technical issue gets priority queue slot

ticket_category: technical, priority: normal, customer_tier: premium

queue: tech-premium, sla_minutes: 120, target_agent: null

Queue is tech-premium (elevated from tech-general); SLA is 120 minutes

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2: Integration and edge case tests

Phase 2 integration tests validate that both agents correctly read from and write to Zendesk, Slack, HubSpot, and Gmail APIs in the staging environment. Each test creates a real ticket in Zendesk staging, runs the automation, and confirms that tags, assignments, Slack messages, and HubSpot logs are created as expected.

Test ID

Scenario

Input

Expected Output

Pass Criteria

T17

Create billing ticket, read from Zendesk API, write tags and assignment back

POST /tickets with { subject: 'Refund request', description: 'Charged twice', customer_email: 'test@example.com' } to Zendesk staging

Zendesk API returns ticket ID; Ticket Analyzer reads ticket via GET /tickets/{id}; Smart Router returns { tags: [billing, high], assignment: billing-urgent }; automation POSTs update with tags and custom fields

Ticket created in Zendesk staging; tags and assignment written without error; ticket status changed to assigned

T18

Send Slack notification to billing-urgent queue channel

After routing, automation calls Slack Webhook with { channel: #billing-urgent, text: 'High-priority billing ticket: Refund request (ID: 1234)', urgency: high }

Slack receives POST request; message appears in #billing-urgent within 5 seconds; message includes ticket link and priority emoji

Slack message posted to correct channel; formatted correctly; contains ticket link and priority indicator

T19

Log triage decision to HubSpot contact record

After routing, automation POSTs to HubSpot API with { contact_email: 'test@example.com', ticket_category: billing, priority: high, assigned_queue: billing-urgent, triage_timestamp: <ISO8601> }

HubSpot API returns 200 OK; custom properties on contact record updated: ticket_category_last = 'billing', triage_priority = 'high', triage_queue = 'billing-urgent'

HubSpot contact record updated; all fields populated; timestamp is current

T20

Read email forwarded from Gmail and create Zendesk ticket

Gmail forwards support email to automation via webhook; automation extracts sender, subject, and body; creates ticket in Zendesk

Zendesk ticket created with { requester_email: <from Gmail>, subject: <Gmail subject>, description: <Gmail body>, source: email }; ticket categorized and routed

Email parsed correctly; Zendesk ticket created with source=email; triage runs without error

T21

Handle Zendesk API rate limit (429 response)

Automation receives 429 Too Many Requests while writing tags; automatic retry with exponential backoff

After 3 retries with 1-2-4 second delays, request succeeds; tag is written; no ticket is lost

Retry logic executes; request succeeds on retry; no data is dropped

T22

Handle missing HubSpot contact (email not in CRM)

Ticket arrives from email not found in HubSpot; automation attempts to log triage to non-existent contact

HubSpot API returns 404; automation creates new contact record with email and ticket metadata; triage data is still logged

New contact created in HubSpot; triage metadata attached; no error in automation flow

T23

Handle Slack channel does not exist or is archived

Smart Router assigns ticket to queue with channel #archived-queue; automation tries to post Slack message

Slack API returns 404 (channel not found); automation logs error, sends fallback notification to #general, and creates ticket in Zendesk with escalation flag

Fallback notification sent to #general; ticket escalation flag set; automation continues

T24

Zendesk custom field does not exist in target organization

Automation tries to write custom field 'sla_target_minutes' that does not exist in staging Zendesk

Zendesk API returns 422 Unprocessable Entity; automation logs field mismatch, writes available standard fields instead, and creates alert in FullSpec dashboard

Error is logged; standard fields are written; alert is visible in dashboard

Edge Case Tests: These scenarios represent unusual or potentially problematic tickets that require special handling, fallback logic, or human review flags.

Test ID

Scenario

Input

Expected Output

Pass Criteria

E01

Abusive language or threat detected

Ticket body contains profanity, threats, or harassment language; sentiment score is < -0.9

category: complaint, priority: escalation, safety_flag: true, target_agent: 'Support Lead', requires_review: true

Safety flag is set; ticket is escalated to Support Lead; marked for manual review before assignment

E02

Ticket references product not in company portfolio

Ticket mentions 'Product X' which customer does not have license for; or mentions a competitor product name

category: unknown, priority: high, escalation_flag: true, target_agent: 'Support Lead', note: 'Customer may be requesting help with wrong product'

Agent flags for manual review; ticket is escalated; note is added to ticket

E03

Customer mentions legal action or lawsuit threat

Ticket body contains phrases like 'lawyer', 'lawsuit', 'legal action', or 'class action'

category: legal, priority: escalation, escalation_flag: true, target_agent: 'Support Lead', requires_legal_review: true

Category is legal; legal review flag is set; escalation is automatic; no standard agent assignment

E04

Ticket is a duplicate of recently resolved ticket

New ticket arrives with same customer email and very similar content to ticket closed 2 days ago

category: duplicate, priority: normal, escalation_flag: false, linked_ticket_id: <previous ticket ID>, target_agent: 'Support Agent', note: 'See linked ticket for context'

Duplicate is detected and linked; agent is assigned to reference previous solution

E05

Ticket contains PII (password, credit card, SSN) in body

Ticket body includes patterns matching credit card numbers, social security numbers, or customer explicitly states 'my password is X'

priority: escalation, pii_flag: true, requires_manual_review: true, sanitize_before_logging: true, target_agent: 'Support Lead'

PII flag is set; ticket is flagged for manual review; PII is redacted before any logging

E06

Very long ticket with multiple issues

Ticket body is > 5000 characters and mentions 5 or more distinct problems

category: multi-issue, priority: high, escalation_flag: true, target_agent: 'Support Lead', note: 'Ticket contains multiple issues. Recommend splitting into sub-tickets.',

Multi-issue flag is set; escalation to Support Lead; note recommends decomposition

E07

Ticket from VIP or high-value customer

Customer email domain matches known enterprise account in HubSpot with ARR > $100k

priority: raised to high if normal, priority: escalation if already high, sla_minutes: reduced by 50%, target_agent: assigned to senior agent if available

Priority is elevated; SLA is reduced; assignment goes to senior team member

E08

Ticket in language other than English

Ticket body detected as Spanish, French, German, or Japanese (supported languages)

language: detected language, translation_provided: true, category: extracted from translated body, priority: assigned based on translated content, note: 'Ticket auto-translated from <language>'

Language is detected; content is translated; triage proceeds on translated text; note is added

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3: End-to-end production test

Production testing runs on live Zendesk, Slack, and HubSpot with real incoming support tickets. The first phase will process 50% of all new tickets (approximately 175 tickets over 5 business days). No production data will be modified, deleted, or rolled back without explicit written approval from your process owner. All errors and edge cases will be logged and reported daily.

Before moving to 100% automation, your support team lead must review a sample of 20 categorized and routed tickets, confirm that routing accuracy is >= 94%, and sign off on the results in the final test case below.

Test ID

Scenario

Input

Expected Output

Pass Criteria

T25

Run automation on first 50% of production tickets

Automation enabled in production Zendesk; all new tickets (approximately 175 over 5 days) are routed through Ticket Analyzer and Smart Router

Each ticket is categorized, assigned priority, routed to correct queue or agent, tagged in Zendesk, logged to HubSpot, and Slack notification sent within 45 seconds

100% of tickets processed; 94% routed to correct queue on first try; 0 critical errors; all Slack notifications delivered

T26

Validate category accuracy on random sample

FullSpec QA pulls 20 random categorized tickets from production; support agent reviews each category assignment

18 of 20 (90%) are categorized correctly; 1 of 20 (5%) is partially correct but acceptable; 1 of 20 (5%) is incorrect and should be manually recategorized

Accuracy is >= 90%; errors are documented and categorized as training feedback

T27

Validate priority assignment accuracy

FullSpec QA pulls 20 random priority assignments from production; support agent reviews each

18 of 20 (90%) are assigned correct priority; 2 of 20 (10%) are one level off (high vs normal) but not critical

Priority accuracy is >= 85%; no high-priority tickets are marked as low

T28

Confirm Slack messages format correctly and arrive on time

FullSpec tracks delivery time and format of all Slack notifications sent over 5-day period

All messages arrive in correct channel; all arrive within 45 seconds of ticket creation; all include ticket ID, category, priority, and clickable link

100% of Slack messages delivered; average delivery time < 10 seconds; format is correct

T29

Confirm HubSpot records are logged for every ticket

FullSpec queries HubSpot API to count contact records updated with triage metadata over 5-day period

All 175 tickets have corresponding HubSpot triage log entry; timestamp, category, priority, and queue are recorded for each

100% of tickets logged to HubSpot; all fields present; no missing or null values

T30

Monitor error rate and fallback triggers

FullSpec logs all errors, retries, fallbacks, and manual review flags over 5-day period

Total error rate < 2% (< 4 errors out of 175 tickets); all errors are documented in daily report; no ticket is lost due to error

Error rate is < 2%; all errors are recoverable; no customer impact

T31

Validate SLA target assignment for different customer tiers

FullSpec pulls sample of tickets from free, standard, and premium customer tiers; confirms SLA targets are applied correctly

Free tier: SLA 480+ minutes for normal, 240 for high. Standard tier: SLA 240 for normal, 120 for high. Premium tier: SLA 120 for normal, 60 for high

All SLA targets match tier-based routing rules; escalation thresholds are respected

T32

Confirm support team can manually override automation decision if needed

Support agent opens a misrouted ticket in Zendesk; manually reassigns it to a different queue; confirms automation does not re-route it back

Manual reassignment is respected; automation recognizes manual change and does not override; ticket stays in new queue

Manual overrides are honored; no re-routing occurs; agent retains control

T33

Support team lead review and sign-off

Process owner (support team lead) reviews production results: category accuracy, routing accuracy, Slack notification delivery, HubSpot logging, error logs, and any escalations flagged for manual review.

Process owner confirms: routing accuracy >= 94%; no critical errors; all alerts are documented; team confidence in automation is high enough to proceed to 100% rollout

Process owner reviews sample tickets and live dashboards; signs off on production readiness; FINAL APPROVAL RECORDED.

If all Phase 3 tests pass and the process owner approves in test case T33, the automation will be ramped to 100% of incoming tickets on day 6 of production testing. FullSpec will monitor the system for 7 additional days and provide a post-launch report documenting performance, any adjustments made, and training recommendations for your support team.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1: Unit test cases

03Phase 2: Integration and edge case tests

04Phase 3: End-to-end production test

More documents for this process