FS-DOC-06Quality
Test and QA Plan
Refund and Returns Processing
[YourCompany.com] · Customer Support Department · Prepared by FullSpec · [Today's Date]
This document defines the full test and quality assurance plan for the Refund and Returns Processing automation. It covers every test case the FullSpec team runs across three sequential phases: unit tests for each agent in isolation, integration and edge-case tests for agent handoffs and failure modes, and a final end-to-end production run with process owner sign-off. No phase begins until all cases in the previous phase reach a passing state. The process owner joins the FullSpec team in Phase 3 only.
01Testing approach
Testing runs in three sequential phases. Phase 1 (Unit) isolates each agent and tests its logic independently. Phase 2 (Integration) tests agent-to-agent handoffs and system-level edge cases. Phase 3 (End-to-end) validates the complete flow in the production environment with real credentials, real orders, and process owner participation. No phase begins until every test case in the preceding phase carries a passing status.
1Phase 1: Unit Testing
Each agent tested in isolation against a sandbox environment
Scope
Returns Triage Agent and Refund Execution Agent individually; no cross-agent calls
Environment
Sandbox only: Zendesk sandbox tenant, Shopify development store, Xero demo company, Gmail test account, Slack test workspace
Credential rule
Sandbox credentials exclusively; no production API keys permitted in Phase 1
Pass condition
All cases T01 through T10 return expected outputs with zero production data touched
Who runs it
FullSpec team only
2Phase 2: Integration and Edge Case Testing
Agent handoffs and failure-mode scenarios in a sandbox environment
Scope
Returns Triage Agent to Refund Execution Agent handoff; all edge-case and exception paths
Environment
Sandbox: same credentials as Phase 1; no production data used
Credential rule
Sandbox credentials exclusively; no production API keys permitted in Phase 2
Pass condition
All integration cases T11 through T16 pass and all edge cases E01 through E06 reach the correct outcome or escalation path
Who runs it
FullSpec team only
3Phase 3: End-to-End Production Test
Full flow validated in the live environment with process owner participation
Scope
Complete workflow from Zendesk ticket trigger through Gmail, Shopify, Xero, Slack, and Zendesk resolution, using production credentials and a designated test order
Environment
Production: live Zendesk tenant, live Shopify store, live Xero organisation, live Gmail account, live Slack workspace
Credential rule
Production credentials used; test must use a designated internal test email address and the test order must be voided or reversed immediately after the run
Pass condition
All cases T17 through T21 pass, including process owner review and approval recorded in the FullSpec confirmation
Who runs it
FullSpec team leads the run; process owner participates in and approves the final case (T21)
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only. All Phase 1 tests must be run against the Zendesk sandbox tenant, Shopify development store, Xero demo company, Gmail test account, and Slack test workspace. Do not connect production API keys at this stage. Any test that inadvertently touches a live system is an automatic fail and must be re-run in the correct environment.
Returns Triage Agent: connects to Zendesk and Shopify, extracts order data from an inbound ticket, and runs the configured eligibility rules to produce an approved or flagged decision.
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: in-policy return, order within return window
Zendesk ticket with valid order number; Shopify order dated 12 days ago; return window 30 days; no excluded categories
Decision: approved; reason code: WITHIN_WINDOW; order data and refund amount attached to ticket
Eligibility decision is 'approved'; reason code present; Shopify order fields (date, amount, items) populated on ticket; no errors logged
T02
Out-of-policy: order outside return window
Zendesk ticket with valid order number; Shopify order dated 45 days ago; return window 30 days
Decision: flagged for review; reason code: OUTSIDE_WINDOW; ticket tagged for human review
Decision is 'flagged'; reason code is OUTSIDE_WINDOW; Zendesk ticket receives human-review tag; no refund initiated
T03
Out-of-policy: excluded product category
Zendesk ticket; Shopify order containing an item in the excluded category list (e.g. digital download)
Decision: flagged for review; reason code: EXCLUDED_CATEGORY
Decision is 'flagged'; reason code is EXCLUDED_CATEGORY; ticket tagged for human review
T04
Order not found in Shopify
Zendesk ticket with an order number that does not exist in the Shopify development store
Agent logs error ORDER_NOT_FOUND; ticket escalated to human review; no eligibility decision emitted
Error code ORDER_NOT_FOUND is recorded; ticket escalated; execution halts at this step; no downstream actions triggered
T05
Missing order reference in ticket
Zendesk ticket body contains no parseable order number or customer email match
Agent logs error MISSING_ORDER_REF; ticket tagged for manual triage; no Shopify lookup attempted
Error MISSING_ORDER_REF logged; no Shopify API call made; ticket tagged for manual triage queue
Refund Execution Agent: receives an approved decision from the triage step and carries out email confirmation, Shopify refund, Xero credit note, Zendesk resolution, and Slack alerting (high-value cases only).
ID
Scenario
Input
Expected Output
Pass Criteria
T06
Happy path: approved refund below high-value threshold
Approved decision; refund amount $45; customer email present; sandbox Shopify order; sandbox Xero company
Gmail confirmation sent; Shopify refund initiated; Xero credit note created; Zendesk ticket resolved and tagged; no Slack alert posted
All five actions complete without error; Slack channel receives no message; Zendesk status is 'solved'
T07
Happy path: approved refund above high-value threshold
Approved decision; refund amount above configured threshold (e.g. $500); customer email present
All T06 outputs plus a structured Slack alert posted to the designated test channel
Slack alert posted with correct order reference, refund amount, and customer name; all other T06 criteria also met
T08
Gmail send failure
Approved decision; Gmail sandbox credentials revoked mid-test to simulate send failure
Agent logs error GMAIL_SEND_FAILED; workflow pauses; retry attempted after configured interval; alert raised if retry fails
Error GMAIL_SEND_FAILED logged; retry logic executes; Shopify refund and Xero credit note are NOT created until email confirms; no silent failure
T09
Shopify refund API error (e.g. 422 Unprocessable Entity)
Approved decision; Shopify development store returns a 422 error on the refund endpoint
Agent logs error SHOPIFY_REFUND_FAILED with HTTP status; Xero credit note not created; Zendesk ticket flagged for manual follow-up
Error logged with status code; Xero step skipped; ticket set to 'pending' with internal note describing the failure; no partial refund created
T10
Xero credit note creation failure (e.g. 403 Forbidden)
Approved decision; Shopify refund succeeds; Xero demo company returns 403 on credit note endpoint
Agent logs error XERO_CREDITNOTE_FAILED; Slack alert posted regardless of value threshold; Zendesk ticket flagged for finance follow-up
Error XERO_CREDITNOTE_FAILED logged; Slack alert fires; Zendesk ticket carries internal note 'Xero credit note pending manual creation'; Shopify refund is not reversed
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 tests the full handoff between the Returns Triage Agent and the Refund Execution Agent, plus all integration points across Zendesk, Shopify, Xero, Gmail, and Slack. The ID sequence continues from Phase 1. A separate edge-case table (E01 onwards) covers duplicates, missing data, unavailable tools, and human timeout scenarios.
ID
Scenario
Input
Expected Output
Pass Criteria
T11
Standard approved handoff: Triage Agent passes approved decision to Execution Agent
In-policy Zendesk ticket; Triage Agent produces approved decision with order data and refund amount
Execution Agent receives the full payload (order reference, customer email, refund amount, reason code) and begins execution within 10 seconds
Execution Agent trigger fires; payload fields are all populated; no data loss between agents; execution log shows handoff timestamp
T12
Flagged handoff: Triage Agent passes a human-review decision; Execution Agent must not fire
Out-of-policy ticket; Triage Agent produces flagged decision
Execution Agent does not trigger; Zendesk ticket is tagged for support lead review; no email, refund, or Xero entry created
Execution Agent shows no run record; Zendesk ticket has human-review tag; no Gmail, Shopify, or Xero API calls made
T13
Zendesk webhook fires twice for the same ticket (duplicate trigger)
Zendesk sandbox sends two webhook events for ticket ID ZD-TEST-001 within 30 seconds
Workflow deduplication logic detects the duplicate; second run is suppressed; only one refund is processed
Execution log shows one completed run and one suppressed duplicate; Shopify refund count for the order is exactly one; Xero has one credit note
T14
Full approved flow across all five tools end-to-end in sandbox
In-policy ticket; all sandbox credentials active; refund amount $120 (below high-value threshold)
Gmail confirmation sent; Shopify refund issued; Xero credit note created; Zendesk ticket resolved; no Slack alert
All five tool API responses return 2xx; execution log shows all steps green; total automated processing time under 60 seconds
T15
Full approved flow with high-value refund across all five tools in sandbox
In-policy ticket; refund amount above high-value threshold; all sandbox credentials active
All T14 outputs plus Slack alert posted to test channel
Slack API returns 200; message appears in test channel with correct structured content; all other T14 criteria met
T16
Xero API rate limit hit during credit note creation (429 Too Many Requests)
Execution Agent reaches Xero step; Xero demo company returns 429
Agent waits the retry-after interval (minimum 60 seconds for Xero), retries up to three times, logs each attempt, and alerts via Slack if all retries fail
Retry log shows up to three attempts with timestamps; if retries succeed, credit note is created normally; if all fail, XERO_RATELIMIT_EXHAUSTED error is logged and Slack alert fires
Edge case tests cover failure modes outside the standard happy and error paths: duplicate submissions, structurally incomplete data, tool unavailability, and scenarios where a human reviewer does not respond within the expected window.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate return request submitted by the same customer within 10 minutes
Two Zendesk tickets for the same order ID and customer email, submitted 8 minutes apart
Second ticket is identified as a duplicate; tagged DUPLICATE_REQUEST; linked to the first ticket; Execution Agent does not fire for the second ticket
Only one refund is processed; second ticket has DUPLICATE_REQUEST tag and internal note linking to the active ticket; no double-refund in Shopify or Xero
E02
Ticket missing customer email address
Zendesk ticket contains order number but no customer email; Shopify order lookup returns an email
Agent falls back to email extracted from Shopify order record; flow continues using that email
Gmail is sent to the Shopify-sourced email; execution log notes the fallback source; no failure logged
E03
Shopify API unavailable (500 Internal Server Error) during order lookup
Returns Triage Agent attempts Shopify order fetch; Shopify returns 500
Agent retries up to three times with exponential backoff; if all retries fail, ticket is tagged SHOPIFY_UNAVAILABLE and held in a pending queue for retry
Retry log shows three attempts; ticket status is 'pending' with correct tag; no eligibility decision is emitted; no downstream steps fire
E04
Refund amount is zero or negative after Shopify order data is parsed
Shopify order found; line items sum to $0 due to a previously applied discount covering the full amount
Agent logs error ZERO_REFUND_AMOUNT; ticket flagged for human review; no refund or credit note initiated
Error ZERO_REFUND_AMOUNT in execution log; ticket tagged for human review; Shopify and Xero APIs not called
E05
Human reviewer does not action a flagged ticket within 48 hours
Out-of-policy ticket routed to support lead; no action taken for 48 hours in the sandbox simulation
Automation sends a reminder notification to the support lead via Slack at the 48-hour mark; ticket remains open and pending human action
Slack reminder is posted at the correct interval; ticket is not auto-resolved or auto-rejected; no financial action taken without human input
E06
Partial refund scenario: customer requests a refund on one of two line items
Zendesk ticket specifies one line item for return; Shopify order has two line items
Triage Agent identifies the specific line item; refund amount reflects that item only; Xero credit note matches the partial amount; full order is not refunded
Shopify refund payload contains only the specified line item ID and quantity; Xero credit note amount matches the partial refund; Zendesk ticket notes the partial nature of the return
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: use a designated internal test email address (not a real customer address) for all production runs. Create a real but low-value test order in the live Shopify store before the session. After the test completes, immediately void or reverse the Shopify refund, delete or credit the Xero credit note, and resolve the Zendesk test ticket with the tag TEST_RUN so it is excluded from SLA and reporting metrics. Do not run Phase 3 during peak support hours.
Phase 3 requires all production credentials to be active and scoped correctly before the session begins: Zendesk API token with ticket read and write access; Shopify API key with orders read, refunds create, and products read scopes; Xero connected app with accounting create scope; Gmail OAuth token for the sending account; Slack bot token with chat:write scope on the target channel. Confirm all credentials with the FullSpec team at least 24 hours before the scheduled Phase 3 date.
ID
Scenario
Input
Expected Output
Pass Criteria
T17
Full production run: approved in-policy return, below high-value threshold
Live Zendesk ticket created with the designated test order number; test order in live Shopify store dated within the return window; internal test email address on the ticket
Returns Triage Agent: approved decision with reason code and order data attached to ticket. Refund Execution Agent: Gmail confirmation sent to test inbox; Shopify refund initiated on the test order; Xero credit note created in the live organisation; Zendesk ticket resolved with correct tags
All four API calls return 2xx in the production execution log; Gmail test inbox receives the confirmation email within 2 minutes; Shopify order shows refund status; Xero has a new credit note matching the refund amount; Zendesk ticket status is 'solved'; no Slack alert (below threshold)
T18
Full production run: approved in-policy return, above high-value threshold
Live Zendesk ticket with test order value above the configured high-value threshold; internal test email
All T17 outputs plus a structured Slack alert posted to the live designated channel
Slack channel receives the alert within 30 seconds of ticket resolution; message contains order reference, refund amount, and customer identifier; all T17 criteria also met
T19
Production flagged case: out-of-policy request routes to human review without triggering refund
Live Zendesk ticket with test order dated outside the return window
Returns Triage Agent: decision is 'flagged'; ticket tagged for support lead review. No Gmail, Shopify, Xero, or Slack actions fired by the Execution Agent
Execution Agent shows no run record for this ticket; Zendesk ticket has human-review tag; production Shopify order has no refund entry; Xero has no new credit note; Gmail test inbox receives no message
T20
Production error log verification: confirm all run logs are accessible and structured correctly
FullSpec team reviews the execution logs from T17, T18, and T19 in the automation platform's production log store
Logs for all three runs are present; each log contains: run ID, trigger timestamp, each step name and status, API response codes, error messages where applicable, and total run duration
All three runs appear in the log store; log entries are complete and machine-readable; no missing steps; run durations for T17 and T18 are under 90 seconds; error fields are null where no error occurred
T21
Process owner review and approval: owner confirms outputs match expected behaviour and approves go-live
Process owner reviews the Gmail test inbox, Shopify order refund status, Xero credit note, Zendesk ticket resolution, Slack alert (from T18), and execution logs alongside the FullSpec team
Process owner confirms: customer email content is accurate and professional; refund amount in Shopify matches the test order; Xero credit note is correctly matched to the test customer account; Zendesk ticket is resolved with correct tags; Slack alert contains the right data. Owner approves go-live in writing via the FullSpec confirmation record.
Process owner sign-off recorded in the FullSpec confirmation. All outputs verified by the owner against expected results from T17 and T18. No discrepancies raised. Go-live approved.
After T21 is complete and go-live is approved: void the Shopify test refund, reverse or delete the Xero credit note, resolve the Zendesk test tickets with the TEST_RUN tag, and archive the Phase 3 execution logs in the FullSpec project record. The FullSpec team will confirm completion of all post-test cleanup before the production workflow is set to active. Contact the FullSpec team at support@gofullspec.com with any questions.
Test and QA PlanPage 4 of 4