Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Returns and Reverse Logistics

[YourCompany.com] · Fulfilment Department · Prepared by FullSpec · [Today's Date]

This document defines the full quality assurance programme for the Returns and Reverse Logistics automation. It covers three sequential testing phases: unit testing of each individual agent, integration and edge-case testing of cross-agent handoffs, and a final end-to-end production run. The FullSpec team runs all phases and owns the test infrastructure. The process owner joins Phase 3 to confirm the production output and record sign-off. No phase begins until every test case in the previous phase carries a passing result.

01Testing approach

Testing is structured as three phases that run in sequence. Phase 1 validates each agent in isolation using sandbox credentials. Phase 2 verifies that agents hand off correctly to one another and that the system behaves predictably under edge conditions. Phase 3 executes one full production run with live credentials to confirm the system is ready for handover. No phase begins until every test case in the preceding phase has passed.

Phase 1 — Unit Testing

Each agent tested in isolation before any cross-agent work begins

Scope

Returns Intake Agent, Dispatch and Monitoring Agent, Refund and Reconciliation Agent — each tested independently against defined inputs and expected outputs

Environment

Sandbox only. No live Shopify orders, ShipStation shipments, Xero accounts, or Gmail sends are used at this stage

Credential rule

Sandbox API keys and test OAuth tokens for all six tools. Production credentials must not be used in Phase 1 under any circumstances

Pass condition

All cases T01 through T15 return the expected output with no unhandled errors. Failed or partial cases must be resolved before Phase 2 begins

Who runs it

FullSpec team only

Phase 2 — Integration and Edge-Case Testing

Cross-agent handoffs and failure modes validated in sandbox

Scope

Agent-to-agent handoffs across all three agents, plus edge cases covering duplicate submissions, missing data, tool unavailability, and human timeout scenarios

Environment

Sandbox. A staging ShipStation account with a simulated delivery webhook is used for tracking event tests

Credential rule

Sandbox credentials only. The Xero sandbox organisation and Shopify development store are used throughout. No real customer records are touched

Pass condition

All cases T16 through T26 and edge cases E01 through E08 pass. Any case that routes incorrectly or silently drops data is treated as a blocking failure

Who runs it

FullSpec team only

Phase 3 — End-to-End Production Test

Full live run with production credentials; process owner participates

Scope

One complete return traced from Shopify portal submission through to Xero credit note creation and Gmail confirmation, using a real but internally controlled test order

Environment

Production. Live Shopify store, live ShipStation account, live Xero organisation, live Gmail account, and the active Slack warehouse channel

Credential rule

Production OAuth tokens and API keys are used. The test order and any records created must be voided or reversed immediately after sign-off

Pass condition

All cases T27 through T31 pass. The process owner reviews output at T31 and records approval in the FullSpec confirmation log

Who runs it

FullSpec team executes T27 through T30. Process owner participates in T31 to review and confirm the production output

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only. Every test in Phase 1 must use sandbox API keys, test OAuth tokens, the Shopify development store, and the Xero demo organisation. Do not connect production credentials at this stage. Any accidental write to a live system must be reported to the FullSpec team immediately so the record can be voided.

Returns Intake Agent — captures return requests from Shopify, checks eligibility, logs to Google Sheets, and creates a ShipStation label. Estimated build time: 16 hours. Complexity: Moderate.

Scenario

Input

Expected output

Pass criteria

T01

Happy path — qualifying return

Shopify webhook: order within return window, standard product, valid customer address

Row appended to Google Sheets; ShipStation label created; label URL stored in sheet

Sheet row present with correct order ID, reason code, and label URL within 30 seconds of trigger

T02

Non-qualifying return — outside return window

Shopify webhook: order date exceeds configured return window by 2 days

Eligibility check fails; record flagged as 'Exception' in Google Sheets; no label created in ShipStation

Sheet row status = 'Exception'; no ShipStation shipment record created; no label URL in row

T03

Non-qualifying return — excluded product type

Shopify webhook: order contains a product tagged as non-returnable (e.g. sale item)

Eligibility check fails; record flagged as 'Exception'; no downstream actions triggered

Sheet row status = 'Exception'; ShipStation API not called; agent execution log shows eligibility rejection reason

T04

Shopify webhook missing required field

Webhook payload with order_id present but customer address fields null

Agent halts at label creation step; error logged; row in Google Sheets marked 'Error — missing address'

No ShipStation API call made; error entry visible in automation platform execution log; sheet row status = 'Error'

T05

ShipStation label creation API failure (simulated 503)

Qualifying return input; ShipStation sandbox returns 503 on label create endpoint

Agent retries up to 3 times with exponential backoff; after final failure, sheet row marked 'Label Error'; no silent drop

Execution log shows 3 retry attempts; sheet status = 'Label Error'; no partial label record in ShipStation

T06

Google Sheets write failure (simulated permissions error)

Qualifying return input; Sheets API returns 403 on append

Agent logs error; does not proceed to ShipStation; alert surfaced in automation platform error log

Execution log contains 403 detail; ShipStation API not called; no orphaned label created

T07

Partial return — multiple line items, one item returned

Shopify webhook: order with 3 line items; return reason specifies line item 2 only

Sheet row records the specific line item; ShipStation label created for correct item weight/dimensions

Sheet row line_item_id matches returned item; ShipStation shipment weight matches item record in Shopify

Dispatch and Monitoring Agent — sends the return label to the customer via Gmail and monitors ShipStation for delivery confirmation before alerting the warehouse in Slack. Estimated build time: 12 hours. Complexity: Moderate.

Scenario

Input

Expected output

Pass criteria

T08

Happy path — label email sent successfully

Label URL present in Google Sheets row; Gmail sandbox account connected

Templated email with label link and packing instructions delivered to test recipient; send timestamp written back to sheet

Email received in test inbox within 60 seconds; sheet column 'label_sent_at' populated with ISO timestamp

T09

Gmail send failure (simulated SMTP rejection)

Valid label URL; Gmail sandbox returns sendFailure on send attempt

Agent retries once; on second failure, execution log records error and sheet row status updated to 'Email Error'

Execution log shows two send attempts; sheet status = 'Email Error'; no email in test inbox

T10

ShipStation delivery webhook fires correctly

ShipStation sandbox delivery event for tracked shipment matching an open sheet row

Agent matches shipment to Google Sheets row by tracking number; Slack message posted to test warehouse channel

Slack message appears in designated test channel within 30 seconds; message contains order number and inspection form link

T11

ShipStation delivery webhook fires for unknown tracking number

ShipStation delivery event with a tracking number not present in the Google Sheets log

Agent logs 'unmatched tracking number' warning; no Slack message sent; no row updated

Execution log contains warning entry; Slack channel unchanged; no sheet row modified

T12

Slack post failure (simulated channel not found)

Delivery event matched; Slack API returns channel_not_found on post

Agent logs error; sheet row status updated to 'Slack Alert Error'; does not proceed to next step

Execution log shows Slack error; sheet status = 'Slack Alert Error'; warehouse channel not affected

Refund and Reconciliation Agent — processes the Shopify refund, adjusts inventory, creates a Xero credit note, and sends a customer confirmation email. Estimated build time: 20 hours. Complexity: Complex.

Scenario

Input

Expected output

Pass criteria

T13

Happy path — full refund, resalable item

Inspection form submission: outcome = resalable; refund_type = full; order_id matches open sheet row

Full refund issued in Shopify; inventory incremented by 1 for the returned SKU; Xero credit note created matching refund amount; confirmation email sent to customer

Shopify refund status = refunded; inventory delta = +1 for correct location; Xero credit note amount matches Shopify refund to the cent; confirmation email in test inbox

T14

Partial refund — damaged item

Inspection form submission: outcome = damaged; refund_type = partial; partial_amount = $35.00; original order total = $80.00

Partial refund of $35.00 issued in Shopify; inventory not incremented (damaged); Xero credit note for $35.00 created; confirmation email references partial amount

Shopify refund amount = $35.00; inventory unchanged; Xero credit note amount = $35.00; email body contains '$35.00' and 'partial refund' text

T15

No refund — write-off outcome

Inspection form submission: outcome = write_off; refund_type = none

No Shopify refund issued; inventory not updated; no Xero credit note created; sheet row status updated to 'Write-off — no refund'

Shopify order refund amount = $0; Xero API not called; sheet status = 'Write-off — no refund'; no customer email sent

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests confirm that data passes correctly between agents at each handoff point. All tests use sandbox credentials. IDs continue from Phase 1.

Scenario

Agents involved

Input

Expected output

Pass criteria

T16

Intake to Dispatch handoff — label URL passed correctly

Returns Intake Agent to Dispatch and Monitoring Agent

Qualifying return processed by Intake Agent; label URL written to Google Sheets

Dispatch Agent reads label URL from sheet and includes it in the Gmail send without manual intervention

Email contains the exact label URL written by the Intake Agent; no truncation or encoding error

T17

Dispatch to Reconciliation handoff — inspection form submission triggers correct agent

Dispatch and Monitoring Agent to Refund and Reconciliation Agent

Slack alert posted by Dispatch Agent; warehouse operative submits inspection form via the link in the message

Form submission fires the Reconciliation Agent with the correct order_id, inspection outcome, and refund type

Reconciliation Agent execution log shows order_id matching the Slack alert; no field mismatch; agent runs within 60 seconds of form submission

T18

Full end-to-end sandbox run — qualifying return, full refund

All three agents in sequence

Shopify sandbox webhook for a qualifying return; simulated delivery event; inspection form submitted with full refund outcome

All automated steps complete: sheet logged, label created, email sent, Slack alert posted, Shopify refund issued, Xero credit note created, confirmation email sent

All seven automated outputs present and correct; execution time under 3 minutes from trigger to confirmation email; no manual steps required

T19

Xero credit note amount matches Shopify refund exactly

Refund and Reconciliation Agent

Shopify partial refund of $47.50 issued by agent

Xero credit note created with net amount = $47.50 and correct customer reference from Shopify order

Xero API response: creditNote.total = 47.50; creditNote.contact matches Shopify customer name; no rounding error

T20

Inventory increment applied to correct Shopify location

Refund and Reconciliation Agent

Resalable return for SKU 'RET-1042'; default warehouse location configured in settings

Shopify inventory_level for SKU 'RET-1042' at the configured location incremented by 1

Shopify inventory_levels API response shows +1 for the correct location_id; no other location affected

T21

Google Sheets row status updated at each stage

All three agents

Full qualifying return run in sandbox

Sheet row status column updated at: intake logged, label created, email sent, delivered, inspection received, refund issued, credit note created, confirmation sent

Sheet column 'status' reflects each stage update; timestamps populated at each transition; no stage skipped

T22

Shopify refund API rate limit handling

Refund and Reconciliation Agent

Shopify sandbox configured to return 429 Too Many Requests on first refund attempt

Agent waits the Retry-After interval specified in the response header, then retries successfully

Execution log shows 429 received; retry fired after correct wait; refund issued on second attempt; no duplicate refund

T23

Xero API token expiry mid-run

Refund and Reconciliation Agent

Xero sandbox access token expired; refresh token valid

Agent detects 401, triggers OAuth refresh, obtains new access token, and completes credit note creation

Execution log shows 401 then successful token refresh; credit note created; no manual intervention required

T24

Multi-item order — only one item returned, inventory delta scoped correctly

Refund and Reconciliation Agent

Order with 3 line items; only line item 1 (SKU 'RET-0091') returned and approved

Refund applied only to line item 1; only SKU 'RET-0091' inventory incremented; other SKUs unchanged

Shopify refund line_items array contains only 'RET-0091'; inventory delta for 'RET-0091' = +1; other SKUs unchanged

T25

Confirmation email not sent if refund type is none

Refund and Reconciliation Agent

Inspection outcome = write_off; refund_type = none

No confirmation email sent; sheet status = 'Write-off — no refund'; Gmail API not called

Gmail API call count for this execution = 0; sheet status correct; no email in test inbox

T26

Exception path record visible in sheet for human review

Returns Intake Agent

Non-qualifying return (excluded product type)

Sheet row created with status = 'Exception'; no downstream agents triggered; row includes reason for exception

Sheet row present; status = 'Exception'; reason_code field populated; no ShipStation, Gmail, or Slack calls in execution log

Edge-case tests cover scenarios outside the standard happy path that could cause silent failures, duplicate records, or unhandled states. These use E-prefixed IDs and run in the same sandbox environment as the integration tests above.

Scenario

Condition

Expected behaviour

Pass criteria

E01

Duplicate return submission — same order submitted twice within 60 seconds

Two identical Shopify webhooks arrive for the same order_id

Second webhook detected as duplicate; second execution halted before sheet write; only one row created

Google Sheets contains exactly one row for the order_id; execution log shows 'duplicate detected' on second run

E02

Shopify webhook arrives with no reason code

Return submitted via portal with reason field left blank

Agent assigns reason_code = 'Not provided'; logs the row; proceeds with all other eligible fields; label still created if otherwise qualifying

Sheet row reason_code = 'Not provided'; label created if order is within window; no unhandled error in execution log

E03

ShipStation delivery event fires before Intake Agent has written the row

Race condition: delivery webhook arrives within seconds of a new return being logged

Dispatch Agent polls Google Sheets with a short retry loop (up to 3 attempts, 10-second intervals); matches row on third attempt

Slack alert eventually posted; execution log shows retry attempts; no permanent failure logged; row matched correctly

E04

Inspection form submitted with no outcome selected

Warehouse operative submits the inspection form leaving the outcome field blank

Reconciliation Agent detects missing outcome field; posts follow-up Slack message to warehouse channel requesting resubmission; does not proceed to refund

No Shopify refund issued; follow-up Slack message posted within 60 seconds; sheet status = 'Awaiting Inspection Resubmission'

E05

Xero is unavailable (simulated 500 from Xero API)

Xero API returns 500 on credit note create endpoint

Agent retries 3 times with 30-second backoff; after final failure, sheet row flagged 'Xero Error — manual credit note required'; Shopify refund is NOT rolled back

Shopify refund status = refunded; Xero credit note not created; sheet status = 'Xero Error — manual credit note required'; execution log contains all 3 retry attempts

E06

Slack warehouse channel is archived or renamed

Dispatch Agent attempts to post Slack alert to a channel that no longer exists

Agent catches channel_not_found error; logs the failure; sheet row updated to 'Slack Alert Error'; FullSpec error log surfaced for manual follow-up

Slack post not delivered silently; sheet status = 'Slack Alert Error'; error visible in automation platform error log

E07

Human timeout — inspection form not submitted within 48 hours of Slack alert

Delivery confirmed; Slack alert sent; 48-hour window elapses with no form submission

Automation sends a second Slack reminder to the warehouse channel referencing the original order; sheet status updated to 'Awaiting Inspection — Reminder Sent'

Second Slack message appears at the 48-hour mark; sheet status updated; no refund issued prematurely; original alert message still present in channel

E08

Return label emailed to an invalid or bounced address

Gmail send succeeds (HTTP 200) but a bounce notification is received within the test window

Agent cannot detect bounce natively; bounce handling relies on Gmail bounce monitoring configured in the Gmail account settings. FullSpec confirms this limitation is documented in the SOP for manual follow-up by the customer service team

Execution log shows send success; SOP reference for bounce handling is confirmed present in the handover documentation

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3. Use an internal email address (e.g. a staff mailbox) as the customer address on the test order so that no real customer receives an automated email. After Phase 3 is complete, void or reverse all records created during the test: cancel the Shopify refund if reversible, delete or void the Xero credit note, archive the Google Sheets test row, and confirm with the FullSpec team that no live inventory was permanently adjusted. Do not run Phase 3 against a real customer order or a live outstanding return.

Phase 3 uses live production credentials across all six tools. The FullSpec team creates a controlled test order in the live Shopify store, triggers the full flow, and traces every output end to end. The process owner joins at T31 to review the complete output set and record approval. All test records must be voided immediately after T31 is complete.

Scenario

Steps covered

Expected output

Pass criteria

T27

Production intake and label creation

Shopify portal return submitted by FullSpec team using a controlled internal test order within the return window; Shopify webhook received by the automation platform

Google Sheets row created with correct order_id, customer name, reason code, eligibility status, and ShipStation label URL; ShipStation return shipment visible in live account

Sheet row present within 60 seconds of submission; label URL valid and downloadable; ShipStation shipment record matches order details; no duplicate rows

T28

Production label email and delivery monitoring

Dispatch Agent reads label URL from sheet; sends Gmail to internal test address; FullSpec team simulates delivery by manually triggering the ShipStation delivery webhook against the live account

Label email received at the internal test address with correct label link and packing instructions; Slack message posted to the live warehouse channel with order number and inspection form link

Email in internal inbox within 90 seconds of label creation; Slack message in warehouse channel within 60 seconds of delivery event; both contain the correct order_id

T29

Production refund and inventory update

FullSpec team submits the inspection form (outcome = resalable, refund_type = full) on behalf of a test warehouse operative; Reconciliation Agent triggered

Full refund issued against the test order in Shopify; inventory for the test SKU incremented by 1 at the production warehouse location; sheet row status updated to 'Refunded'

Shopify order shows refund_status = refunded; inventory delta = +1 for the correct SKU and location; sheet status = 'Refunded'; no unintended inventory changes on other SKUs

T30

Production Xero credit note and confirmation email

Reconciliation Agent proceeds from T29; creates Xero credit note; sends confirmation Gmail to internal test address

Xero credit note created in the live organisation matching the Shopify refund amount and customer reference; confirmation email received at internal test address; sheet row status updated to 'Complete'

Xero credit note total matches Shopify refund to the cent; credit note contact matches the test order customer; confirmation email received within 90 seconds; sheet status = 'Complete'; Xero credit note subsequently voided by the FullSpec team

T31

Process owner review and approval

Process owner reviews the complete Google Sheets log row, the Shopify refund record, the Xero credit note (before voiding), the Slack warehouse channel message, and both Gmail outputs for the end-to-end test run

Process owner confirms all outputs are correct, complete, and consistent with the agreed return policy and the outputs described in the Developer Handover Pack. Approval is recorded by the FullSpec team in the FullSpec confirmation log with the reviewer's name, role, and the date of sign-off

Process owner has reviewed and verbally or digitally confirmed all five output artefacts. FullSpec team records name, role, and date in the FullSpec confirmation log. This constitutes the final QA sign-off for the Returns and Reverse Logistics automation. No separate signature block is required.

After T31 is complete, the FullSpec team will void the Xero credit note, cancel the Shopify test refund where the platform permits, archive the Google Sheets test row, and confirm in writing to your team that all test records have been cleared. The automation is then considered QA-complete and ready for live operation. Contact support@gofullspec.com with any questions about the test results or the voiding process.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process