FS-DOC-06Quality
Test and QA Plan
Order Processing and Fulfilment
[YourCompany.com] · Fulfilment Department · Prepared by FullSpec · [Today's Date]
This document defines the structured test and quality assurance programme for the Order Processing and Fulfilment automation, covering both the Order Intake Agent and the Fulfilment Dispatch Agent. It is written for the FullSpec build team and is organised into three sequential phases: unit testing of individual agent logic, integration and edge-case testing across agent handoffs and failure modes, and a final end-to-end production run. No phase begins until the previous phase fully passes. The process owner joins Phase 3 to review results and record formal approval.
01Testing approach
Testing is structured across three phases that must be completed in strict sequence. Each phase has a defined scope, environment, credential rule, and pass condition. A phase is only considered complete when every test case within it reaches a passing state. No phase begins until the previous phase is fully signed off by the FullSpec team lead. The process owner is not required to attend Phase 1 or Phase 2, but must participate in Phase 3 to confirm the production run meets business expectations.
1Phase 1: Unit Testing
Each agent tested in isolation against sandbox data
Scope
Individual agent logic: Order Intake Agent trigger and stock-check routing; Fulfilment Dispatch Agent invoice, courier, tracking, email, and Slack steps tested independently
Environment
Sandbox only. No production credentials used at any point during Phase 1
Credential rule
All API keys must be sandbox or test-mode credentials. Shopify test orders, Cin7 staging environment, Xero demo company, StarShipIt sandbox, Gmail test alias, Slack test workspace
Pass condition
All cases T01 through T12 return the expected output with no unhandled errors
Who runs it
FullSpec team only
2Phase 2: Integration and Edge Case Testing
Agent handoffs and failure scenarios tested in sandbox
Scope
End-to-end agent handoff from Order Intake Agent to Fulfilment Dispatch Agent; duplicate order handling; missing field scenarios; tool unavailability; human timeout on stock-short alerts
Environment
Sandbox only. Integration tests simulate real API responses using sandbox endpoints
Credential rule
Sandbox credentials only. No production API keys may be loaded into the test environment at this stage
Pass condition
All cases T13 through T18 and edge cases E01 through E06 return the expected output or correctly handled error state
Who runs it
FullSpec team only
3Phase 3: End-to-End Production Test
Full production run with live credentials and process owner sign-off
Scope
One complete order journey in production: Shopify paid order through stock check, Xero invoice, StarShipIt booking, Shopify fulfilment update, Gmail customer email, and Slack dispatch notification
Environment
Production. Live credentials active. Test order placed using an internal test address and voided after the run
Credential rule
Production API keys loaded for Shopify, Cin7, Xero, StarShipIt, Gmail, and Slack. Sandbox credentials must be removed before this phase begins
Pass condition
All cases T19 through T22 pass, including process owner review and approval recorded in the FullSpec confirmation (case T22)
Who runs it
FullSpec team executes; process owner joins to review results and record approval
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only. All Phase 1 tests must be executed against sandbox or test-mode API endpoints. Do not load production API keys into the test environment during this phase. Shopify test orders, the Xero demo company, the StarShipIt sandbox account, and a Gmail test alias must be configured before any case below is run.
Order Intake Agent: monitors Shopify for new paid orders, queries Cin7 for stock levels per SKU, and routes the order forward or raises a Slack exception alert.
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: paid order, all SKUs in stock
Shopify test order with 2 line items; Cin7 sandbox shows sufficient stock for both SKUs
Order record validated and passed downstream to Fulfilment Dispatch Agent; no Slack alert raised
Downstream agent receives order payload within 10 seconds; Slack channel shows no alert
T02
Single SKU out of stock
Shopify test order with 2 line items; Cin7 shows zero on-hand for SKU 2
Order held; Slack alert posted to fulfilment channel with order number and short SKU
Slack message received within 30 seconds; order not passed to Fulfilment Dispatch Agent; alert contains correct SKU reference
T03
All SKUs out of stock
Shopify test order with 3 line items; Cin7 shows zero stock for all three SKUs
Order held; Slack alert posted listing all three short SKUs
Slack message lists all three SKUs; no downstream action triggered
T04
Shopify webhook fires for an unpaid order
Shopify order webhook payload with payment_status: pending
Automation ignores the event; no Cin7 query made; no Slack alert
No Cin7 API call logged; no downstream action; workflow exits cleanly
T05
Cin7 API returns a timeout error
Shopify paid order; Cin7 sandbox configured to return a 503 response
Automation retries up to 3 times with exponential back-off; after final failure a Slack alert is posted flagging the Cin7 lookup failure
Retry count logged as 3; Slack alert received; order not silently dropped
T06
Order with missing shipping address fields
Shopify test order where the address2 and phone fields are null
Agent proceeds normally; null optional fields do not block the stock check or downstream routing
Order passes stock check and is forwarded; no error thrown on null optional fields
Fulfilment Dispatch Agent: receives a validated order record, creates the Xero invoice, books the courier via StarShipIt, updates Shopify with the tracking number, sends the Gmail confirmation email, and posts the Slack dispatch notification.
ID
Scenario
Input
Expected Output
Pass Criteria
T07
Happy path: Xero invoice creation
Validated order payload with line items, totals, and customer details
Invoice created in Xero demo company with correct line items, amounts, and customer name; invoice status set to Authorised
Xero invoice ID returned; line item count and totals match Shopify order; status is Authorised
T08
StarShipIt courier booking success
Validated order with destination address, weight, and dimensions; StarShipIt sandbox configured with a single courier rule
Shipment created in StarShipIt sandbox; tracking number returned; label URL available in response
Tracking number is a non-null string; label URL resolves; courier selected matches the configured rule
T09
StarShipIt returns no matching courier service
Order with a destination that matches no configured courier rule in StarShipIt sandbox
Booking fails gracefully; Slack alert posted to fulfilment channel with order number and error reason
No unhandled exception; Slack alert received within 30 seconds; Shopify order not marked fulfilled
T10
Shopify fulfilment update with tracking number
Tracking number and carrier name returned from StarShipIt
Shopify order marked as fulfilled; tracking number and carrier name written to the fulfilment record
Shopify fulfilment status is fulfilled; tracking number in Shopify matches the value from StarShipIt
T11
Gmail shipping confirmation email sent
Customer email address from Shopify order; tracking number from StarShipIt; courier name
Email delivered to the Gmail test alias; subject line includes order number; body contains tracking number and courier name
Email received at test alias within 60 seconds; tracking number in email body matches StarShipIt value; no duplicate send
T12
Slack dispatch notification posted
Order number, customer name, courier name, and tracking reference from upstream steps
Message posted to the configured fulfilment Slack channel containing all four data points
Slack message received in correct channel; all four fields present and accurate; message posted within 10 seconds of Gmail send
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 verifies that the two agents hand off correctly to each other, that data integrity is maintained across the full sandbox chain, and that the system responds correctly to realistic failure and edge-case scenarios. All tests remain in the sandbox environment. IDs continue the T sequence from Phase 1.
ID
Scenario
Input
Expected Output
Pass Criteria
T13
Full sandbox chain: Order Intake Agent to Fulfilment Dispatch Agent
Shopify test order with 2 in-stock SKUs; all sandbox credentials active
Order Intake Agent validates stock, passes record to Fulfilment Dispatch Agent, which creates Xero invoice, books StarShipIt, updates Shopify, sends Gmail, and posts Slack notification
All six downstream actions complete without error; data values are consistent across all systems (order number, customer name, tracking number)
T14
Agent handoff payload integrity
Validated order record with 3 line items, mixed SKUs, and a partial discount
Fulfilment Dispatch Agent receives all fields including discounted totals; Xero invoice reflects the discounted amount
Xero invoice total matches the Shopify order total including discount; no field truncation in the handoff payload
T15
Xero API rate limit hit during invoice creation
Burst of 5 simultaneous test orders triggering 5 concurrent Xero invoice calls
Automation queues requests; retries after the rate limit window; all 5 invoices created without data loss
All 5 Xero invoices created; retry logic logged; no invoice duplicated or skipped
T16
StarShipIt label generation delay
Order submitted to StarShipIt sandbox configured to respond after a 15-second delay
Automation waits for the label response up to a configurable timeout; if timeout is exceeded a Slack alert is raised
Tracking number received if response arrives within timeout; Slack alert raised if timeout is exceeded; order not marked fulfilled prematurely
T17
Gmail send failure due to authentication error
Invalid or expired Gmail OAuth token injected into sandbox credentials
Gmail step fails; error is caught; Slack alert posted to fulfilment channel noting email delivery failure; Shopify fulfilment is still updated
Shopify marked fulfilled with tracking number; Gmail error logged; Slack alert received; no unhandled exception propagated
T18
Slack notification failure during dispatch
Slack webhook URL configured as invalid in sandbox
Slack post fails silently; upstream steps (Xero, StarShipIt, Shopify, Gmail) are not rolled back; error is logged
All upstream steps completed; error log entry present for Slack failure; no upstream data corruption
Edge case tests cover scenarios that fall outside the happy path and are most likely to surface in production. These are tested in the sandbox environment and use the E prefix to distinguish them from the main sequence.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate Shopify webhook received for the same order
Two identical paid order webhook payloads with the same Shopify order ID fired within 5 seconds
Automation processes the first payload and deduplicates the second using the order ID; only one Xero invoice and one StarShipIt booking created
Xero contains exactly one invoice for the order; StarShipIt has exactly one shipment; deduplication log entry present
E02
Order payload missing customer email address
Shopify test order where the customer email field is null or empty
Stock check and Xero invoice proceed normally; Gmail step is skipped with an error log entry; Slack alert raised noting missing customer email
Xero invoice created; StarShipIt booked; Shopify updated; Gmail step skipped without crashing; Slack alert received
E03
Cin7 returns a stock level of zero for a SKU that was in stock at order placement
Shopify paid order placed; Cin7 sandbox updated to zero stock between order placement and stock-check query
Order Intake Agent detects zero stock, holds the order, and raises a Slack alert; Fulfilment Dispatch Agent is not triggered
Slack alert received with correct SKU and order number; no downstream fulfilment actions taken
E04
StarShipIt API is completely unavailable
StarShipIt sandbox endpoint returns a connection refused error
Automation retries 3 times; after final failure the order is held and a Slack alert is posted; Xero invoice is not rolled back
3 retry attempts logged; Slack alert received; Xero invoice retained; Shopify order not marked fulfilled; error log entry present
E05
Human reviewer does not action a stock-short Slack alert within 2 hours
Stock-short Slack alert sent; no human response or manual override within the defined timeout window
Automation sends a follow-up Slack escalation alert after the timeout; order remains held; no automatic resolution attempted
Escalation alert posted after timeout; order still in held state; original alert not cleared automatically
E06
Order with an unrecognised or non-standard SKU format
Shopify test order containing a SKU that does not match any record in the Cin7 sandbox product catalogue
Cin7 lookup returns no match; automation treats this as a stock-short scenario; Slack alert raised with the unrecognised SKU value
Slack alert received noting the unknown SKU; order held; no crash or unhandled exception; SKU value correctly surfaced in the alert message
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: Use an internal test email address as the customer email for all production test orders. Confirm that the Shopify test order is placed to an internal or dummy delivery address. Void the Xero invoice and cancel the StarShipIt shipment immediately after each test run. Confirm with the operations manager that no courier will be dispatched for the test order. Do not proceed with Phase 3 unless Phase 1 and Phase 2 are fully passed and documented.
Phase 3 uses live production credentials across all six connected tools: Shopify, Cin7, Xero, StarShipIt, Gmail, and Slack. A real paid order is placed using an internal test account and traced through every automated step. Results are reviewed by the FullSpec team and then presented to the process owner for final approval. The process owner's sign-off is recorded in the FullSpec confirmation system as the final case in this table.
ID
Scenario
Input
Expected Output
Pass Criteria
T19
Full production run: paid order to dispatch notification
A real paid Shopify order placed via an internal test account with a single in-stock SKU; delivery address set to an internal address; customer email set to an internal test alias
Order Intake Agent queries Cin7 production stock, confirms availability, and passes the record to Fulfilment Dispatch Agent; Xero invoice created in production; StarShipIt shipment booked in production; Shopify order marked fulfilled with tracking number; Gmail confirmation email delivered to the internal test alias; Slack dispatch notification posted to the fulfilment channel
All six steps complete without error within 3 minutes of order placement; data values are consistent across all production systems; no duplicate records created
T20
Error log verification across the full production run
Execution log from the T19 production run
Log contains a timestamped entry for every step: Shopify webhook receipt, Cin7 stock query, Xero invoice creation, StarShipIt booking, Shopify fulfilment update, Gmail send, and Slack post; no error or warning entries present
All 7 log entries present with timestamps; no error-level entries; execution time per step within acceptable thresholds (Cin7 and Xero under 5 seconds; StarShipIt under 15 seconds; Gmail under 30 seconds)
T21
Post-run cleanup verification
Xero invoice and StarShipIt shipment created during T19
Xero invoice voided; StarShipIt shipment cancelled; Shopify test order archived; no live courier booking resulting from the test run
Xero invoice status is Voided; StarShipIt shipment shows Cancelled; operations manager confirms no courier was dispatched; Cin7 stock level restored to pre-test value if deducted
T22
Process owner review and approval
Full Phase 3 test results summary including T19 execution log, T20 log review, and T21 cleanup confirmation, presented to the process owner
Process owner reviews results, confirms the automation meets the business requirements for the Order Processing and Fulfilment process, and records approval in the FullSpec confirmation system
Process owner approval recorded in FullSpec confirmation system with name, date, and outcome noted; no outstanding issues flagged; automation cleared for go-live
Once T22 is recorded in the FullSpec confirmation system, the automation is cleared for go-live. The FullSpec team will decommission the sandbox credentials, activate production monitoring alerts, and hand over the SOP and runbook to the operations team. For any questions during or after Phase 3, contact the FullSpec team at support@gofullspec.com.
Test and QA PlanPage 4 of 4