FS-DOC-06Quality
Test and QA Plan
Order Processing and Fulfilment Automation
[YourCompany.com] · Fulfilment Department · Prepared by FullSpec · [Today's Date]
This document defines the complete test and quality assurance plan for the Order Processing and Fulfilment automation, covering all three agents: Order Intake, Dispatch, and Finance. Testing is structured across three sequential phases. Unit tests validate each agent in isolation using sandbox credentials; integration tests confirm agent handoffs and edge-case resilience; and end-to-end tests run the full automation in production against live data. The FullSpec team designs, executes, and records all phases. The process owner joins Phase 3 to confirm the automation meets the operational standard agreed at discovery, and that sign-off is recorded in the FullSpec confirmation log.
01Testing approach
Testing is structured in three phases that must be completed in strict sequence. No phase begins until every test case in the preceding phase carries a pass status. A single failing case blocks progression. This sequencing ensures that integration faults are not masked by unit-level failures and that production runs are only attempted once both agent isolation and handoff behaviour have been independently verified.
1Phase 1: Unit Testing
Each agent tested in isolation against sandbox environments
Scope
Individual agent logic: Order Intake Agent, Dispatch Agent, Finance Agent. Each agent tested independently with mocked upstream and downstream tool responses.
Environment
Sandbox only. Shopify development store, ShipStation sandbox account, Linnworks staging environment, Xero demo company, Slack test workspace, Gmail test account.
Credential rule
No production API keys may be used in Phase 1. All credentials must be sandbox-scoped and stored in the Make sandbox scenario environment, separate from production variables.
Pass condition
All test cases T01 through T15 return the expected output with no unhandled errors. Any error must be caught by the agent's own error handler and produce the correct fallback behaviour.
Who runs it
FullSpec team only. Process owner is not required for Phase 1.
2Phase 2: Integration and Edge Case Testing
Agent handoffs, data flow across tool boundaries, and failure resilience
Scope
End-to-end agent handoff sequences: Order Intake to Dispatch, Dispatch to Linnworks stock deduction, wholesale tag detection to Finance Agent. Edge cases including duplicate orders, missing data fields, unavailable tools, and human-timeout scenarios.
Environment
Sandbox for agent logic; staging webhooks pointed at the Make staging scenario. Tool unavailability is simulated by temporarily disabling sandbox API keys during execution.
Credential rule
Sandbox credentials only. ShipStation and Linnworks staging tokens must have write-access scoped to test accounts only. No customer-facing records are touched.
Pass condition
All integration cases T16 through T22 pass and all edge cases E01 through E08 produce the correct fallback or alert behaviour. Zero unhandled exceptions in the Make scenario execution log.
Who runs it
FullSpec team only. Process owner is not required for Phase 2.
3Phase 3: End-to-End Production Test
Full live run traced from Shopify webhook to customer email and Xero draft invoice
Scope
Complete automation run in the production Make scenario using live API credentials. Covers the standard order flow, out-of-stock branch, wholesale invoice creation, and error log verification.
Environment
Production. Live Shopify store, live ShipStation account, live Linnworks, live Xero, live Slack workspace, live Gmail sender. Test orders are placed using an internal test account and voided after each run.
Credential rule
Production credentials are used. All test orders must be placed from an internal test customer account, not a real customer address. Test shipments must not be physically dispatched. Test records must be voided or cancelled immediately after each case is confirmed.
Pass condition
Test cases T23 through T26 all pass, including T26 (process owner review and approval). Process owner confirms the automation output matches expected operational behaviour and sign-off is recorded in the FullSpec confirmation log.
Who runs it
FullSpec team runs T23 through T25. Process owner joins for T26 to review outputs and record approval.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only for all Phase 1 tests. No production API keys, no live Shopify store, no real ShipStation shipments, no real Xero records, no real customer emails. Use the dedicated Make sandbox scenario with environment variables prefixed SANDBOX_. Any test that inadvertently touches a production record must be voided immediately and reported to the FullSpec team lead before continuing.
Order Intake Agent — complexity: Moderate. Estimated build time: 14 hours.
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: all SKUs in stock
Shopify sandbox webhook payload, 2-line order, all SKUs available in Linnworks staging
Order record flagged as ready for dispatch; no Slack alert sent
Dispatch flag set to true; Slack API call count = 0; execution log shows no errors
T02
Single SKU out of stock
Shopify sandbox payload, 2-line order, 1 SKU at 0 quantity in Linnworks staging
Order flagged as out-of-stock; Slack alert posted with order ID and affected SKU
Dispatch flag set to false; Slack message body contains order ID and correct SKU reference; Linnworks not decremented
T03
All SKUs out of stock
Shopify sandbox payload, 1-line order, SKU quantity 0 in Linnworks staging
Order flagged as out-of-stock; Slack alert posted; no dispatch action initiated
Dispatch flag false; Slack alert contains correct order ID; agent terminates cleanly after alert
T04
Shopify webhook payload missing shipping address
Sandbox payload with customer.shipping_address omitted
Agent catches missing field; posts error alert to Slack; order not passed to Dispatch Agent
Error handler triggered; Slack error message sent; Make execution shows handled error, not unhandled exception
T05
Linnworks API returns 503 (unavailable)
Valid Shopify sandbox payload; Linnworks sandbox endpoint forced to return HTTP 503
Agent retries up to 3 times with exponential backoff; after 3 failures posts Slack alert and halts
Make execution log shows 3 retry attempts; Slack alert posted after final failure; order not passed to Dispatch Agent
T06
Order with zero line items
Shopify sandbox payload with empty line_items array
Agent detects empty line items; posts error alert to Slack; terminates without Linnworks query
No Linnworks API call made; Slack error alert sent; execution log shows clean handled termination
Dispatch Agent — complexity: Complex. Estimated build time: 20 hours.
ID
Scenario
Input
Expected Output
Pass Criteria
T07
Happy path: standard consumer order dispatched
Verified order record from Order Intake Agent (all in stock); consumer address; standard service level
ShipStation shipment created; tracking number returned; Shopify order marked fulfilled; Gmail confirmation sent; Linnworks stock decremented
All five downstream actions confirmed in Make execution log; ShipStation sandbox shipment ID present; Shopify fulfilment status = fulfilled; Gmail sandbox outbox shows one sent message; Linnworks quantity reduced by correct amount
T08
Expedited service level selected at checkout
Verified order record with shipping method mapped to express service level in ShipStation
ShipStation shipment uses correct express carrier and service code; tracking number returned
ShipStation sandbox record shows correct service_code matching the express mapping table; no standard-rate shipment created
T09
ShipStation returns error on shipment creation
Valid verified order; ShipStation sandbox forced to return HTTP 422 (unprocessable entity)
Agent catches error; posts structured Slack alert to fulfilment channel; does not write tracking to Shopify; does not send Gmail email; does not decrement Linnworks
No Shopify fulfilment update; no Gmail send; no Linnworks deduction; Slack alert contains order ID and ShipStation error code; Make execution log shows handled error
T10
Gmail send fails (SMTP error)
Valid verified order; ShipStation sandbox returns tracking number; Gmail sandbox forced to return send error
ShipStation and Shopify steps complete successfully; agent retries Gmail once; on second failure posts Slack alert with order ID; Linnworks still decremented
Shopify fulfilment status = fulfilled; Linnworks decremented; Gmail retry logged; Slack alert sent; execution does not terminate unhandled
T11
Linnworks stock deduction fails after dispatch
Valid verified order; ShipStation and Shopify steps succeed; Linnworks sandbox returns 500 on deduction call
Agent posts Slack alert flagging manual stock deduction required for the order ID; does not retry indefinitely
Slack alert contains order ID and SKU quantities needing manual deduction; Make execution log shows handled error; no infinite loop
T12
Tracking number missing from ShipStation response
Valid verified order; ShipStation sandbox returns shipment ID but omits tracking_number field
Agent catches missing tracking number; does not update Shopify; does not send Gmail; posts Slack alert
No Shopify fulfilment update; no Gmail send; Slack alert sent with order ID and description of missing field
Finance Agent — complexity: Simple. Estimated build time: 8 hours.
ID
Scenario
Input
Expected Output
Pass Criteria
T13
Happy path: wholesale order creates Xero draft invoice
Shopify sandbox order tagged wholesale; 3 line items with quantities and unit prices; Shopify order ID present
Draft invoice created in Xero demo company with matching line items, amounts, and Shopify order ID reference
Xero demo company shows one new draft invoice; line item count = 3; totals match Shopify order; invoice status = DRAFT; Shopify order ID in invoice reference field
T14
Order tagged B2B (not wholesale) creates Xero draft invoice
Shopify sandbox order tagged b2b; 2 line items
Draft invoice created in Xero demo company identical to T13 behaviour
Xero demo company shows one new draft invoice matching the B2B order; agent treats b2b tag identically to wholesale tag
T15
Consumer order (no wholesale or B2B tag) does not trigger Finance Agent
Shopify sandbox order with no wholesale or b2b tag
Finance Agent not triggered; no Xero invoice created
Xero demo company invoice count unchanged; Make execution log shows agent branched away from Finance Agent route; no Xero API call made
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 integration tests confirm that data passes correctly across agent boundaries and that the full Make scenario routes orders through the correct branches. All tests use sandbox environments. IDs continue from Phase 1.
ID
Scenario
Input
Expected Output
Pass Criteria
T16
Order Intake to Dispatch Agent handoff: standard in-stock order
Shopify sandbox webhook triggers full Make scenario; all SKUs available in Linnworks staging
Order Intake Agent passes dispatch-ready payload to Dispatch Agent; Dispatch Agent creates ShipStation shipment and completes all downstream steps
Make execution log shows clean module-to-module handoff; all five Dispatch Agent outputs confirmed; no data loss between agents
T17
Order Intake Agent halts on out-of-stock: Dispatch Agent not triggered
Shopify sandbox webhook; 1 SKU out of stock in Linnworks staging
Order Intake Agent posts Slack alert and terminates; Dispatch Agent route is not entered
Make execution log shows Dispatch Agent module not executed; Slack alert sent; ShipStation sandbox shows no new shipment
T18
Dispatch Agent to Linnworks stock deduction handoff
Verified in-stock order; ShipStation sandbox returns tracking number
Dispatch Agent writes tracking to Shopify, sends Gmail email, then decrements Linnworks. All three steps complete in correct sequence.
Make module execution order confirmed in log: Shopify fulfilment before Gmail before Linnworks deduction; all three sandbox records updated
T19
Wholesale order: Dispatch Agent to Finance Agent handoff
In-stock wholesale-tagged Shopify sandbox order processed through full scenario
Dispatch Agent completes all fulfilment steps; Finance Agent then creates Xero draft invoice referencing the same Shopify order ID
Make log shows Finance Agent triggered after Dispatch Agent completion; Xero demo invoice references correct Shopify order ID; no duplication of steps
T20
Consumer order: Finance Agent not triggered after Dispatch Agent
In-stock consumer Shopify sandbox order with no wholesale or b2b tag, processed through full scenario
Dispatch Agent completes all fulfilment steps; Finance Agent route not entered; no Xero invoice created
Make log shows correct branch taken after wholesale tag check; Xero demo company unchanged; execution terminates cleanly
T21
Concurrent order processing: two orders arrive within 5 seconds
Two Shopify sandbox webhooks fired with a 4-second interval; both orders have different SKUs; both in stock
Both orders processed independently and completely; no data cross-contamination between the two execution threads
Make execution log shows two separate scenario runs; each run's ShipStation shipment, Shopify update, Gmail send, and Linnworks deduction reference the correct order ID; no fields from order A appear in order B's records
T22
Concurrent low-stock scenario: two orders for the same SKU arrive within 5 seconds; only 1 unit in stock
Two Shopify sandbox webhooks for the same SKU; Linnworks staging quantity = 1
First order passes stock check and proceeds to dispatch; second order fails stock check and triggers Slack out-of-stock alert
Only one ShipStation shipment created; Linnworks quantity after first deduction = 0; Slack alert fired for second order citing 0 available; no oversell in Linnworks staging record
Edge case tests below use a separate ID sequence (E01 onwards) and cover conditions that fall outside the normal order flow. All are run in sandbox.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate Shopify webhook for the same order ID
Same Shopify sandbox order webhook fired twice within 10 seconds (Shopify delivery retry simulation)
Scenario detects duplicate order ID; second execution is skipped or de-duplicated; no duplicate ShipStation shipment or Xero invoice created
Make execution log shows idempotency check triggered on second webhook; only one ShipStation record per order ID in sandbox; Xero demo company shows one invoice per order ID
E02
Order payload missing customer email address
Shopify sandbox payload with customer.email field null or empty string
Order Intake Agent catches missing email; flags order with error note; Slack alert sent; Gmail step not attempted
No Gmail API call made; Slack alert contains order ID and description of missing email field; Make execution log shows handled error
E03
Order payload contains a SKU not present in Linnworks at all (unknown SKU)
Shopify sandbox payload with a line item SKU that returns a 404 from Linnworks staging
Order Intake Agent treats unknown SKU as out-of-stock; Slack alert sent with order ID and unrecognised SKU identifier
Dispatch Agent not triggered; Slack alert body contains SKU string; Linnworks returns 404 gracefully handled; no unhandled exception in Make log
E04
ShipStation unavailable for more than the retry window (all retries exhausted)
Valid verified order; ShipStation sandbox endpoint disabled for duration of test
Agent retries 3 times with exponential backoff (5 s, 15 s, 45 s); after third failure posts detailed Slack alert to fulfilment channel; order not marked fulfilled in Shopify; Linnworks not decremented
Make log shows exactly 3 HTTP attempts to ShipStation; timestamps confirm backoff intervals; Slack alert sent after third failure; Shopify fulfilment status unchanged; Linnworks quantity unchanged
E05
Xero API token expired during Finance Agent execution
Wholesale-tagged sandbox order triggering Finance Agent; Xero OAuth access token forced to expire mid-execution
Finance Agent detects 401 from Xero; attempts token refresh using stored refresh token; if refresh succeeds, invoice creation continues; if refresh fails, Slack alert sent
Make log shows OAuth refresh attempt; if refresh succeeds, Xero demo invoice created and log shows no error; if refresh fails, Slack alert sent and execution terminates cleanly with no unhandled exception
E06
Wholesale order with a zero-price line item (discount or sample line)
Shopify sandbox wholesale-tagged order containing one line item with unit_price = 0
Finance Agent creates Xero draft invoice including the zero-price line item without error; invoice total reflects only the non-zero lines
Xero demo invoice contains all line items including the zero-price row; invoice total is mathematically correct; no Finance Agent error or Slack alert
E07
Slack API unavailable when out-of-stock alert should fire
Shopify sandbox order with out-of-stock SKU; Slack sandbox API endpoint forced to return 503
Order Intake Agent catches Slack failure; logs error to Make execution log; order remains flagged as out-of-stock and is not passed to Dispatch Agent
Make log contains Slack error entry; Dispatch Agent not triggered; no ShipStation shipment created; execution does not terminate unhandled
E08
Shopify webhook arrives with no line_items array key at all (malformed payload)
Sandbox webhook payload JSON with line_items key entirely absent
Order Intake Agent schema validation catches missing key before any Linnworks query; posts Slack error alert; terminates cleanly
No Linnworks API call; no Dispatch Agent trigger; Slack error alert sent; Make execution log shows handled validation error at the first module
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: All test orders must be placed using an internal test customer account with a test email address that is monitored by the FullSpec team. Do not use a real customer name, real customer email, or a real shipping address that could result in a parcel being physically dispatched. Every test record, including Shopify orders, ShipStation shipments, Linnworks stock movements, and Xero draft invoices, must be voided or cancelled by the FullSpec team immediately after each test case is confirmed as passed. Failure to void test records will corrupt live inventory levels and financial records.
Phase 3 runs the automation in the live production Make scenario using live API credentials. Each case below is executed by the FullSpec team and the results are recorded in the FullSpec confirmation log. The process owner joins for T26, the final approval case, and their sign-off is captured in the same log. No separate signature block is required.
ID
Scenario
Steps Traced
Expected Production Outcome
Pass Criteria
T23
Full production run: standard consumer order, all items in stock
Shopify live webhook fires on test order creation. Order Intake Agent queries Linnworks live. Dispatch Agent creates ShipStation live shipment. Tracking number written back to Shopify. Gmail confirmation sent to test email address. Linnworks live stock decremented. Wholesale tag absent: Finance Agent not triggered.
Test order appears as fulfilled in Shopify within 90 seconds of placement. Tracking number present on Shopify order detail. Test email inbox receives confirmation email with correct tracking link. Linnworks live stock reduced by correct quantities. ShipStation live account shows one new shipment. No Xero invoice created.
All six expected production records confirmed: Shopify fulfilment status, ShipStation shipment, Gmail delivery to test inbox, Linnworks quantity update, no Xero invoice, Make execution log shows zero unhandled errors. All records voided after confirmation.
T24
Full production run: out-of-stock branch
Shopify live webhook fires on test order creation with a SKU temporarily set to 0 in Linnworks live. Order Intake Agent detects 0 quantity. Slack alert posted to fulfilment channel. Dispatch Agent not triggered.
Slack fulfilment channel receives structured alert within 60 seconds of order placement. Alert contains correct order ID and SKU. Order not marked fulfilled in Shopify. No ShipStation shipment created. No Gmail confirmation sent.
Slack alert received in production fulfilment channel; alert body verified for order ID and SKU accuracy. Shopify order remains unfulfilled. Make execution log shows Dispatch Agent module not entered. Linnworks SKU quantity restored to previous value after test. All records voided after confirmation.
T25
Full production run: wholesale order triggers Finance Agent and creates Xero draft invoice
Shopify live webhook fires on test wholesale-tagged order. Order Intake Agent confirms all SKUs in stock. Dispatch Agent completes all fulfilment steps. Finance Agent detects wholesale tag. Xero draft invoice created with correct line items and Shopify order ID reference.
All standard fulfilment outcomes from T23 confirmed. Additionally: Xero live account shows one new draft invoice in DRAFT status. Invoice line items and totals match the Shopify test order exactly. Invoice reference field contains the correct Shopify order ID.
All T23 pass criteria met. Xero draft invoice present with correct status, line items, totals, and order reference. Make execution log shows Finance Agent triggered after Dispatch Agent completion. No duplicate invoices. All records including Xero draft invoice voided after confirmation.
T26
Process owner review and approval
Process owner (Operations Manager) reviews the Make execution logs, Slack alerts, Shopify fulfilment records, ShipStation shipment records, Gmail test inbox, Linnworks stock movements, and Xero draft invoice from T23, T24, and T25 in sequence.
Process owner confirms that the automation outputs match the operational standard agreed at discovery: order-to-tracking time under 5 minutes, correct courier service level applied, out-of-stock alerts actionable and accurate, Xero draft invoice correctly structured for approval, and no unhandled errors in any execution.
Process owner records a named approval in the FullSpec confirmation log, including the date, their name, and a written confirmation that all three agents have been verified against the agreed acceptance criteria. This entry constitutes the Phase 3 sign-off. No separate signature block is required. FullSpec archives the confirmation log entry against this project record.
Once T26 is recorded as approved in the FullSpec confirmation log, the automation is cleared for full production operation. Any issues discovered post-approval are handled under the FullSpec support process. Contact the FullSpec team at support@gofullspec.com for any post-launch queries, error investigations, or change requests.
Test and QA PlanPage 4 of 4