Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Order Processing and Fulfilment Automation

[YourCompany.com] · Fulfilment Department · Prepared by FullSpec · [Today's Date]

This document defines the complete test and quality assurance plan for the Order Processing and Fulfilment automation, covering all three agents: Order Routing, Label and Despatch, and Customer Notification. The FullSpec team runs every test case in Phases 1 and 2; the process owner joins the FullSpec team for Phase 3 production validation. No phase begins until the previous phase has fully passed. All test results are recorded in the FullSpec confirmation log, which serves as the sign-off record for go-live approval.

01Testing approach

Testing runs in three sequential phases: Unit, Integration, and End-to-end. Each phase must reach a full pass state before the next phase begins. A single failing test case in any phase blocks progression until the defect is resolved and the affected case is re-run and passed. This structure ensures that individual components are verified before agent handoffs are tested, and handoffs are verified before any production data is touched.

Phase 1: Unit Testing

Verify each agent in isolation

Scope

Each agent tested independently: Order Routing Agent, Label and Despatch Agent, Customer Notification Agent. No cross-agent handoffs tested in this phase.

Environment

Sandbox only. No production credentials, no live Shopify orders, no real ShipStation labels, no Klaviyo sends to real customers.

Credential rule

All sandbox API keys must be loaded. Production keys must be absent from the Make scenario environment. Verified before any test run begins.

Pass condition

All test cases T01 through T15 return the expected output with no errors logged. Zero failures permitted before Phase 2 may begin.

Who runs it

FullSpec team only.

Phase 2: Integration and Edge Case Testing

Verify agent handoffs and exception scenarios

Scope

Full agent-to-agent handoffs across the three-agent chain, plus structured edge cases: duplicate orders, missing data fields, tool unavailability, and human-timeout scenarios for out-of-stock exceptions.

Environment

Sandbox environment for all integration runs. Shopify test orders used throughout. Slack alerts fire to a dedicated test channel, not the live fulfilment channel.

Credential rule

Sandbox credentials only. Klaviyo test-mode flag must be active so no real emails are delivered to customer addresses.

Pass condition

All integration test cases T16 through T24 pass, and all edge cases E01 through E07 produce the correct exception behaviour. Zero unhandled errors permitted.

Who runs it

FullSpec team only.

Phase 3: End-to-End Production Test

Validate the full flow against live systems

Scope

Full end-to-end flow using real production credentials: live Shopify webhook, Linnworks stock check, ShipStation label creation, Shopify fulfilment write-back, Klaviyo tracking email, and Xero cost entry.

Environment

Production. A dedicated internal test address and a zero-cost test SKU are used. All test records are voided after each run.

Credential rule

Production credentials active. FullSpec confirms all keys are scoped correctly before Phase 3 begins. No sandbox tokens may remain in the production scenario.

Pass condition

All test cases T25 through T29 pass, including process owner review and approval (T29). FullSpec records the outcome in the confirmation log.

Who runs it

FullSpec team leads all cases. Process owner joins for T29 (review and approval).

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only: before running any case in Phase 1, confirm that all Make scenario connections use sandbox or test-mode API keys. Production keys must not be present. No real orders, no real labels, and no real emails will be generated during this phase.

Order Routing Agent — Shopify, Linnworks, Slack. Estimated build time: 20 hours. Complexity: Moderate.

Scenario

Input

Expected output

Pass criteria

T01

Happy path: all SKUs in stock

Shopify test order with 2 line items; Linnworks sandbox returns sufficient stock for both SKUs

Order routed to automated fulfilment path; pick list created in Linnworks sandbox; no Slack alert fired

Linnworks pick list record created; Slack test channel shows no message; Make execution log shows no errors

T02

All SKUs out of stock

Shopify test order; Linnworks sandbox returns 0 quantity for all line items

Order not routed to fulfilment; Slack alert posted to test channel with order ID and SKU details

Slack test channel receives alert within 30 seconds; alert contains correct order ID and all affected SKUs; no label triggered

T03

Partial stock: one SKU in stock, one out of stock

Shopify test order with 2 line items; Linnworks returns stock for SKU-A, zero for SKU-B

Exception path triggered; Slack alert posted for SKU-B; order held pending manual decision

Slack alert references SKU-B specifically; automated fulfilment path not entered; Make log shows correct branch taken

T04

Shopify webhook payload missing shipping address

Shopify test webhook with shipping_address field omitted

Agent captures error; execution halts gracefully; error logged in Make scenario history

Make scenario shows error step at address-parse module; no downstream action triggered; error message is human-readable

T05

Linnworks API returns timeout

Shopify test order; Linnworks sandbox endpoint set to simulate 30-second timeout

Retry logic fires up to 3 times; after final timeout, Slack alert posted flagging API unavailability

Make execution log shows 3 retry attempts; Slack test channel receives timeout alert with order ID; no silent failure

T06

SKU in Shopify order not found in Linnworks

Shopify test order with SKU that has no matching record in Linnworks sandbox

Exception raised; Slack alert posted with order ID and unmatched SKU; fulfilment path not entered

Slack alert contains the unmatched SKU string; Make log shows SKU-lookup error step; no label creation attempted

Label and Despatch Agent — ShipStation, Shopify, Xero. Estimated build time: 22 hours. Complexity: Moderate.

Scenario

Input

Expected output

Pass criteria

T07

Happy path: standard domestic order, carrier rule matched

Stock-confirmed order record; weight and destination match a pre-configured ShipStation carrier rule

Label PDF generated in ShipStation sandbox; tracking number returned; Shopify order marked Fulfilled; Xero sandbox receives cost entry

ShipStation sandbox shows new label; Shopify test order status is Fulfilled with tracking number populated; Xero sandbox shows matching line item

T08

No carrier rule matches order weight and destination

Order with weight or destination zone outside all configured ShipStation rules

Agent raises exception; Slack alert posted to test channel; no label created

ShipStation sandbox shows no new label; Slack alert received with order ID and reason; Make log shows routing-failure step

T09

ShipStation API returns 429 rate-limit error

Valid order input; ShipStation sandbox configured to return HTTP 429 on first request

Make retries after back-off delay (60 seconds); second attempt succeeds; label created normally

Make log shows retry after 60-second delay; label created on second attempt; no error surfaced to end state

T10

Xero cost-entry write fails (API error)

Valid label created in ShipStation; Xero sandbox endpoint returns 500 error

Shopify fulfilment write-back still completes; Xero failure logged; alert posted to Slack for manual Xero entry

Shopify order is Fulfilled with tracking number; Make log shows Xero error; Slack alert references order ID and shipping cost for manual entry

T11

Multi-line order with mixed carrier services required

Order with 3 line items requiring 2 different carrier services (e.g. domestic standard and oversized)

Agent selects the highest-priority matching carrier service for the combined shipment per configured rule hierarchy

Single label created using the correct rule; Make log shows rule-selection logic applied; no duplicate labels

T12

Tracking number write-back to Shopify fails

Label created in ShipStation; Shopify sandbox API returns error on fulfilment update

Retry attempted up to 3 times; after failure, Slack alert sent with tracking number for manual Shopify update

Slack alert contains the tracking number string; Shopify order status remains Unfulfilled in sandbox; Make log shows retry sequence

Customer Notification Agent — Klaviyo, Shopify. Estimated build time: 12 hours. Complexity: Simple.

Scenario

Input

Expected output

Pass criteria

T13

Happy path: Shopify fulfilment event triggers tracking email

Shopify order status updated to Fulfilled with tracking number and carrier name present

Klaviyo test-mode send fires within 30 seconds; email contains tracking link and carrier name; no real customer email delivered

Klaviyo activity log shows test send event; tracking URL in email body resolves correctly; carrier name matches ShipStation output

T14

Fulfilment event received but tracking number field is empty

Shopify fulfilment webhook with tracking_number field null or empty string

Agent catches missing field; Klaviyo send suppressed; Slack alert posted flagging missing tracking number

No Klaviyo send event in activity log; Slack test channel receives alert with order ID; Make log shows suppression step

T15

Klaviyo API returns authentication error

Valid fulfilment event; Klaviyo sandbox API key deliberately invalidated

Agent logs authentication error; Slack alert posted; email send not silently skipped

Make log shows Klaviyo 401 error; Slack alert received referencing order ID; no silent pass-through

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests verify that data passes correctly across agent boundaries. All three agents run in sequence in sandbox mode. The Slack test channel, Linnworks sandbox, ShipStation sandbox, Klaviyo test mode, and Xero sandbox must all be active before these cases run.

Scenario

Input

Expected output

Pass criteria

T16

Full in-stock flow: Order Routing to Label and Despatch handoff

Shopify test order; all SKUs in stock in Linnworks sandbox

Order Routing Agent confirms stock and passes routed order record to Label and Despatch Agent; label created without manual trigger

Label and Despatch Agent execution begins automatically following Order Routing Agent completion; no gap or manual step in Make scenario log

T17

Full in-stock flow: Label and Despatch to Customer Notification handoff

Stock-confirmed order; label created in ShipStation sandbox; Shopify order marked Fulfilled

Customer Notification Agent triggers on Shopify fulfilment event; Klaviyo test send fires with correct tracking number

Klaviyo activity log shows send event within 30 seconds of Shopify fulfilment update; tracking number in email matches ShipStation output

T18

Full three-agent chain: end-to-end sandbox run

Shopify test order from webhook through to Xero cost entry

All three agents execute in correct sequence; Linnworks pick list, ShipStation label, Shopify fulfilment, Klaviyo test send, and Xero line item all created

All five downstream records exist in their respective sandbox environments; total elapsed time from webhook to Klaviyo send is under 90 seconds

T19

Out-of-stock order: Label and Despatch Agent never triggered

Shopify test order; Linnworks returns zero stock for one SKU

Order Routing Agent posts Slack alert; Label and Despatch Agent does not execute; no label created

ShipStation sandbox shows no new label; Make execution log shows fulfilment branch not entered; Slack alert received

T20

High-volume burst: 15 orders fired in rapid succession

15 Shopify test webhook events fired within 60 seconds

All 15 orders processed without data collision; each produces its own label, fulfilment update, and Klaviyo test send

15 distinct label records in ShipStation sandbox; 15 Shopify test orders marked Fulfilled with unique tracking numbers; no orders merged or skipped

T21

Xero cost entry matches ShipStation label cost exactly

Standard test order with a known shipping cost returned by ShipStation sandbox ($7.50)

Xero sandbox receives line item for $7.50 against the correct sales order reference

Xero sandbox line item amount is $7.50; sales order reference matches the Shopify order number; no rounding or currency error

T22

Make scenario mid-run failure recovery

Full order flow initiated; Make module for Shopify fulfilment write-back set to fail on first attempt

Make retries the failed module; on success, downstream agents continue from that point without re-running completed steps

Make log shows retry for Shopify write-back only; prior steps (label creation) not duplicated; Klaviyo send and Xero entry complete normally after recovery

T23

Slack exception alert contains all required fields

Out-of-stock test order with 2 flagged SKUs

Slack alert in test channel includes: order ID, customer name, flagged SKU codes, quantities ordered, and a link to the Shopify order

All five data fields present in the Slack message; Shopify order URL resolves correctly; no truncated or missing values

T24

Linnworks pick list zone-sorting applied correctly

Test order with 4 SKUs mapped to 3 different warehouse zones in Linnworks sandbox

Pick list in Linnworks groups SKUs by zone in ascending zone order

Linnworks sandbox pick list shows zone grouping; SKUs within each zone listed together; no mixed-zone rows

The following edge cases test the automation's resilience to abnormal inputs and external failures. These cases run in the same sandbox environment as the integration tests above.

Scenario

Input

Expected output

Pass criteria

E01

Duplicate Shopify webhook: same order ID received twice within 60 seconds

Two identical Shopify paid-order webhook payloads with the same order_id fired 10 seconds apart

Idempotency check detects duplicate; second webhook discarded; only one label created

ShipStation sandbox shows exactly one label for the order ID; Make log shows duplicate-detection step on second run; no double-charge in Xero sandbox

E02

Order line item has no SKU value (null or blank)

Shopify test order where one line item has an empty sku field

Order Routing Agent catches missing SKU; exception raised; Slack alert posted; fulfilment path not entered

Slack alert references the line item by product title and flags missing SKU; no Linnworks query attempted for the blank SKU; Make log shows validation-failure step

E03

Linnworks API unavailable for more than 5 minutes

Shopify test order received; Linnworks sandbox endpoint set to return 503 for 6 minutes

Order held in Make queue; retries at 1-minute intervals; after 5 failed attempts, Slack alert posted flagging Linnworks unavailability with order ID

Make log shows 5 retry attempts at 60-second intervals; Slack alert received after fifth failure; order not silently dropped

E04

ShipStation returns a label for the wrong carrier service

Test order where ShipStation sandbox returns a label with a carrier_code not matching the pre-configured rule for the order's weight and zone

Label and Despatch Agent detects carrier_code mismatch; label voided in ShipStation sandbox; Slack alert posted for manual carrier selection

ShipStation sandbox shows voided label; Slack alert contains order ID and the mismatched carrier code; no Shopify fulfilment write-back triggered

E05

Customer email address missing from Shopify order

Shopify fulfilment event with contact_email field null

Customer Notification Agent suppresses Klaviyo send; Slack alert posted with order ID flagging missing email address

No Klaviyo activity log entry for the order; Slack alert received referencing the order ID; Make log shows email-validation failure step

E06

Fulfilment manager does not respond to out-of-stock Slack alert within 4 hours

Out-of-stock exception Slack alert posted; no human response or order action taken within 4-hour window

Make scenario sends a follow-up Slack reminder to the fulfilment manager channel after 4 hours; order remains on hold

Slack test channel receives follow-up reminder at the 4-hour mark; reminder references original order ID; order status unchanged in Shopify sandbox

E07

Xero sales order reference not found for the Shopify order number

ShipStation returns valid label and cost; Xero sandbox has no matching sales order for the Shopify order number

Xero write step fails gracefully; Slack alert posted with order ID and shipping cost for manual Xero entry; all other agent steps complete normally

Shopify order is Fulfilled with tracking number; Klaviyo test send fires; Slack alert contains the exact shipping cost and order reference; Xero sandbox has no orphan entry

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: use only the designated internal test address (a real email inbox controlled by your team) for all production runs. Use a zero-cost test SKU that is mapped in both Shopify and Linnworks with a stock quantity of at least 5. Void the ShipStation label immediately after each test run so no carrier charge is incurred. Delete or cancel the Shopify test order and reverse the Xero line item after each case. Do not use a real customer email address or a live SKU that is active in current fulfilment.

FullSpec will confirm that all production API keys are correctly scoped before Phase 3 begins. The process owner should have view access to the Shopify admin, the Linnworks console, the ShipStation account, Klaviyo reporting, and the Xero general ledger during this phase so that each output can be verified independently.

Scenario

Input

Expected output

Pass criteria

T25

Full production flow: in-stock order traced end to end

Shopify paid order placed using internal test address and zero-cost test SKU; real production webhook fires

Order Routing Agent confirms stock in Linnworks; Label and Despatch Agent creates real ShipStation label; Shopify order marked Fulfilled with live tracking number; Klaviyo tracking email delivered to internal test inbox; Xero cost line item posted

All five downstream records verified in their live systems within 90 seconds of Shopify payment confirmation; tracking email received in internal test inbox; Xero line item amount matches ShipStation cost; label voided post-test

T26

Production exception: out-of-stock SKU routes to Slack correctly

Shopify paid order for a test SKU with Linnworks stock set to 0 in production

Order Routing Agent posts alert to the live fulfilment Slack channel with order ID, SKU, and Shopify order link; no label created; no Klaviyo email sent

Fulfilment Slack channel receives alert within 30 seconds; alert fields correct; ShipStation shows no new label; Shopify order remains Unfulfilled; Linnworks stock count restored to pre-test value after run

T27

Production error-log verification: all execution steps recorded correctly

Review Make scenario execution history for T25 and T26 immediately after those runs

Make execution log shows every module step, duration, input, and output for both runs with no silent errors or skipped modules

Make history shows complete execution for each run; no modules with status 'skipped' or 'ignored'; any warning-level entries reviewed and documented by FullSpec; log retained in the FullSpec confirmation record

T28

Production Xero cost entry accuracy across two successive orders

Two sequential test orders with different test SKU weights triggering different carrier services and costs

Two separate Xero line items posted, each with the correct shipping cost matching the ShipStation cost for that carrier service

Xero general ledger shows two distinct line items; amounts match ShipStation costs to the cent; sales order references match respective Shopify order numbers; both lines reversed after test

T29

Process owner review and approval

Process owner independently reviews outputs from T25 through T28 in Shopify, Linnworks, ShipStation, Klaviyo, and Xero; reviews Make execution logs provided by FullSpec

Process owner confirms all outputs are accurate and the automation meets the agreed acceptance criteria: order-to-label time under 90 seconds, tracking email delivered, Xero entries correct, exception alerts actionable

Process owner confirms acceptance in writing to support@gofullspec.com. FullSpec records the confirmation in the project log. This confirmation is the go-live gate. No separate signoff block is required.

Once T29 is confirmed, FullSpec will activate production monitoring for all three agents and schedule a 5-day parallel observation window before the manual process is fully retired. Contact support@gofullspec.com for any issues arising during or after the parallel run.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process