FS-DOC-06Quality
Test and QA Plan
3PL / Warehouse Coordination Automation
[YourCompany.com] · Fulfilment Department · Prepared by FullSpec · [Today's Date]
This document is the formal test and quality assurance plan for the 3PL / Warehouse Coordination automation. It defines every test case the FullSpec team will execute across three sequential phases: unit testing of individual agents, integration and edge-case testing of agent handoffs and failure scenarios, and a final end-to-end production run. No phase begins until every test case in the previous phase has reached a passing state. The process owner joins Phase 3 to verify outputs and record approval. All queries about this plan should be directed to support@gofullspec.com.
01Testing approach
Testing is structured across three sequential phases. Each phase must be fully passed before the next phase begins. Phase 1 validates individual agent logic in isolation. Phase 2 validates agent handoffs, end-to-end data flow between tools, and edge cases. Phase 3 executes a live production run with real credentials and requires the process owner to review and record approval.
1Phase 1 — Unit Testing
Each agent tested in isolation before integration
Scope
Individual agent logic: ShipBob data fetch and SKU normalisation, Cin7 reconciliation and classification, Cin7/Shopify update actions and Slack notifications
Environment
Sandbox only. ShipBob sandbox account, Cin7 sandbox/staging, Shopify development store, Slack test workspace
Credential rule
No production API keys may be used. All sandbox credentials stored in the automation platform's secret manager under the prefix SANDBOX_
Pass condition
All test cases T01 through T14 return the expected output with zero unhandled exceptions
Who runs it
FullSpec team only
2Phase 2 — Integration and Edge Case Testing
Agent handoffs and failure mode validation
Scope
Full agent chain data passing (3PL Fetch Agent to Reconciliation Agent to Update and Notification Agent), plus edge cases: duplicates, missing fields, unavailable tools, and human timeout scenarios
Environment
Sandbox only. Integration tests run against the same sandbox stack as Phase 1. No production data is used.
Credential rule
Sandbox credentials only. If a sandbox environment does not support a specific failure mode, a mock/stub response is injected at the integration layer.
Pass condition
All integration test cases T15 through T22 and all edge cases E01 through E07 return the expected output or correctly handled error state
Who runs it
FullSpec team only
3Phase 3 — End-to-End Production Test
Live credentials, real systems, process owner sign-off
Scope
Complete automation run from trigger to Google Sheets log using production credentials, traced against real ShipBob, Cin7, Shopify, Slack, and Google Sheets accounts
Environment
Production. A dedicated internal test SKU is used to avoid modifying live inventory counts for real products.
Credential rule
Production API keys used for the first time in this phase. Keys must be stored in the automation platform's secret manager under the prefix PROD_ before Phase 3 begins.
Pass condition
All test cases T23 through T27 pass; process owner reviews outputs and records approval in the FullSpec confirmation record
Who runs it
FullSpec team executes; process owner (Operations Manager or Operations Coordinator) joins to review outputs and record approval for T27
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
All Phase 1 tests must be executed using sandbox credentials exclusively. No production API keys, production ShipBob accounts, live Cin7 environments, or real Shopify stores may be connected during this phase. If a test case requires a payload that the sandbox does not produce natively, inject a fixture JSON file that matches the documented ShipBob API response schema.
Agent 1: 3PL Data Fetch and Normalisation Agent
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Scheduled trigger fires correctly
Daily cron expression fires at 06:00 local time
Workflow execution starts; ShipBob API call initiated within 30 seconds
Execution log shows trigger fired; first API request recorded with HTTP 200
T02
ShipBob webhook received (inbound ASN)
POST webhook payload from ShipBob: new inbound receipt event, SKU SB-1042
Workflow execution starts from webhook node; payload parsed correctly
SKU SB-1042 extracted from payload; normalisation step invoked
T03
Successful inventory data fetch
Valid ShipBob sandbox credentials; inventory endpoint returns 15 SKUs
All 15 SKUs retrieved; raw response stored in memory for normalisation
Response count matches 15; no HTTP error codes in execution log
T04
SKU normalisation against mapping table
ShipBob SKU 'SB-1042' maps to internal SKU 'INT-042' in the mapping table
Normalised dataset uses 'INT-042'; original ShipBob SKU retained in raw field
Output payload contains both sb_sku: 'SB-1042' and internal_sku: 'INT-042'
T05
SKU not found in mapping table
ShipBob returns SKU 'SB-9999' which has no entry in the mapping table
SKU flagged as unmapped; passed downstream with normalisation_status: 'unmapped'
Downstream agent receives record with normalisation_status: 'unmapped'; no crash
T06
ShipBob API returns HTTP 401 (invalid credentials)
Expired or invalid sandbox API token supplied
Workflow halts at fetch step; error logged with code AUTH_FAILURE_SHIPBOB
Error event written to execution log; no downstream agents triggered
T07
ShipBob API returns HTTP 429 (rate limit)
Mock response returns 429 with Retry-After: 60 header
Workflow pauses for 60 seconds then retries; succeeds on retry with valid response
Retry attempt recorded in log; final execution status is success after retry
Agent 2: Reconciliation and Discrepancy Agent
ID
Scenario
Input
Expected Output
Pass Criteria
T08
Cin7 inventory fetch succeeds
Normalised dataset of 15 SKUs; valid Cin7 sandbox credentials
Cin7 returns current stock levels for all 15 SKUs; stored for comparison
15 Cin7 records returned; no HTTP errors; all matched by internal_sku
T09
SKU classified as matched
ShipBob count: 100 units; Cin7 count: 100 units for SKU INT-042
SKU tagged classification: 'matched'; no alert generated
Output record shows classification: 'matched'; not routed to alert branch
T10
SKU classified as minor variance (below threshold)
ShipBob count: 98 units; Cin7 count: 100 units; threshold set to 5 units
SKU tagged classification: 'minor_variance'; auto-update flag set to true
Output record shows classification: 'minor_variance'; routed to auto-update branch
T11
SKU classified as major discrepancy (above threshold)
ShipBob count: 60 units; Cin7 count: 100 units; threshold set to 5 units
SKU tagged classification: 'major_discrepancy'; alert flag set to true
Output record shows classification: 'major_discrepancy'; routed to Slack alert branch
T12
Missing ASN detected
Purchase order PO-2201 expected at ShipBob but no confirmed ASN in ShipBob response
PO-2201 flagged with missing_asn: true; Slack alert payload generated
Output includes missing_asn: true; alert payload contains PO number and expected date
T13
Google Sheets reconciliation row appended
Classified dataset of 15 SKUs with mix of matched, minor, and major results
One summary row appended to the reconciliation sheet with run timestamp, counts, and open exceptions
Google Sheets API returns HTTP 200; row visible in sandbox sheet with correct column values
T14
Cin7 API unavailable (HTTP 503)
Cin7 sandbox returns HTTP 503 on inventory fetch call
Workflow halts reconciliation step; error logged with code CIN7_UNAVAILABLE; Slack alert sent to ops channel
Error code in execution log; Slack message received in sandbox workspace with failure notice
Agent 3: Update and Notification Agent
ID
Scenario
Input
Expected Output
Pass Criteria
T15 (Phase 1)
Cin7 stock level updated for minor variance SKU
SKU INT-042; classification: 'minor_variance'; ShipBob count: 98; Cin7 count: 100
Cin7 inventory adjusted to 98; adjustment note 'Auto-reconcile [timestamp]' written against record
Cin7 API returns HTTP 200; record shows updated quantity and adjustment note
T16 (Phase 1)
Shopify stock level synced after Cin7 update
SKU INT-042 updated to 98 in Cin7; Shopify variant ID mapped to INT-042
Shopify inventory level updated to 98 for the corresponding variant and location
Shopify API returns HTTP 200; variant inventory_quantity reads 98 in sandbox store
T17 (Phase 1)
Slack alert sent for major discrepancy
SKU INT-077; classification: 'major_discrepancy'; ShipBob: 40, Cin7: 100
Structured Slack message posted to #fulfilment-alerts with SKU, counts, and Cin7 link
Slack API returns HTTP 200; message visible in sandbox Slack channel with all required fields
T18 (Phase 1)
Daily summary Slack message sent
Reconciliation run complete: 15 SKUs processed, 1 major discrepancy, 2 minor variances, 1 missing ASN
Formatted daily summary posted to #fulfilment-ops Slack channel
Summary message received in sandbox channel; counts match the test dataset totals
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 integration tests confirm that data passes correctly between all three agents in sequence and that the full automated flow from trigger to Google Sheets log behaves as specified. Test IDs continue from the Phase 1 sequence.
ID
Scenario
Input
Expected Output
Pass Criteria
T19
Full agent chain: scheduled trigger to normalised dataset handoff
Cron trigger fires; ShipBob sandbox returns 15 SKUs; mapping table has 14 matches and 1 unmapped
3PL Fetch Agent outputs normalised dataset; Reconciliation Agent receives it without data loss
Reconciliation Agent input payload contains all 15 records; normalisation_status fields present on each
T20
Reconciliation Agent to Update Agent handoff for mixed classification set
Dataset: 12 matched, 2 minor variance, 1 major discrepancy
Update Agent receives only the 14 non-major records for update actions; 1 major discrepancy routed to Slack alert only
Cin7 update called 14 times; Slack alert called once for major discrepancy; no crossover
T21
End-to-end: webhook trigger through to Google Sheets log
ShipBob inbound ASN webhook fires for SKU INT-099; count matches Cin7
Full chain executes: fetch, normalise, reconcile (matched), update Cin7, update Shopify, append Google Sheets row
Google Sheets row present within 90 seconds of webhook receipt; all columns populated correctly
T22
Agent 1 to Agent 2 handoff with unmapped SKU in dataset
Dataset includes SKU SB-9999 with normalisation_status: 'unmapped' alongside 14 normal records
Reconciliation Agent skips SB-9999 for Cin7 comparison; logs it to Google Sheets as 'unmapped_sku'; processes remaining 14 normally
Execution log shows 14 Cin7 lookups; Google Sheets row contains unmapped_sku count of 1
Edge case tests cover conditions outside the normal flow that the automation must handle without crashing or producing silent incorrect outputs.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate SKU rows in ShipBob response
ShipBob API returns two rows for SKU INT-042 with different quantities (98 and 95)
Normalisation step deduplicates by taking the most recent record by timestamp; single row passed downstream
Downstream dataset contains exactly one record for INT-042; deduplication_flag: true in execution log
E02
ShipBob response contains SKU with null quantity field
ShipBob returns SKU INT-055 with quantity: null
Record flagged with data_quality_flag: 'null_quantity'; skipped for Cin7 update; included in Google Sheets log with flag
No Cin7 update called for INT-055; Google Sheets row shows null_quantity flag for that SKU
E03
Shopify API unavailable during stock sync
Cin7 update succeeds for INT-042; Shopify returns HTTP 503 on inventory update call
Error logged with code SHOPIFY_SYNC_FAILURE; Slack alert sent to #fulfilment-alerts; Cin7 update retained; Shopify sync retried after 5 minutes
Slack message received noting Shopify sync failure for INT-042; retry attempt visible in execution log
E04
Google Sheets API write fails
Reconciliation complete; Google Sheets API returns HTTP 403 (permission denied)
Error logged with code SHEETS_WRITE_FAILURE; Slack alert sent with log data inline so audit record is not lost
Slack message contains the full reconciliation summary inline; execution log records SHEETS_WRITE_FAILURE
E05
Missing ASN where PO was expected 48+ hours ago
Purchase order PO-2210 has expected receipt date 2 days prior; no ASN confirmed in ShipBob
Alert escalated with priority flag 'overdue_asn'; Slack message includes age of expected receipt in days
Slack message shows 'Overdue ASN: 2 days' for PO-2210; standard missing ASN cases show 'Missing ASN' only
E06
Cin7 returns a SKU not present in the ShipBob response
Cin7 has INT-088 in stock at 50 units; ShipBob response contains no record for INT-088
INT-088 flagged as 'not_in_3pl_response'; logged to Google Sheets with flag; no Cin7 update applied; no Slack alert (below threshold for this condition)
Google Sheets row for this run includes INT-088 with status 'not_in_3pl_response'; Cin7 quantity unchanged
E07
Operations coordinator does not action a major discrepancy Slack alert within 4 hours
Major discrepancy alert sent at 07:00; no acknowledgement or Cin7 update recorded by 11:00
Automation sends a follow-up Slack reminder to #fulfilment-alerts at the 4-hour mark referencing the original alert
Follow-up Slack message visible at 11:00 in sandbox workspace; original alert timestamp referenced in message
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: use a dedicated internal test SKU (agreed with the operations manager before this phase begins) for all production test runs. Do not use a SKU tied to live customer-facing stock. Any Cin7 and Shopify adjustments made during Phase 3 must be reversed or voided immediately after each test case completes. Confirm with the operations coordinator that Google Sheets log rows written during testing are clearly marked 'QA TEST RUN' in the notes column so they are excluded from finance reporting. Production Slack alerts during testing should be posted to a designated #qa-testing channel, not #fulfilment-alerts, to avoid confusing the live operations team.
Phase 3 test cases use live production credentials across all five integrated tools. The FullSpec team executes each case and documents the result. The process owner joins for T27, the final approval case. Test IDs continue from Phase 2.
ID
Scenario
Steps Covered
Expected Output
Pass Criteria
T23
Production scheduled trigger: full run, all SKUs matched
Trigger fires at scheduled time; ShipBob production API called; all test SKUs return counts matching Cin7
Full chain executes in production: fetch, normalise, reconcile (all matched), Cin7 no-change confirmed, Shopify no-change confirmed, Google Sheets row appended with run timestamp and zero discrepancies
Execution completes within 5 minutes; Google Sheets row present; Cin7 and Shopify stock unchanged; no Slack alerts generated for test SKU
T24
Production run: minor variance auto-update
Test SKU manually adjusted in ShipBob sandbox to create a 2-unit variance below threshold; production run triggered
Reconciliation classifies as minor_variance; Cin7 auto-updated; Shopify synced; Google Sheets row records adjustment; daily Slack summary includes variance count
Cin7 production record shows updated quantity with adjustment note; Shopify variant matches; Google Sheets row has classification 'minor_variance'; Slack summary count is correct. Adjustment voided in Cin7 and Shopify after test.
T25
Production run: major discrepancy Slack alert
Test SKU adjusted in ShipBob to create a variance above the agreed threshold; production run triggered
Reconciliation classifies as major_discrepancy; no Cin7 or Shopify update applied; structured Slack alert posted to #qa-testing channel with SKU, expected count, actual count, and Cin7 link
Slack alert received in #qa-testing within 2 minutes of run completion; Cin7 and Shopify quantities unchanged for test SKU; Google Sheets row shows classification 'major_discrepancy'
T26
Error log and execution audit verification
Review execution logs from T23, T24, and T25 in the automation platform
All three runs show complete execution logs with timestamps, API call outcomes, classification decisions, and any error codes; no silent failures or empty log entries
Execution log entries present for all three prior cases; each log contains: trigger_time, shipbob_call_status, cin7_call_status, classification_summary, update_actions, slack_status, sheets_status
T27
Process owner review and approval
Process owner (Operations Manager or Operations Coordinator) is walked through the Google Sheets reconciliation log, the Slack summary message, the Cin7 adjustment note, and the execution audit log from T23 through T26
Process owner confirms: outputs are accurate, Slack formatting is clear and actionable, Google Sheets log is complete and audit-ready, and the automation is ready to replace the manual process
Process owner approval recorded in the FullSpec confirmation record. No separate sign-off block is required. Phase 3 is complete when FullSpec logs the approval against this test case ID in the project record.
Once T27 is approved, the FullSpec team will schedule the parallel run week as defined in the Launch Plan and coordinate the cutover date with the operations manager. Any issues discovered during the parallel run are treated as regression cases and re-tested against the relevant Phase 1, 2, or 3 test case ID before go-live is confirmed. Contact support@gofullspec.com for all post-approval queries.
Test and QA PlanPage 4 of 4