FS-DOC-06Quality
Test and QA Plan
Vendor & Subscription Renewal Tracking
[YourCompany.com] · IT Department · Prepared by FullSpec · [Today's Date]
This document defines the full quality assurance programme for the Vendor and Subscription Renewal Tracking automation. It is written for the FullSpec build team and covers every test case across three sequential phases: unit testing of individual agents, integration and edge-case testing across agent handoffs, and end-to-end production verification. No phase begins until the previous phase has fully passed. The process owner joins Phase 3 to review the live run and record formal sign-off within the FullSpec confirmation system.
01Testing approach
Testing is structured across three phases that must be completed in sequence. Phase 1 isolates each agent and validates its logic against sandbox data. Phase 2 tests the handoffs between agents and covers edge cases including missing data, duplicate records, and unavailable tools. Phase 3 runs a single traced end-to-end flow in the production environment with real credentials and live register data. A phase may not begin until every test case in the preceding phase has achieved a passing status. Any failing test case blocks progression and must be resolved and re-tested before the next phase opens.
1Phase 1: Unit Testing
Each agent tested in isolation against sandbox credentials
Scope
Individual agent logic: Renewal Triage Agent, Stakeholder Notification Agent, Finance Record Agent — each tested independently
Environment
Sandbox only. No production credentials, no live Google Sheets register, no real Xero supplier contacts
Credential rule
Sandbox OAuth tokens for Google Sheets (test sheet), Slack (test workspace channel), Gmail (test sender address), Xero (sandbox org), and Notion (test workspace) must be used exclusively. Production credentials must not be loaded during Phase 1
Pass condition
All test cases T01 through T13 return the expected output with no unhandled errors
Who runs it
FullSpec team only. Process owner is not present for Phase 1
2Phase 2: Integration and Edge Case Testing
Agent handoffs and failure-mode scenarios validated in sandbox
Scope
Full agent-to-agent handoffs across the complete automated flow; edge cases covering duplicates, missing fields, tool unavailability, and human timeout
Environment
Sandbox only. Synthetic register data with intentional anomalies injected to trigger each edge case
Credential rule
Same sandbox credential set as Phase 1. No production tokens are permitted. Xero sandbox org must be used for all bill-creation tests
Pass condition
All integration test cases T14 through T19 pass, and all edge-case test cases E01 through E06 are handled gracefully with correct fallback behaviour and no silent failures
Who runs it
FullSpec team only. Process owner is not present for Phase 2
3Phase 3: End-to-End Production Test
Single full run traced in the live environment with process owner present
Scope
Complete automated flow from daily trigger through to Xero draft bill creation, using the live production register and real credentials
Environment
Production. Live Google Sheets register, production Slack workspace, production Gmail sender, Xero live org (draft bills only, not submitted), and live Notion workspace
Credential rule
Production OAuth tokens loaded for all five integrations. All test contract records inserted into the register must be clearly labelled TEST and voided or deleted after the run
Pass condition
All test cases T20 through T23 pass, including the final process owner review and approval case (T23)
Who runs it
FullSpec team runs the automation; the process owner joins to observe the live run, verify outputs, and record approval in the FullSpec confirmation system
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
All Phase 1 tests must be executed using sandbox credentials only. Do not load production OAuth tokens for Google Sheets, Slack, Gmail, Xero, or Notion at any point during this phase. Use the designated test Google Sheet (separate from the live register), the FullSpec test Slack workspace, the test Gmail sender address (test-renewals@[YourCompany.com]), the Xero sandbox organisation, and the Notion test workspace. Injecting real vendor data or production credentials during unit testing risks live notifications being sent to real contract owners and finance staff.
Renewal Triage Agent — Build complexity: Moderate. Estimated build time: 10 hours. Trigger: scheduled daily run reads all rows from the Google Sheets register and the agent categorises contracts by alert window.
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: contract exactly 60 days from renewal
Register row with renewal_date = today + 60 days, decision_logged = FALSE
Row categorised as urgency: HIGH, alert_window: 60-day, flagged for notification
Output JSON contains urgency=HIGH, window=60, decision_required=true
T02
Happy path: contract exactly 30 days from renewal
Register row with renewal_date = today + 30 days, decision_logged = FALSE
Row categorised as urgency: CRITICAL, alert_window: 30-day, flagged for notification
Output JSON contains urgency=CRITICAL, window=30, decision_required=true
T03
Happy path: contract exactly 7 days from renewal
Register row with renewal_date = today + 7 days, decision_logged = FALSE
Row categorised as urgency: IMMEDIATE, alert_window: 7-day, additional flag: act_now=true
Output JSON contains urgency=IMMEDIATE, act_now=true
T04
Decision already logged: contract in window but actioned
Register row with renewal_date = today + 15 days, decision_logged = TRUE
Row excluded from notification list; logged as already_actioned=true
Row absent from output notification array; already_actioned flag present in audit log
T05
Contract outside all alert windows
Register row with renewal_date = today + 90 days, decision_logged = FALSE
Row excluded from all output lists; no flag raised
Row absent from notification array and urgency output
T06
Failure: notice period shorter than alert window
Register row with renewal_date = today + 5 days, notice_period_days = 14
Row flagged as urgency: IMMEDIATE with additional flag: notice_window_breach=true
Output contains notice_window_breach=true; row appears at top of output list
T07
Failure: renewal date already passed with no decision recorded
Register row with renewal_date = today - 2 days, decision_logged = FALSE
Row flagged as urgency: OVERDUE, auto_charge_risk=true; escalation alert queued
Output contains urgency=OVERDUE, auto_charge_risk=true
T08
Failure: renewal_date field empty
Register row with renewal_date = NULL
Row skipped from triage; error logged as missing_renewal_date; row ID recorded in error list
Error log contains row reference and error_type=missing_renewal_date; no unhandled exception
Stakeholder Notification Agent — Build complexity: Moderate. Estimated build time: 8 hours. Trigger: receives categorised contract list from the Renewal Triage Agent and routes Slack and Gmail messages to the correct recipients.
ID
Scenario
Input
Expected Output
Pass Criteria
T09
Happy path: Slack alert sent to contract owner
Triage output with one contract, urgency=HIGH, owner_slack_id=@test_owner, vendor=VendorA, cost=$1,200/year, renewal_date=+30 days
Slack message posted to test channel containing vendor name, cost, renewal date, and decision prompt
Slack API returns 200; message visible in test channel with all four data fields present
T10
Happy path: Gmail finance summary sent
Triage output with three contracts due in next 30 days, finance_email=test-finance@[YourCompany.com]
Single Gmail sent to finance address listing all three contracts with vendor, amount, and due date
Gmail send API returns 200; email received at test address with correct contract count and amounts
T11
Failure: owner_slack_id missing from register row
Triage output with contract flagged for notification, owner_slack_id = NULL
Slack message skipped; error logged as missing_slack_id; fallback Gmail sent to process owner address
Error log contains missing_slack_id; fallback Gmail delivered to process owner test address
T12
Failure: Slack API rate limit exceeded
Burst of 10 simultaneous Slack messages triggered
Retry logic activates with 2-second backoff; messages queued and delivered within 30 seconds
All 10 messages delivered; no messages lost; retry count visible in execution log
Finance Record Agent — Build complexity: Moderate. Estimated build time: 10 hours. Trigger: fires when a renewal decision is confirmed (decision_logged = RENEW) in the Google Sheets register; creates a draft bill in Xero.
ID
Scenario
Input
Expected Output
Pass Criteria
T13
Happy path: draft Xero bill created for confirmed renewal
Google Sheets row with decision=RENEW, vendor=VendorA, amount=$1,200, due_date=+30 days; VendorA exists as supplier contact in Xero sandbox
Draft bill created in Xero sandbox with correct vendor, amount, due date, and status=DRAFT
Xero API returns 200; bill visible in sandbox with status=DRAFT; no payment initiated
T14
Failure: vendor not found as Xero supplier contact
Google Sheets row with decision=RENEW, vendor=NewVendorZ; NewVendorZ does not exist in Xero sandbox
Bill creation skipped; error logged as xero_supplier_not_found; alert queued to process owner to create supplier record manually
Error log contains xero_supplier_not_found; no bill created; alert delivered to process owner test address
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 validates the full handoff sequence between agents and confirms that data passed from one agent to the next arrives correctly formatted and complete. All tests remain in the sandbox environment with the same credential set used in Phase 1. Test IDs continue the sequential numbering from Phase 1.
ID
Scenario
Agents Involved
Expected Handoff Behaviour
Pass Criteria
T15
Triage Agent to Notification Agent: single contract in 30-day window
Renewal Triage Agent → Stakeholder Notification Agent
Triage output array passes to Notification Agent; Slack message sent to owner and Gmail sent to finance within one execution cycle
Both messages delivered; execution log shows clean handoff with no data loss between agents
T16
Triage Agent to Notification Agent: multiple contracts across different urgency levels
Renewal Triage Agent → Stakeholder Notification Agent
Each contract routed to the correct owner Slack channel; single consolidated Gmail to finance lists all contracts sorted by urgency descending
All owner Slack messages contain correct individual contract data; Gmail lists contracts in urgency order
T17
Full flow: triage to notification to register write to Notion update
Renewal Triage Agent → Stakeholder Notification Agent → Google Sheets write → Notion update
After owner logs decision, Google Sheets row updated with decision, decision_date, and actioning owner; Notion vendor page updated with new renewal date and confirmed cost
Google Sheets row contains correct decision value and timestamp; Notion page reflects updated renewal date and decision within same execution cycle
T18
Full flow: register write triggers Finance Record Agent
Google Sheets write (decision=RENEW) → Finance Record Agent → Xero sandbox
Finance Record Agent detects RENEW decision in register, creates draft Xero bill with correct vendor, amount, and due date
Xero sandbox bill created with status=DRAFT; no submission or payment action taken; finance alert email delivered
T19
Cancel decision: Xero bill not created
Google Sheets write (decision=CANCEL) → Finance Record Agent
Finance Record Agent detects CANCEL decision; no Xero bill created; Notion vendor page updated with decision=CANCEL and cancellation date
No bill record in Xero sandbox; Notion page updated correctly; execution log confirms CANCEL branch followed
Edge case tests below cover scenarios outside the normal flow: duplicate register rows, missing required fields, tool unavailability, and human decision timeout. Each is assigned an E-series ID to distinguish it from the sequential test case run.
ID
Edge Case
Injected Condition
Expected Behaviour
Pass Criteria
E01
Duplicate contract rows in register
Two Google Sheets rows with identical vendor name and renewal date but different row IDs
Triage Agent flags duplicate; only the most recently updated row is processed; error log records duplicate_row with both row IDs
Single notification sent for the vendor; error log contains duplicate_row entry with both row references
E02
Contract owner field missing from register row
Register row with renewal in 30-day window, owner_name = NULL, owner_slack_id = NULL
Slack notification skipped; fallback Gmail sent to process owner; error logged as missing_owner; row flagged in register
Fallback Gmail delivered; error log contains missing_owner; no unhandled exception
E03
Cost field missing or non-numeric
Register row with cost = blank or cost = 'TBC'
Row included in triage output with cost marked as unconfirmed; Slack message sent with cost displayed as 'Amount not confirmed'; Xero bill not created until cost is resolved
Slack message contains 'Amount not confirmed' text; no Xero bill created; error log records invalid_cost_value
E04
Notion API unavailable during vendor page update
Notion API returns 503 during Notion update step
Automation retries up to three times with 5-second backoff; if all retries fail, update is queued and error logged as notion_update_failed; Google Sheets write and Xero bill are not blocked
Google Sheets and Xero steps complete successfully; error log contains notion_update_failed with retry count; queued retry visible in execution log
E05
Xero API returns authentication error
Xero sandbox OAuth token expired or invalid during bill creation
Finance Record Agent catches 401 response; execution halts for Xero step only; alert sent to FullSpec support queue; no silent failure; other agents unaffected
Error log contains xero_auth_failure with HTTP 401; support alert triggered; Google Sheets and Notion steps unaffected
E06
Human decision not logged within 7 days of 7-day alert
Register row remains with decision_logged = FALSE for 7 days after 7-day alert was sent
Triage Agent re-flags row on next daily run as urgency=OVERDUE; escalation Slack message sent to process owner; Gmail escalation sent to finance contact
Escalation Slack and Gmail both delivered; row re-appears at top of urgency output list with urgency=OVERDUE
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: All contract records inserted into the live Google Sheets register for this test must use an internal test address (test-renewals@[YourCompany.com]) as the owner email and a dedicated internal Slack channel (#renewals-qa-test) as the notification destination. Do not use a real contract owner's Slack ID or personal email during the production run. Any Xero draft bills created during Phase 3 must be voided in the Xero production org immediately after the test is confirmed as passing. The test register rows must be clearly labelled 'TEST - DELETE POST QA' in the vendor name field and removed from the register on the same day as the test run. FullSpec will confirm deletion in the post-run checklist.
Phase 3 uses the live production environment and real credentials for all five integrations. The FullSpec team runs the automation while the process owner observes the full execution. Every step is traced through the execution log and each output is verified against the expected result before the next step is confirmed. Test IDs continue the sequential numbering from Phase 2.
ID
Scenario
Production Action
Expected Output
Pass Criteria
T20
Full production run: daily trigger fires and register is read
Scheduled daily trigger activates at 7:00 AM; automation reads all active rows from the live production Google Sheets register
All active contract rows returned; row count matches total rows in register; execution log confirms read timestamp and row count
Execution log shows successful read with correct row count; no rows skipped without an error entry
T21
Production run: triage identifies test contract and routes notification
Test register row (vendor='TEST - DELETE POST QA', renewal_date = today + 30, decision_logged = FALSE) is processed by Renewal Triage Agent; Stakeholder Notification Agent fires
Slack message posted to #renewals-qa-test channel with correct vendor label, cost, renewal date, and decision prompt; Gmail sent to test-renewals@[YourCompany.com] with 30-day charge summary
Slack message visible in #renewals-qa-test with all required fields; Gmail received at test address within 2 minutes of trigger; both messages contain correct data from register row
T22
Production run: decision logged triggers register write, Notion update, and Xero draft bill
Process owner logs decision=RENEW manually in the test register row; Finance Record Agent fires; Google Sheets write and Notion update execute
Google Sheets test row updated with decision=RENEW, decision_date=today, actioning_owner confirmed; Notion test vendor page updated with new renewal date; Xero draft bill created with status=DRAFT, vendor='TEST - DELETE POST QA', correct amount and due date
Google Sheets row reflects all three written fields; Notion page updated correctly; Xero bill visible in production org with status=DRAFT; execution log shows all three write steps completed without error
T23
Production run: error log verification and process owner review and approval
FullSpec team presents execution log to the process owner, confirming: zero unhandled errors, all expected steps completed, all outputs match the register data, Xero bill voided, and test rows removed from register
Process owner confirms all outputs are correct and consistent with the live register; process owner records approval in the FullSpec confirmation system; FullSpec logs the approval timestamp and closes the QA sign-off record
Process owner approval recorded in FullSpec confirmation system with timestamp. Execution log archived with zero unhandled errors. Test records deleted from register and Xero bill voided. QA plan status set to PASSED.
Once T23 is marked as passing and the process owner approval is recorded in the FullSpec confirmation system, the QA programme for this automation is complete. FullSpec will archive the full execution log, the test credential inventory, and the approved test results. The automation is cleared for live operation. Any issues discovered post-launch should be reported to support@gofullspec.com with the execution log reference included.
Test and QA PlanPage 4 of 4