FS-DOC-06Quality
Test and QA Plan
Accounts Payable Management
[YourCompany.com] · Finance Department · Prepared by FullSpec · [Today's Date]
This document defines the complete testing programme for the Accounts Payable Management automation. It is written for the FullSpec build team and covers all three testing phases: unit tests for each individual agent, integration and edge-case tests for inter-agent handoffs and failure scenarios, and a final end-to-end production test conducted with live credentials. No phase begins until every test case in the preceding phase has fully passed. The process owner joins Phase 3 and records their approval as the final test case.
01Testing approach
Testing runs in three sequential phases. Each phase must reach a full pass before the next phase is initiated. Phase 1 validates each agent in isolation using sandbox credentials. Phase 2 validates the handoffs between agents and covers edge cases including missing data, tool unavailability, and human timeout scenarios. Phase 3 runs a controlled end-to-end flow in the production environment with real credentials, real invoices, and process owner oversight.
1Phase 1: Unit Testing
Each agent tested in isolation against sandbox credentials
Scope
Invoice Intake Agent, Coding and Approval Agent, and Payment and Remittance Agent tested independently. No cross-agent data flow tested in this phase.
Environment
Sandbox only. Dext demo account, Xero demo company, Slack test workspace, Gmail test alias.
Credential rule
No production API keys, OAuth tokens, or live Xero organisation IDs may be used in Phase 1.
Pass condition
All test cases T01 through T15 return expected outputs with no unhandled errors.
Who runs it
FullSpec team. Process owner is not required for Phase 1.
2Phase 2: Integration and Edge Case Testing
Agent handoffs, tool failures, and boundary conditions
Scope
End-to-end data flow between all three agents, plus isolated edge-case scenarios covering duplicates, missing fields, tool unavailability, and approval timeouts.
Environment
Sandbox for agent handoff tests (T16 through T22). Simulated failure states created by the FullSpec team; no production tooling involved.
Credential rule
Sandbox credentials only. Rate-limit and webhook failure simulations are triggered programmatically within the test environment.
Pass condition
All integration cases T16 through T22 pass, and all edge cases E01 through E06 produce the correct exception routing or error handling behaviour.
Who runs it
FullSpec team. Process owner is not required for Phase 2.
3Phase 3: End-to-End Production Test
Live credentials, real invoice flow, process owner sign-off
Scope
Full invoice lifecycle from Dext detection through to Xero bill posting, payment scheduling, and Gmail remittance, run against the live production environment.
Environment
Production. Live Dext organisation, live Xero company, live Slack workspace, live Gmail account.
Credential rule
Production OAuth tokens and API keys are used. All test records must be voided or deleted in Xero after each run. Test remittances must be sent to an internal alias only.
Pass condition
All production cases T23 through T27 pass. Final case T27 records process owner approval in the FullSpec confirmation system.
Who runs it
FullSpec team executes T23 through T26. The process owner participates in T27 and records their review and approval.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only. Do not connect any production Dext organisation, Xero company, Slack workspace, or Gmail account during Phase 1. If a test requires a real API response, use the Dext demo account and Xero demo company exclusively. All test invoice PDFs must be synthetic documents with no real supplier names or payment details.
Invoice Intake Agent — estimated build complexity: Moderate. The following cases validate extraction accuracy, duplicate detection logic, and failure handling for the intake layer.
ID
Scenario
Input
Expected output
Pass criteria
T01
Happy path: clean PDF invoice via Dext
Well-formatted supplier PDF with supplier name, invoice number, date, due date, and two line items
Structured record with all five fields extracted; duplicate-clear flag set to true; record passed to Coding and Approval Agent
All five fields populated with correct values; no extraction errors logged; downstream agent receives payload within 10 seconds
T02
Happy path: invoice arrives via Gmail AP inbox
PDF attachment on inbound email to designated AP alias
Email detected; attachment extracted and submitted to Dext OCR pipeline; structured record produced as per T01
Email trigger fires within 60 seconds of arrival; attachment routed correctly; output matches T01 structure
T03
Duplicate invoice detected
Invoice with same supplier name and invoice number as an existing Xero draft bill
Duplicate flag set to true; record NOT forwarded to coding agent; Slack alert posted to bookkeeper exception channel
Xero query returns matching bill ID; Slack message contains supplier name, invoice number, and a link to the existing Xero bill
T04
OCR extraction failure: unreadable PDF
Scanned image PDF with less than 40% text confidence score returned by Dext
Extraction flagged as failed; record routed to bookkeeper exception queue with original file attached
No partial data is forwarded to downstream agent; exception record contains original PDF and a human-readable error reason
T05
Missing invoice number field
Valid PDF but invoice number field is blank or absent
Record created with invoice number set to null; agent routes invoice to exception queue rather than proceeding
Exception flag is set; bookkeeper notified via Slack; no bill created in Xero
T06
Dext webhook delivery failure
Dext document event webhook returns HTTP 500
Automation platform retries webhook delivery up to three times with exponential backoff; after third failure, error is logged and bookkeeper is alerted
Retry log shows three attempts; final alert sent to bookkeeper channel; no silent failure
Coding and Approval Agent — estimated build complexity: Moderate. The following cases validate account-code suggestion logic, Slack approval routing, and reminder behaviour.
ID
Scenario
Input
Expected output
Pass criteria
T07
Happy path: known supplier with coding history
Duplicate-clear invoice record for a supplier with at least three prior Xero transactions
Account code suggested matches the most frequently used code for that supplier; Slack approval message posted with supplier, amount, due date, and suggested code
Suggested code matches historical majority code; Slack message formatted correctly with approve and query buttons present
T08
Happy path: approver approves via Slack
Slack approval message posted; approver clicks Approve within the configured window
Approval event captured; invoice record marked as approved; payload forwarded to Payment and Remittance Agent
Downstream agent receives approved record within 15 seconds of Slack interaction; Slack message updated to show Approved status
T09
New supplier with no coding history
Invoice from a supplier name not present in Xero transaction history
Agent applies best-guess code based on line item description keywords; Slack message flagged as unconfirmed coding requiring approver review
Slack message clearly indicates suggested code is unverified; approver can override before approving
T10
Approver queries invoice in Slack
Approver clicks Query rather than Approve
Invoice record flagged as queried; bookkeeper receives Slack notification with approver's reason; record held, not forwarded
No bill created in Xero; bookkeeper notification contains invoice reference and query reason
T11
Approval reminder: no response within configured window
Approval message posted; no Slack interaction within the configured reminder interval (default 24 hours)
Automated reminder posted as a thread reply; escalation message sent after second interval if still no response
First reminder fires at exactly the configured interval; escalation fires at second interval; no bill posted until approval is received
T12
Slack API unavailable during approval post
Approval message post returns HTTP 503 from Slack API
Automation retries three times; on persistent failure, email fallback sent to approver's address and bookkeeper alerted
Email fallback delivered; error logged with timestamp; no approval message silently dropped
Payment and Remittance Agent — estimated build complexity: Simple. The following cases validate bill creation in Xero, payment scheduling, and remittance delivery via Gmail.
ID
Scenario
Input
Expected output
Pass criteria
T13
Happy path: bill created and payment scheduled
Approved invoice record with supplier, amount, account code, due date, and PDF attachment
Bill created in Xero with all fields populated and PDF linked; payment date set per configured payment run schedule; bill status is AUTHORISED
Xero API returns 200 with bill ID; bill visible in Xero demo company with correct fields; payment date matches schedule logic
T14
Happy path: remittance email sent to supplier
Payment record confirmed in Xero; supplier email address on file in Xero contact record
Remittance advice email sent via Gmail to supplier address containing invoice reference, amount paid, and payment date
Gmail send API returns 200; email content contains correct invoice reference and amount; supplier address matches Xero contact
T15
Xero bill creation failure: duplicate bill ID returned
Approved record submitted to Xero bills API; Xero returns 400 with DuplicateInvoiceNumber error code
Agent catches error; bill not created; bookkeeper alerted via Slack with invoice reference and error detail; record held for manual review
No duplicate bill in Xero; Slack alert delivered with actionable detail; error logged with Xero response body
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 validates the data handoffs between agents and tests the system's behaviour under abnormal conditions. Integration tests confirm that the full three-agent chain produces the correct output when each agent performs as expected. Edge case tests confirm that failure modes, boundary conditions, and human inaction are handled safely without data loss or silent errors.
Integration tests: agent handoffs
ID
Scenario
Input
Expected output
Pass criteria
T16
Full chain: Intake Agent to Coding Agent handoff
Clean invoice record produced by Invoice Intake Agent (T01 output)
Coding and Approval Agent receives the complete structured payload; account code suggestion triggered immediately; Slack message posted
No field loss between agents; Slack message appears within 30 seconds of intake completion; all extracted fields present in Slack message
T17
Full chain: Coding Agent to Payment Agent handoff
Approved invoice record produced by Coding and Approval Agent (T08 output)
Payment and Remittance Agent receives approved record; bill created in Xero demo company; payment scheduled; remittance sent
Bill ID returned from Xero; payment date set; Gmail send confirmed; end-to-end latency under 5 minutes from Slack approval to bill creation
T18
Exception path: duplicate flag triggers correctly mid-chain
Second copy of an invoice already processed in T01 submitted via Gmail
Intake Agent raises duplicate flag; Coding Agent does not receive payload; bookkeeper receives Slack alert with both invoice references
Coding Agent trigger does not fire; Slack alert contains original bill ID and new submission reference; no double bill in Xero
T19
Exception path: queried invoice held and not posted
Approver queries invoice in Slack (T10 scenario) while Payment Agent is active
Payment Agent does not receive any payload for a queried invoice; Xero bill is not created until manual resolution clears the query
Zero bills created in Xero for queried invoices; bookkeeper alert contains clear resolution instructions
T20
Invoice due within 48 hours: urgent flag applied
Approved invoice with due date 36 hours from current timestamp
Payment Agent sets urgent flag on Xero bill; Slack notification sent to bookkeeper with due date and amount highlighted
Xero bill due date correct; urgent Slack notification delivered before bill is scheduled to the next standard payment run
T21
Coding Agent failure: Xero history query timeout
Xero API returns 504 Gateway Timeout when historical transactions are queried for code suggestion
Agent retries up to three times; on persistent timeout, invoice is held and bookkeeper is notified to assign code manually; no partial code applied
Retry log shows three attempts; no code applied automatically; bookkeeper notification delivered; invoice remains in held state
T22
Payment Agent failure: Xero payment scheduling API error
Xero payments endpoint returns 422 Unprocessable Entity during payment scheduling
Bill is created but payment is not scheduled; bookkeeper alerted via Slack with bill ID and error detail for manual scheduling
Bill exists in Xero with AUTHORISED status; payment not scheduled; Slack alert includes Xero bill URL and error reason
Edge case tests
ID
Scenario
Input
Expected output
Pass criteria
E01
Duplicate invoice submitted within the same processing run
Two identical PDFs submitted to the Dext queue within 60 seconds of each other
First submission processed normally; second submission detected as duplicate by Xero bill query; bookkeeper alerted; second invoice not posted
Only one bill exists in Xero for the supplier and invoice number combination; duplicate alert references both submission timestamps
E02
Invoice with no supplier email address in Xero contact record
Approved, posted bill where the Xero contact has no email field populated
Payment and Remittance Agent detects missing email; remittance step is skipped; bookkeeper alerted to add supplier email and trigger remittance manually
No Gmail send attempted; bookkeeper alert contains supplier name, invoice reference, and Xero contact edit link
E03
Dext unavailable: document capture tool returns 503
Dext API returns 503 Service Unavailable for a document extraction request
Automation retries after a configurable backoff period (default 15 minutes); after three retries, bookkeeper alerted with original file held in queue
Three retry attempts logged with timestamps; bookkeeper alert delivered; invoice file not lost; no silent discard
E04
Approver does not respond after two reminder cycles
Approval message posted in Slack; no interaction after two full reminder cycles
After second reminder, an escalation message is sent to the finance manager; invoice remains held; no bill created in Xero
Escalation Slack message delivered to finance manager; invoice still in held state; audit log entry created for each reminder and escalation
E05
Invoice in foreign currency (not the Xero base currency)
PDF invoice denominated in GBP submitted to a USD-base-currency Xero organisation
Extraction captures currency field; Coding Agent flags invoice as requiring manual currency confirmation; Slack message includes currency code and equivalent amount at current rate
No bill created without currency confirmation; Slack message clearly shows foreign currency flag; bookkeeper can confirm or override rate before approving
E06
Credit note received instead of invoice
Dext extracts a document with a negative total amount
Agent identifies negative total as a potential credit note; routes to bookkeeper exception queue rather than the standard approval flow
Standard approval flow not triggered; bookkeeper exception message includes extracted total, supplier name, and document type indicator
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: all test invoices must be sent to an internal test email alias, not a real supplier address. Any bills created in the live Xero organisation during this phase must be voided immediately after the test run is confirmed as passing. Do not schedule any real payments during Phase 3 testing. Confirm with the process owner that the live Slack approval channel is set to a private test channel for the duration of Phase 3 to avoid live approvers receiving test messages.
Phase 3 runs the complete production flow using live credentials across Dext, Xero, Slack, and Gmail. The FullSpec team executes each case and records the outcome. The process owner reviews the full run log before recording their approval in T27, which is the final acceptance gate for go-live.
ID
Scenario
Input
Expected output
Pass criteria
T23
Full production run: clean invoice from Dext to posted Xero bill
Synthetic test invoice PDF submitted via live Dext organisation (supplier name: TEST SUPPLIER, invoice number: TEST-001, amount: $100.00)
Invoice extracted by Dext; duplicate check passes in live Xero organisation; account code suggested; Slack approval message posted to test channel; on approval, bill created in live Xero with status AUTHORISED; payment scheduled; remittance sent to internal alias
Bill TEST-001 visible in live Xero with correct supplier, amount, account code, and AUTHORISED status; payment date set; Gmail send confirmed to internal alias; total elapsed time from Dext detection to bill creation under 10 minutes
T24
Production duplicate detection on live Xero data
Second submission of TEST-001 from the same supplier immediately after T23 completes
Duplicate detected against the live Xero bill created in T23; Slack alert posted to bookkeeper channel; no second bill created
Zero additional bills for TEST-001 in live Xero; Slack alert references the existing bill ID; test bill from T23 voided after confirmation
T25
Production approval timeout and reminder in live Slack
Test invoice submitted; approval message posted to test Slack channel; no interaction simulated for the configured reminder interval
Reminder posted as thread reply at the configured interval; finance manager escalation triggered after second interval; invoice remains in held state
Reminder and escalation messages visible in live Slack test channel with correct timestamps; no bill created in Xero during hold period
T26
Error log verification: confirm all Phase 3 events are captured
Review automation platform run logs for T23 through T25
Each workflow run shows a complete audit trail: trigger timestamp, each step with input and output payloads, any retry events, and a final status (success or exception)
FullSpec team confirms log entries are present and accurate for every step across T23, T24, and T25; no missing or truncated log entries; error states correctly labelled in the run log
T27
Process owner review and approval: production go-live acceptance
Process owner reviews the complete Phase 3 run log, the voided test records in Xero, the Slack message history from the test channel, and the remittance email delivered to the internal alias
Process owner confirms all outputs match expectations, confirms exception handling is understood, and records approval for production go-live in the FullSpec confirmation system
Process owner approval recorded in the FullSpec platform with name, date, and confirmation that all Phase 3 cases were reviewed. This entry serves as the formal go-live acceptance. No separate sign-off document is required.
Once T27 is recorded, the FullSpec team will void all test bills in Xero, restore the live Slack approval channel to its production routing, and confirm the automation is cleared for live invoice processing. Contact support@gofullspec.com with the process name and your organisation ID to complete the go-live handover.
Test and QA PlanPage 4 of 4